Undetected state gap. Can't continue.
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
MySQL patches by Codership | Status tracked in 5.6 | |||||
5.5 |
Fix Committed
|
High
|
Teemu Ollakka | |||
5.6 |
Fix Released
|
High
|
Teemu Ollakka | |||
Percona XtraDB Cluster moved to https://jira.percona.com/projects/PXC | Status tracked in 5.6 | |||||
5.5 |
Fix Released
|
High
|
Unassigned | |||
5.6 |
Fix Released
|
High
|
Unassigned |
Bug Description
Nod that had been previously part of the cluster was started so that it first entered into non-primary component. For some reason local state was cleared on mysqld side, leading to undetected state gap error.
2014-06-26 10:49:37 6056 [Note] WSREP: Read nil XID from storage engines, skippi
ng position init
2014-06-26 10:49:37 6056 [Note] WSREP: wsrep_load(): loading provider library '/
home/vagrant/
2014-06-26 10:49:38 6056 [Note] WSREP: wsrep_load(): Galera 3.6dev(rXXXX) by Cod
ership Oy <email address hidden> loaded successfully.
2014-06-26 10:49:38 6056 [Note] WSREP: CRC-32C: using hardware acceleration.
2014-06-26 10:49:38 6056 [Note] WSREP: Found saved state: 2a7f5131-
a-e6e6e4d4557c:0
2014-06-26 10:49:38 6056 [Note] WSREP: Passing config to GCS: base_host = 192.16
8.17.11; base_port = 4567; cert.log_conflicts = no; debug = no; evs.debug_log_ma
sk = 0x4; evs.inactive_
info_log_mask = 0x4; evs.join_
3; evs.send_window = 4; evs.stats_
5S; evs.user_
e/vagrant/
= 0; gcache.name = /home/vagrant/
age_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.
fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_th
rottle = 0.25; gcs.recv_
t = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announ
ce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb =
false; pc.npvo
2014-06-26 10:49:38 6056 [Note] WSREP: Service thread queue flushed.
2014-06-26 10:49:38 6056 [Note] WSREP: Assign initial position for certification
: 0, protocol version: -1
2014-06-26 10:49:38 6056 [Note] WSREP: wsrep_sst_grab()
2014-06-26 10:49:38 6056 [Note] WSREP: Start replication
2014-06-26 10:49:38 6056 [Note] WSREP: Setting initial position to 2a7f5131-fc61
-11e3-809a-
2014-06-26 10:49:38 6056 [Note] WSREP: protonet asio version 0
2014-06-26 10:49:38 6056 [Note] WSREP: Using CRC-32C (optimized) for message che
cksums.
2014-06-26 10:49:38 6056 [Note] WSREP: backend: asio
2014-06-26 10:49:38 6056 [Warning] WSREP: access file(gvwstate.dat) failed(No su
ch file or directory)
2014-06-26 10:49:38 6056 [Note] WSREP: restore pc from disk failed
2014-06-26 10:49:38 6056 [Note] WSREP: GMCast version 0
2014-06-26 10:49:38 6056 [Note] WSREP: (91a8037e, 'tcp://
ng at tcp://0.0.0.0:10041
2014-06-26 10:49:38 6056 [Note] WSREP: (91a8037e, 'tcp://
st: , ttl: 1
2014-06-26 10:49:38 6056 [Note] WSREP: EVS version 0
2014-06-26 10:49:38 6056 [Note] WSREP: PC version 0
2014-06-26 10:49:38 6056 [Note] WSREP: gcomm: connecting to group 'my_wsrep_clus
ter', peer '192.168.
2014-06-26 10:49:38 6056 [Note] WSREP: (91a8037e, 'tcp://
message relay requesting on, nonlive peers:
2014-06-26 10:49:38 6056 [Note] WSREP: declaring 918e2134 at tcp://192.168.17.13
:10031 stable
2014-06-26 10:49:38 6056 [Warning] WSREP: no nodes coming from prim view, prim n
ot possible
2014-06-26 10:49:38 6056 [Note] WSREP: gcomm: connected
2014-06-26 10:49:38 6056 [Note] WSREP: Changing maximum packet size to 64500, re
sulting msg size: 32636
2014-06-26 10:49:38 6056 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2014-06-26 10:49:38 6056 [Note] WSREP: Opened channel 'my_wsrep_cluster'
2014-06-26 10:49:38 6056 [Note] WSREP: New COMPONENT: primary = no, bootstrap =
no, my_idx = 1, memb_num = 2
2014-06-26 10:49:38 6056 [Note] WSREP: Waiting for SST to complete.
2014-06-26 10:49:38 6056 [Note] WSREP: Flow-control interval: [23, 23]
2014-06-26 10:49:38 6056 [Note] WSREP: Received NON-PRIMARY.
2014-06-26 10:49:38 6056 [Note] WSREP: New cluster view: global state: 2a7f5131-
fc61-11e3-
ex: 1, protocol version -1
2014-06-26 10:49:38 6056 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2014-06-26 10:49:46 6056 [Note] WSREP: (91a8037e, 'tcp://
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 8e9b4e2a at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 918e2134 at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 91931b6f at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 91aae867 at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: Node 8e9b4e2a state prim
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 8e9b4e2a at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 918e2134 at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 91931b6f at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 9199b15d at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 91a2005e at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 91a93081 at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 91aae867 at tcp://192.
2014-06-26 10:49:48 6056 [Note] WSREP: declaring 9455a28c at tcp://192.
2014-06-26 10:49:49 6056 [Note] WSREP: Node 8e9b4e2a state prim
2014-06-26 10:49:49 6056 [Note] WSREP: save pc into disk
2014-06-26 10:49:49 6056 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 5, memb_num = 9
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: sent state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: got state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: got state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: got state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: got state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: got state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: got state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: got state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: got state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: STATE EXCHANGE: got state msg: 98906d2d-
2014-06-26 10:49:49 6056 [Note] WSREP: Quorum results:
version = 3,
component = PRIMARY,
conf_id = 2,
members = 9/9 (joined/total),
act_id = 0,
last_appl. = -1,
protocols = 1/5/3 (gcs/repl/appl),
group UUID = 2a7f5131-
2014-06-26 10:49:49 6056 [Note] WSREP: Flow-control interval: [48, 48]
2014-06-26 10:49:49 6056 [Note] WSREP: Restored state OPEN -> JOINED (0)
2014-06-26 10:49:49 6056 [Note] WSREP: New cluster view: global state: 2a7f5131-
2014-06-26 10:49:49 6056 [ERROR] WSREP: Undetected state gap. Can't continue.
2014-06-26 10:49:49 6056 [ERROR] WSREP: WSREP: Group state: 2a7f5131-
2014-06-26 10:49:49 6056 [ERROR] WSREP: WSREP: Local state: 00000000-
2014-06-26 10:49:49 6056 [ERROR] Aborting
Fixes pushed: bazaar. launchpad. net/~codership/ codership- mysql/wsrep- 5.5/revision/ 4004 bazaar. launchpad. net/~codership/ codership- mysql/5. 6/revision/ 4103
5.5 - http://
5.6 - http://