Galera

  1. Series 2.x
  2. Bug #1288528
  3. Comment #1

Comment 1 for bug 1288528

Revision history for this message
Jervin R (revin) wrote : Re: SST Resumes Even When Donor Was Already Detected as SYNCED State

> Step #1
2014-03-05 20:34:30 6855 [Note] WSREP: Node 2.0 (pxc02) resyncs itself to group
2014-03-05 20:34:30 6855 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 5957732)
2014-03-05 20:34:30 6855 [Note] WSREP: Member 2 (pxc02) synced with group.

> Step #2
2014-03-05 20:34:30 6855 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 5957732)
2014-03-05 20:34:30 6855 [Note] WSREP: Synchronized with group, ready for connections
2014-03-05 20:34:31 6855 [Note] WSREP: Node 0.0 (pxc03) requested state transfer from 'pxc02'. Selected 2.0 (pxc02)(SYNCED) as donor.
2014-03-05 20:34:31 6855 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 5957732)
2014-03-05 20:34:31 6855 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2014-03-05 20:34:31 6855 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'donor' --address '192.168.56.44:4444/xtrabackup_sst' --auth '(null)' --socket '/var/lib/mysql/mysqld.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' '' --gtid 'de9d6a8d-a105-11e3-9d36-7789b5225560:5957732''
2014-03-05 20:34:31 6855 [Note] WSREP: sst_donor_thread signaled with 0
WSREP_SST: [INFO] Streaming with xbstream (20140305 20:34:31.664)
WSREP_SST: [INFO] Using socat as streamer (20140305 20:34:31.665)
WSREP_SST: [INFO] Streaming GTID file before SST (20140305 20:34:31.760)
WSREP_SST: [INFO] Evaluating xbstream -c ${INFO_FILE} | socat -u stdio TCP:192.168.56.44:4444; RC=( ${PIPESTATUS[@]} ) (20140305 20:34:31.762)
WSREP_SST: [INFO] Sleeping before data transfer for SST (20140305 20:34:31.766)
2014-03-05 20:34:32 6855 [Note] WSREP: 2.0 (pxc02): State transfer to 0.0 (pxc03) complete.
2014-03-05 20:34:32 6855 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 5957732)

> Step #5
2014-03-05 20:34:32 6855 [Note] WSREP: Member 2 (pxc02) synced with group.
2014-03-05 20:34:32 6855 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 5957732)
2014-03-05 20:34:32 6855 [Note] WSREP: Synchronized with group, ready for connections
2014-03-05 20:34:32 6855 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2014-03-05 20:34:35 6855 [Warning] WSREP: Protocol violation. JOIN message sender 2.0 (pxc02) is not in state transfer (SYNCED). Message ignored.
WSREP_SST: [INFO] Streaming the backup to joiner at 192.168.56.44 4444 (20140305 20:34:41.768)
WSREP_SST: [INFO] Evaluating innobackupex --defaults-file=/etc/my.cnf --no-version-check $INNOEXTRA --galera-info --stream=$sfmt ${TMPDIR} 2>${DATA}/innobackup.backup.log | socat -u stdio TCP:192.168.56.44:4444; RC=( ${PIPESTATUS[@]} ) (20140305 20:34:41.770)
2014-03-05 20:34:46 6855 [Note] WSREP: Provider paused at de9d6a8d-a105-11e3-9d36-7789b5225560:5957732 (3195887)
2014-03-05 20:34:48 6855 [Note] WSREP: resuming provider at 3195887
2014-03-05 20:34:48 6855 [Note] WSREP: Provider resumed.

> ERROR Indicated here, jowever the joiner (pxc03) still completed SST
2014-03-05 20:34:48 6855 [ERROR] WSREP: sst sent called when not SST donor, state SYNCED
WSREP_SST: [INFO] Total time on donor: 0 seconds (20140305 20:34:48.152)
2014-03-05 20:34:52 6855 [Note] WSREP: 0.0 (pxc03): State transfer from 2.0 (pxc02) complete.
2014-03-05 20:34:52 6855 [Note] WSREP: Member 0 (pxc03) synced with group.

There should be consistent polling mechanism to abort an SST if the donor state goes back to SYNCED state. This is especially problematic for large datasets which could cause issue if another SST starts while another is in progress.