I've gotten this assertion failure as well on a 3-node config:
2014-08-27 12:29:31 10938 [Warning] WSREP: Ignoring error for TO isolated action: source: eca5bcc5-2d2f-11e4-8f7e-0f8a990eca4c version: 3 local: 0 state: APPLYING flags: 65 conn_id: 13012 trx_id: -1 seqnos (l: 656, g: 498, s: 497, d: 497, ts: 79063041752262)
2014-08-27 13:23:21 7fe702492700 InnoDB: Assertion failure in thread 140630152521472 in file lock0lock.cc line 3980
InnoDB: Failing assertion: lock != ctx->wait_lock
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
13:23:21 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona XtraDB Cluster better by reporting any
bugs at https://bugs.launchpad.net/percona-xtradb-cluster
key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=6
max_threads=102
thread_count=9
connection_count=4
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 57084 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7fe6f8338ee0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fe702491e50 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x8c646e]
/usr/sbin/mysqld(handle_fatal_signal+0x379)[0x67f7c9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf030)[0x7fe7b490f030]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35)[0x7fe7b2b2a545]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x180)[0x7fe7b2b2d7c0]
/usr/sbin/mysqld[0x92053f]
/usr/sbin/mysqld[0x9219c3]
/usr/sbin/mysqld[0x921e6b]
/usr/sbin/mysqld[0x925bd1]
/usr/sbin/mysqld[0x9986df]
/usr/sbin/mysqld[0x99d2bd]
/usr/sbin/mysqld[0x8f2961]
/usr/sbin/mysqld(_ZN7handler17ha_index_read_mapEPhPKhm16ha_rkey_function+0x7f)[0x5de5bf]
/usr/sbin/mysqld(_ZN7handler16read_range_firstEPK12st_key_rangeS2_bb+0xdb)[0x5debfb]
/usr/sbin/mysqld(_ZN7handler21multi_range_read_nextEPPc+0x86)[0x5da716]
/usr/sbin/mysqld(_ZN18QUICK_RANGE_SELECT8get_nextEv+0x52)[0x7ed582]
/usr/sbin/mysqld[0x80ca69]
/usr/sbin/mysqld(_Z12mysql_updateP3THDP10TABLE_LISTR4ListI4ItemES6_PS4_jP8st_ordery15enum_duplicatesbPySB_+0x11a2)[0x75b3a2]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1e69)[0x6f5dd9]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x5c8)[0x6f9ad8]
/usr/sbin/mysqld[0x6f9bdd]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xc4e)[0x6facbe]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x159)[0x6fbc89]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x27d)[0x6cdaed]
/usr/sbin/mysqld(handle_one_connection+0x42)[0x6cdb72]
/usr/sbin/mysqld(pfs_spawn_thread+0x140)[0xb0f410]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50)[0x7fe7b4906b50]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fe7b2bd420d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (3b73610): UPDATE jos_session
SET session_data = '[redacted]', session_updated = 1409145801
WHERE session_id = '77ce721028e3eda29e58b8d81a239497'
Connection ID (thread ID): 14000
Status: NOT_KILLED
You may download the Percona XtraDB Cluster operations manual by visiting http://www.percona.com/software/percona-xtradb-cluster/. You may find information
in the manual which will help you identify the cause of the crash.
140827 13:23:21 mysqld_safe Number of processes running now: 0
140827 13:23:21 mysqld_safe WSREP: not restarting wsrep node automatically
140827 13:23:21 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
I've gotten this assertion failure as well on a 3-node config:
2014-08-27 12:29:31 10938 [Warning] WSREP: Ignoring error for TO isolated action: source: eca5bcc5- 2d2f-11e4- 8f7e-0f8a990eca 4c version: 3 local: 0 state: APPLYING flags: 65 conn_id: 13012 trx_id: -1 seqnos (l: 656, g: 498, s: 497, d: 497, ts: 79063041752262) bugs.mysql. com. dev.mysql. com/doc/ refman/ 5.6/en/ forcing- innodb- recovery. html /bugs.launchpad .net/percona- xtradb- cluster
2014-08-27 13:23:21 7fe702492700 InnoDB: Assertion failure in thread 140630152521472 in file lock0lock.cc line 3980
InnoDB: Failing assertion: lock != ctx->wait_lock
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://
InnoDB: about forcing recovery.
13:23:21 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Please help us make Percona XtraDB Cluster better by reporting any
bugs at https:/
key_buffer_ size=16777216 size=131072 connections= 6 size)*max_ threads = 57084 K bytes of memory
read_buffer_
max_used_
max_threads=102
thread_count=9
connection_count=4
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7fe6f8338ee0 mysqld( my_print_ stacktrace+ 0x2e)[0x8c646e] mysqld( handle_ fatal_signal+ 0x379)[ 0x67f7c9] 64-linux- gnu/libpthread. so.0(+0xf030) [0x7fe7b490f030 ] 64-linux- gnu/libc. so.6(gsignal+ 0x35)[0x7fe7b2b 2a545] 64-linux- gnu/libc. so.6(abort+ 0x180)[ 0x7fe7b2b2d7c0] mysqld[ 0x92053f] mysqld[ 0x9219c3] mysqld[ 0x921e6b] mysqld[ 0x925bd1] mysqld[ 0x9986df] mysqld[ 0x99d2bd] mysqld[ 0x8f2961] mysqld( _ZN7handler17ha _index_ read_mapEPhPKhm 16ha_rkey_ function+ 0x7f)[0x5de5bf] mysqld( _ZN7handler16re ad_range_ firstEPK12st_ key_rangeS2_ bb+0xdb) [0x5debfb] mysqld( _ZN7handler21mu lti_range_ read_nextEPPc+ 0x86)[0x5da716] mysqld( _ZN18QUICK_ RANGE_SELECT8ge t_nextEv+ 0x52)[0x7ed582] mysqld[ 0x80ca69] mysqld( _Z12mysql_ updateP3THDP10T ABLE_LISTR4List I4ItemES6_ PS4_jP8st_ ordery15enum_ duplicatesbPySB _+0x11a2) [0x75b3a2] mysqld( _Z21mysql_ execute_ commandP3THD+ 0x1e69) [0x6f5dd9] mysqld( _Z11mysql_ parseP3THDPcjP1 2Parser_ state+0x5c8) [0x6f9ad8] mysqld[ 0x6f9bdd] mysqld( _Z16dispatch_ command19enum_ server_ commandP3THDPcj +0xc4e) [0x6facbe] mysqld( _Z10do_ commandP3THD+ 0x159)[ 0x6fbc89] mysqld( _Z24do_ handle_ one_connectionP 3THD+0x27d) [0x6cdaed] mysqld( handle_ one_connection+ 0x42)[0x6cdb72] mysqld( pfs_spawn_ thread+ 0x140)[ 0xb0f410] 64-linux- gnu/libpthread. so.0(+0x6b50) [0x7fe7b4906b50 ] 64-linux- gnu/libc. so.6(clone+ 0x6d)[0x7fe7b2b d420d]
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fe702491e50 thread_stack 0x30000
/usr/sbin/
/usr/sbin/
/lib/x86_
/lib/x86_
/lib/x86_
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/usr/sbin/
/lib/x86_
/lib/x86_
Trying to get some variables. a29e58b8d81a239 497'
Some pointers may be invalid and cause the dump to abort.
Query (3b73610): UPDATE jos_session
SET session_data = '[redacted]', session_updated = 1409145801
WHERE session_id = '77ce721028e3ed
Connection ID (thread ID): 14000
Status: NOT_KILLED
You may download the Percona XtraDB Cluster operations manual by visiting www.percona. com/software/ percona- xtradb- cluster/. You may find information mysqld/ mysqld. pid ended
http://
in the manual which will help you identify the cause of the crash.
140827 13:23:21 mysqld_safe Number of processes running now: 0
140827 13:23:21 mysqld_safe WSREP: not restarting wsrep node automatically
140827 13:23:21 mysqld_safe mysqld from pid file /var/run/