Kiwi Linux
mysql-5.5 package

mysql-server crash when syncing neutron database schema

Bug #1378359 reported by James Page
256
This bug affects 1 person
Affects Status Importance Assigned to Milestone
mysql-5.5 (Kiwi Linux)
Fix Released
High
James Page

Bug Description

Version: '5.5.39-0ubuntu4-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
141007 13:18:36 InnoDB: Assertion failure in thread 139656749573888 in file dict0dict.c line 2563
InnoDB: Failing assertion: UT_LIST_GET_LEN(table->referenced_list) == rbt_size(table->referenced_rbt)
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
13:18:36 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=13
max_threads=151
thread_count=13
connection_count=13
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 338508 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f04b1845050
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f045ef2de98 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x20)[0x7f04ad8092f0]
/usr/sbin/mysqld(handle_fatal_signal+0x3d4)[0x7f04ad6fad94]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfc90)[0x7f04ac49bc90]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x7f04abae8d27]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7f04abaea418]
/usr/sbin/mysqld(+0x58a76d)[0x7f04ad8bb76d]
/usr/sbin/mysqld(+0x59333d)[0x7f04ad8c433d]
/usr/sbin/mysqld(+0x596507)[0x7f04ad8c7507]
/usr/sbin/mysqld(+0x51ae5d)[0x7f04ad84be5d]
/usr/sbin/mysqld(+0x500e1c)[0x7f04ad831e1c]
/usr/sbin/mysqld(_Z18mysql_rename_tableP10handlertonPKcS2_S2_S2_j+0x123)[0x7f04ad674673]
/usr/sbin/mysqld(_Z17mysql_alter_tableP3THDPcS1_P24st_ha_create_informationP10TABLE_LISTP10Alter_infojP8st_orderb+0x1eb6)[0x7f04ad6782a6]
/usr/sbin/mysqld(_ZN21Alter_table_statement7executeEP3THD+0x48a)[0x7f04ad7edc3a]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x163f)[0x7f04ad61818f]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x160)[0x7f04ad61da60]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1bcd)[0x7f04ad61fd6d]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x176)[0x7f04ad6ac386]
/usr/sbin/mysqld(handle_one_connection+0x41)[0x7f04ad6ac3e1]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x80a5)[0x7f04ac4940a5]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f04abbac84d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f0444279920): is an invalid pointer
Connection ID (thread ID): 76
Status: NOT_KILLED

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
141007 13:18:36 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.

ProblemType: Bug
DistroRelease: Ubuntu 14.10
Package: mysql-server-5.5 5.5.39-0ubuntu4
ProcVersionSignature: User Name 3.16.0-20.27-generic 3.16.3
Uname: Linux 3.16.0-20-generic x86_64
ApportVersion: 2.14.7-0ubuntu3
Architecture: amd64
Date: Tue Oct 7 13:24:12 2014
Ec2AMI: ami-000000aa
Ec2AMIManifest: FIXME
Ec2AvailabilityZone: nova
Ec2InstanceType: m1.small
Ec2Kernel: aki-00000002
Ec2Ramdisk: ari-00000002
Logs.var.log.daemon.log:

MySQLConf.etc.mysql.conf.d..keepme:
MySQLConf.etc.mysql.conf.d.binlog.cnf: Error: [Errno 13] Permission denied: '/etc/mysql/conf.d/binlog.cnf'
MySQLConf.etc.mysql.conf.d.mysqld.safe.syslog.cnf:
 [mysqld_safe]
 syslog
MySQLConf.etc.mysql.my.cnf: Error: [Errno 13] Permission denied: '/etc/mysql/my.cnf'
MySQLVarLibDirListing: False
SourcePackage: mysql-5.5
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
James Page (james-page) wrote :
Ryan Beisner (1chb1n)
tags: added: openstack osci
Revision history for this message
James Page (james-page) wrote :
Download full text (8.3 KiB)

And from neutron:

unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/alembic/environment.py", line 688, in run_migrations
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed self.get_context().run_migrations(**kw)
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/alembic/migration.py", line 258, in run_migrations
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed change(**kw)
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/versions/1d6ee1ae5da5_db_healing.py", line 32, in upgrade
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed heal_script.heal()
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/heal_script.py", line 80, in heal
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed execute_alembic_command(el)
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/heal_script.py", line 91, in execute_alembic_command
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed METHODS[command[0]](*command[1:])
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/neutron/db/migration/alembic_migrations/heal_script.py", line 174, in add_column
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed op.add_column(table_name, column.copy(), schema=schema)
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "<string>", line 7, in add_column
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/alembic/operations.py", line 428, in add_column
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed schema=schema
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 127, in add_column
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed self._exec(base.AddColumn(table_name, column, schema=schema))
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 76, in _exec
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed conn.execute(construct, *multiparams, **params)
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 729, in execute
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed return meth(self, multiparams, params)
unit-neutron-api-0: 2014-10-07 13:30:26 INFO shared-db-relation-changed File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/ddl.py", line 69, in _execute_on_connection
unit-neutron-api-0: ...

Read more...

Revision history for this message
James Page (james-page) wrote :

Fairly repeatable:

2014-10-07 13:13:44 INFO shared-db-relation-changed oslo.db.exception.DBConnectionError: (OperationalError) (2013, 'Lost connection to MySQL server during query') 'ALTER TABLE nuage_routerroutes_mapping ADD COLUMN destination VARCHAR(64) NOT NULL' ()
2014-10-07 13:18:36 INFO shared-db-relation-changed oslo.db.exception.DBConnectionError: (OperationalError) (2013, 'Lost connection to MySQL server during query') 'ALTER TABLE nuage_routerroutes_mapping ADD COLUMN destination VARCHAR(64) NOT NULL' ()
2014-10-07 13:30:26 INFO shared-db-relation-changed oslo.db.exception.DBConnectionError: (OperationalError) (2013, 'Lost connection to MySQL server during query') 'ALTER TABLE nuage_routerroutes_mapping ADD COLUMN destination VARCHAR(64) NOT NULL' ()

information type: Public → Private Security
Revision history for this message
James Page (james-page) wrote :

Looks like http://bugs.mysql.com/bug.php?id=73761

James Page (james-page)
information type: Private Security → Public Security
Changed in mysql-5.5 (Ubuntu):
status: New → Triaged
importance: Undecided → High
assignee: nobody → James Page (james-page)
status: Triaged → In Progress
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mysql-5.5 - 5.5.40-0ubuntu1

---------------
mysql-5.5 (5.5.40-0ubuntu1) utopic; urgency=medium

  * New upstream point release, including fixes for:
    - mysql crash during neutron database sync (LP: #1378359).
 -- James Page <email address hidden> Tue, 07 Oct 2014 19:39:53 +0100

Changed in mysql-5.5 (Ubuntu):
status: In Progress → Fix Released
Vison.Chen (vison)
affects: mysql-5.5 (Ubuntu) → mysql-5.5 (Kiwi Linux)
To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.