Foundation cloud Builds are seeing many lost connections to the MySQL server from neutron-api.
The bundle and crashdump are attached.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines [req-ea783de8-dbd5-4b41-972e-8d76c0a135fb - - - - -] Database connection was found disconnected; reconnecting: oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: 'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8) 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last): 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines context) 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 508, in do_execute 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines cursor.execute(statement, parameters) 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 165, in execute 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines result = self._query(query) 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 321, in _query 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines conn.query(q) 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 860, in query 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1061, in _read_query_result 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines result.read() 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1349, in read 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines first_packet = self.connection._read_packet() 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 991, in _read_packet 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines packet_header = self._read_bytes(4) 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1037, in _read_bytes 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") 2018-12-04 00:21:16.595 182093 ERROR oslo_db.sqlalchemy.engines pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query')
Foundation cloud Builds are seeing many lost connections to the MySQL server from neutron-api.
The bundle and crashdump are attached.
2018-12-04 00:21:16.595 182093 ERROR oslo_db. sqlalchemy. engines [req-ea783de8- dbd5-4b41- 972e-8d76c0a135 fb - - - - -] Database connection was found disconnected; reconnecting: oslo_db. exception. DBConnectionErr or: (pymysql. err.Operational Error) (2013, 'Lost connection to MySQL server during query') [SQL: 'SELECT 1'] (Background on this error at: http:// sqlalche. me/e/e3q8) sqlalchemy. engines Traceback (most recent call last): sqlalchemy. engines File "/usr/lib/ python3/ dist-packages/ sqlalchemy/ engine/ base.py" , line 1193, in _execute_context sqlalchemy. engines context) sqlalchemy. engines File "/usr/lib/ python3/ dist-packages/ sqlalchemy/ engine/ default. py", line 508, in do_execute sqlalchemy. engines cursor. execute( statement, parameters) sqlalchemy. engines File "/usr/lib/ python3/ dist-packages/ pymysql/ cursors. py", line 165, in execute sqlalchemy. engines result = self._query(query) sqlalchemy. engines File "/usr/lib/ python3/ dist-packages/ pymysql/ cursors. py", line 321, in _query sqlalchemy. engines conn.query(q) sqlalchemy. engines File "/usr/lib/ python3/ dist-packages/ pymysql/ connections. py", line 860, in query sqlalchemy. engines self._affected_rows = self._read_ query_result( unbuffered= unbuffered) sqlalchemy. engines File "/usr/lib/ python3/ dist-packages/ pymysql/ connections. py", line 1061, in _read_query_result sqlalchemy. engines result.read() sqlalchemy. engines File "/usr/lib/ python3/ dist-packages/ pymysql/ connections. py", line 1349, in read sqlalchemy. engines first_packet = self.connection ._read_ packet( ) sqlalchemy. engines File "/usr/lib/ python3/ dist-packages/ pymysql/ connections. py", line 991, in _read_packet sqlalchemy. engines packet_header = self._read_bytes(4) sqlalchemy. engines File "/usr/lib/ python3/ dist-packages/ pymysql/ connections. py", line 1037, in _read_bytes sqlalchemy. engines CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") sqlalchemy. engines pymysql. err.Operational Error: (2013, 'Lost connection to MySQL server during query')
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.
2018-12-04 00:21:16.595 182093 ERROR oslo_db.