Another instance, with Kilo on Ubuntu:
2017-03-03 00:52:23.799 19437 WARNING networking_calico.plugins.ml2.drivers.calico.mech_calico [req-14b3e0a3-e85f-4107-8099-75ecc653892c ] Missing ports: set([]) 2017-03-03 00:52:23.799 19437 WARNING networking_calico.plugins.ml2.drivers.calico.mech_calico [req-14b3e0a3-e85f-4107-8099-75ecc653892c ] Extra ports: set([u'e13902a4-eb8a-48d3-8214-10fd7da6b3e8']) 2017-03-03 00:52:25.398 19437 WARNING neutron.plugins.ml2.plugin [req-f2ae6591-69f9-428e-9305-1b2f62f6613c ] Port e13902a4-eb8a-48d3-8214-10fd7da6b3e8 updated up by agent not found 2017-03-03 00:52:28.063 19456 ERROR neutron.api.v2.resource [req-7d3cf55c-097d-4b57-ad60-e51edae1f901 ] create failed 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource Traceback (most recent call last): 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/resource.py", line 83, in resource 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource result = method(request=request, **args) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 464, in create 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource obj = obj_creator(request.context, **kwargs) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_db/api.py", line 131, in wrapper 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource return f(*args, **kwargs) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 970, in create_port 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource result, mech_context = self._create_port_db(context, port) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/plugin.py", line 944, in _create_port_db 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource result = super(Ml2Plugin, self).create_port(context, port) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/neutron/db/db_base_plugin_v2.py", line 1850, in create_port 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource context, ip_address, network_id, subnet_id, port_id) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 470, in __exit__ 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource self.rollback() 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__ 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource compat.reraise(exc_type, exc_value, exc_tb) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 467, in __exit__ 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource self.commit() 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 377, in commit 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource self._prepare_impl() 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 357, in _prepare_impl 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource self.session.flush() 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1919, in flush 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource self._flush(objects) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2037, in _flush 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource transaction.rollback(_capture_exception=True) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__ 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource compat.reraise(exc_type, exc_value, exc_tb) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2001, in _flush 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource flush_context.execute() 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource rec.execute(self) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 526, in execute 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource uow 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 60, in save_obj 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource mapper, table, update) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 518, in _emit_update_statements 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource execute(statement, params) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 729, in execute 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource return meth(self, multiparams, params) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource return connection._execute_clauseelement(self, multiparams, params) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource compiled_sql, distilled_params 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 958, in _execute_context 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource context) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/compat/handle_error.py", line 261, in _handle_dbapi_exception 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource e, statement, parameters, cursor, context) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1155, in _handle_dbapi_exception 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource util.raise_from_cause(newraise, exc_info) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource reraise(type(exception), exception, tb=exc_tb) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 951, in _execute_context 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource context) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 436, in do_execute 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource cursor.execute(statement, parameters) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource self.errorhandler(self, exc, value) 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource raise errorclass, errorvalue 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE ipavailabilityranges SET first_ip=%s WHERE ipavailabilityranges.allocation_pool_id = %s AND ipavailabilityranges.first_ip = %s AND ipavailabilityranges.last_ip = %s' ('10.29.0.3', '6a20f0c9-bf36-4571-8dce-9ad2bba35f7c', '10.29.0.2', '10.29.255.254') 2017-03-03 00:52:28.063 19456 TRACE neutron.api.v2.resource
Another instance, with Kilo on Ubuntu:
2017-03-03 00:52:23.799 19437 WARNING networking_ calico. plugins. ml2.drivers. calico. mech_calico [req-14b3e0a3- e85f-4107- 8099-75ecc65389 2c ] Missing ports: set([]) calico. plugins. ml2.drivers. calico. mech_calico [req-14b3e0a3- e85f-4107- 8099-75ecc65389 2c ] Extra ports: set([u' e13902a4- eb8a-48d3- 8214-10fd7da6b3 e8']) plugins. ml2.plugin [req-f2ae6591- 69f9-428e- 9305-1b2f62f661 3c ] Port e13902a4- eb8a-48d3- 8214-10fd7da6b3 e8 updated up by agent not found api.v2. resource [req-7d3cf55c- 097d-4b57- ad60-e51edae1f9 01 ] create failed api.v2. resource Traceback (most recent call last): api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ api/v2/ resource. py", line 83, in resource api.v2. resource result = method( request= request, **args) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ api/v2/ base.py" , line 464, in create api.v2. resource obj = obj_creator( request. context, **kwargs) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ oslo_db/ api.py" , line 131, in wrapper api.v2. resource return f(*args, **kwargs) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ plugins/ ml2/plugin. py", line 970, in create_port api.v2. resource result, mech_context = self._create_ port_db( context, port) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ plugins/ ml2/plugin. py", line 944, in _create_port_db api.v2. resource result = super(Ml2Plugin, self).create_ port(context, port) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ neutron/ db/db_base_ plugin_ v2.py", line 1850, in create_port api.v2. resource context, ip_address, network_id, subnet_id, port_id) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/session. py", line 470, in __exit__ api.v2. resource self.rollback() api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ util/langhelper s.py", line 60, in __exit__ api.v2. resource compat. reraise( exc_type, exc_value, exc_tb) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/session. py", line 467, in __exit__ api.v2. resource self.commit() api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/session. py", line 377, in commit api.v2. resource self._prepare_ impl() api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/session. py", line 357, in _prepare_impl api.v2. resource self.session. flush() api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/session. py", line 1919, in flush api.v2. resource self._flush( objects) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/session. py", line 2037, in _flush api.v2. resource transaction. rollback( _capture_ exception= True) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ util/langhelper s.py", line 60, in __exit__ api.v2. resource compat. reraise( exc_type, exc_value, exc_tb) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/session. py", line 2001, in _flush api.v2. resource flush_context. execute( ) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/unitofwork. py", line 372, in execute api.v2. resource rec.execute(self) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/unitofwork. py", line 526, in execute api.v2. resource uow api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/persistence .py", line 60, in save_obj api.v2. resource mapper, table, update) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ orm/persistence .py", line 518, in _emit_update_ statements api.v2. resource execute(statement, params) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ engine/ base.py" , line 729, in execute api.v2. resource return meth(self, multiparams, params) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ sql/elements. py", line 322, in _execute_ on_connection api.v2. resource return connection. _execute_ clauseelement( self, multiparams, params) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ engine/ base.py" , line 826, in _execute_ clauseelement api.v2. resource compiled_sql, distilled_params api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ engine/ base.py" , line 958, in _execute_context api.v2. resource context) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ oslo_db/ sqlalchemy/ compat/ handle_ error.py" , line 261, in _handle_ dbapi_exception api.v2. resource e, statement, parameters, cursor, context) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ engine/ base.py" , line 1155, in _handle_ dbapi_exception api.v2. resource util.raise_ from_cause( newraise, exc_info) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ util/compat. py", line 199, in raise_from_cause api.v2. resource reraise( type(exception) , exception, tb=exc_tb) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ engine/ base.py" , line 951, in _execute_context api.v2. resource context) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ sqlalchemy/ engine/ default. py", line 436, in do_execute api.v2. resource cursor. execute( statement, parameters) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ MySQLdb/ cursors. py", line 174, in execute api.v2. resource self.errorhandl er(self, exc, value) api.v2. resource File "/usr/lib/ python2. 7/dist- packages/ MySQLdb/ connections. py", line 36, in defaulterrorhandler api.v2. resource raise errorclass, errorvalue api.v2. resource DBDeadlock: (OperationalError) (1213, 'Deadlock found when trying to get lock; try restarting transaction') 'UPDATE ipavailabilityr anges SET first_ip=%s WHERE ipavailabilityr anges.allocatio n_pool_ id = %s AND ipavailabilityr anges.first_ ip = %s AND ipavailabilityr anges.last_ ip = %s' ('10.29.0.3', '6a20f0c9- bf36-4571- 8dce-9ad2bba35f 7c', '10.29.0.2', '10.29.255.254') api.v2. resource
2017-03-03 00:52:23.799 19437 WARNING networking_
2017-03-03 00:52:25.398 19437 WARNING neutron.
2017-03-03 00:52:28.063 19456 ERROR neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.
2017-03-03 00:52:28.063 19456 TRACE neutron.