Comment 0 for bug 1393391

Under some conditions -TBD-, (port being created, and port being deleted?), neutron-openvswitch-agent can get stuck
trying to subscribe a fanout queue,

It stops responding to any other messages, so it stops effectively working at all.

2014-11-17 06:15:55.842 3042 ERROR root [-] Unexpected exception occurred 61 time(s)... retrying.
2014-11-17 06:15:55.842 3042 TRACE root Traceback (most recent call last):
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func
2014-11-17 06:15:55.842 3042 TRACE root return infunc(*args, **kwargs)
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread
2014-11-17 06:15:55.842 3042 TRACE root self.consume()
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume
2014-11-17 06:15:55.842 3042 TRACE root six.next(it)
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume
2014-11-17 06:15:55.842 3042 TRACE root yield self.ensure(_error_callback, _consume)
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure
2014-11-17 06:15:55.842 3042 TRACE root return method(*args, **kwargs)
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume
2014-11-17 06:15:55.842 3042 TRACE root queues_tail.consume(nowait=False)
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume
2014-11-17 06:15:55.842 3042 TRACE root self.queue.consume(*args, callback=_callback, **options)
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume
2014-11-17 06:15:55.842 3042 TRACE root nowait=nowait)
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume
2014-11-17 06:15:55.842 3042 TRACE root (60, 21), # Channel.basic_consume_ok
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait
2014-11-17 06:15:55.842 3042 TRACE root return self.dispatch_method(method_sig, args, content)
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method
2014-11-17 06:15:55.842 3042 TRACE root return amqp_method(self, args)
2014-11-17 06:15:55.842 3042 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close
2014-11-17 06:15:55.842 3042 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id))
2014-11-17 06:15:55.842 3042 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_c5de1f41f554475492301e03557cdd66' in vhost '/', (60, 20), None)
2014-11-17 06:15:55.842 3042 TRACE root
s /var/log/neutron/os /var/log/neutron/openvswitch-agent.log
[root@mrg-03 ~]# tail /var/log/neutron/openvswitch-agent.log &2014-11-17 06:16:56.085 3042 ERROR root [-] Unexpected exception occurred 60 time(s)... retrying.
2014-11-17 06:16:56.085 3042 TRACE root Traceback (most recent call last):
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/excutils.py", line 92, in inner_func
2014-11-17 06:16:56.085 3042 TRACE root return infunc(*args, **kwargs)
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 746, in _consumer_thread
2014-11-17 06:16:56.085 3042 TRACE root self.consume()
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 737, in consume
2014-11-17 06:16:56.085 3042 TRACE root six.next(it)
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 664, in iterconsume
2014-11-17 06:16:56.085 3042 TRACE root yield self.ensure(_error_callback, _consume)
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 579, in ensure
2014-11-17 06:16:56.085 3042 TRACE root return method(*args, **kwargs)
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 657, in _consume
2014-11-17 06:16:56.085 3042 TRACE root queues_tail.consume(nowait=False)
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/neutron/openstack/common/rpc/impl_kombu.py", line 190, in consume
2014-11-17 06:16:56.085 3042 TRACE root self.queue.consume(*args, callback=_callback, **options)
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/kombu/entity.py", line 598, in consume
2014-11-17 06:16:56.085 3042 TRACE root nowait=nowait)
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 1769, in basic_consume
2014-11-17 06:16:56.085 3042 TRACE root (60, 21), # Channel.basic_consume_ok
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 71, in wait
2014-11-17 06:16:56.085 3042 TRACE root return self.dispatch_method(method_sig, args, content)
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/amqp/abstract_channel.py", line 88, in dispatch_method
2014-11-17 06:16:56.085 3042 TRACE root return amqp_method(self, args)
2014-11-17 06:16:56.085 3042 TRACE root File "/usr/lib/python2.7/site-packages/amqp/channel.py", line 224, in _close
2014-11-17 06:16:56.085 3042 TRACE root raise ChannelError(reply_code, reply_text, (class_id, method_id))
2014-11-17 06:16:56.085 3042 TRACE root ChannelError: 404: (NOT_FOUND - no queue 'q-agent-notifier-port-update_fanout_c5de1f41f554475492301e03557cdd66' in vhost '/', (60, 20), None)
2014-11-17 06:16:56.085 3042 TRACE root