volume attach test race fails in dsvm-tempest-xen CI due to: "libvirtError: internal error: libxenlight failed to attach disk 'xvdb'"

Bug #1518007 reported by Matt Riedemann
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Confirmed
Medium
Unassigned

Bug Description

Seen in a CI run here:

http://logs.openstack.xenproject.org/79/241179/9/check/dsvm-tempest-xen/6901c94/logs/screen-n-cpu.txt.gz

This was unrelated to the change being tested: https://review.openstack.org/#/c/241179/

2015-11-19 05:41:56.395 ERROR nova.compute.manager [req-0f579e1a-f928-4f24-8beb-8288a2699284 tempest-AttachVolumeTestJSON-174551048 tempest-AttachVolumeTestJSON-1798726384] [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] Failed to attach 853230d9-8e71-416b-a27b-0107ca181020 at /dev/xvdb
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] Traceback (most recent call last):
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/compute/manager.py", line 4653, in _attach_volume
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] do_check_attach=False, do_driver_attach=True)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/block_device.py", line 56, in wrapped
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] ret_val = method(obj, context, *args, **kwargs)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/block_device.py", line 297, in attach
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] connector)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 204, in __exit__
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] six.reraise(self.type_, self.value, self.tb)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/block_device.py", line 288, in attach
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] device_type=self['device_type'], encryption=encryption)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1123, in attach_volume
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] self._disconnect_volume(connection_info, disk_dev)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 204, in __exit__
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] six.reraise(self.type_, self.value, self.tb)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1112, in attach_volume
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] guest.attach_device(conf, persistent=True, live=live)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 233, in attach_device
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] self._domain.attachDeviceFlags(conf.to_xml(), flags=flags)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] result = proxy_call(self._autowrap, f, *args, **kwargs)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] rv = execute(f, *args, **kwargs)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] six.reraise(c, e, tb)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] rv = meth(*args, **kwargs)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 554, in attachDeviceFlags
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d] libvirtError: internal error: libxenlight failed to attach disk 'xvdb'
2015-11-19 05:41:56.395 22173 ERROR nova.compute.manager [instance: 264d4140-70ad-4eae-b8b7-2602f52bcd3d]

Tags: libvirt xen
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.