seen in upstream ci in grenade multi node job
as part of a livemigation.
in this canse the error happens on the destination host.
Traceback (most recent call last):
File "/opt/stack/old/nova/nova/virt/libvirt/host.py", line 605, in _get_domain
return conn.lookupByUUIDString(instance.uuid)
File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 190, in doit
result = proxy_call(self._autowrap, f, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 148, in proxy_call
rv = execute(f, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 129, in execute
six.reraise(c, e, tb)
File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise
raise value
File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 83, in tworker
rv = meth(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/libvirt.py", line 4151, in lookupByUUIDString
if ret is None:raise libvirtError('virDomainLookupByUUIDString() failed', conn=self)
libvirt.libvirtError: Domain not found: no domain with matching uuid '386113bf-cca1-438a-9ab5-4714c147bbfc'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/stack/old/nova/nova/compute/manager.py", line 8005, in _do_pre_live_migration_from_source
instance, block_device_info=block_device_info)
File "/opt/stack/old/nova/nova/virt/libvirt/driver.py", line 9934, in get_instance_disk_info
self._get_instance_disk_info(instance, block_device_info))
File "/opt/stack/old/nova/nova/virt/libvirt/driver.py", line 9915, in _get_instance_disk_info
guest = self._host.get_guest(instance)
File "/opt/stack/old/nova/nova/virt/libvirt/host.py", line 589, in get_guest
return libvirt_guest.Guest(self._get_domain(instance))
File "/opt/stack/old/nova/nova/virt/libvirt/host.py", line 609, in _get_domain
raise exception.InstanceNotFound(instance_id=instance.uuid)
nova.exception.InstanceNotFound: Instance 386113bf-cca1-438a-9ab5-4714c147bbfc could not be found.
this seam similar to
https://bugs.launchpad.net/nova/+bug/1662626 but its not really clear why this fails
see https://zuul.opendev.org/t/openstack/build/cfda29fa579544e481c803c4c5de51fb/log/logs/subnode-2/screen-n-cpu.txt#9697-9729
of https://zuul.opendev.org/t/openstack/build/cfda29fa579544e481c803c4c5de51fb/ for full logs
we just experienced the same in our production environment with ubuntu 20.04 running ussuri with the following package versions:
libvirt-clients 6.0.0-0ubuntu8.10 daemon- driver- qemu 6.0.0-0ubuntu8.10 daemon- driver- storage- rbd 6.0.0-0ubuntu8.10 daemon- system 6.0.0-0ubuntu8.10 daemon- system- systemd 6.0.0-0ubuntu8.10 libvirt 2:21.2.0-0ubuntu1 libvirt 2:21.2.0-0ubuntu1
libvirt-daemon 6.0.0-0ubuntu8.10
libvirt-
libvirt-
libvirt-
libvirt-
libvirt0:amd64 6.0.0-0ubuntu8.10
nova-compute-
python3-libvirt 6.1.0-1
nova-common 2:21.2.0-0ubuntu1
nova-compute 2:21.2.0-0ubuntu1
nova-compute-kvm 2:21.2.0-0ubuntu1
nova-compute-
python3-nova 2:21.2.0-0ubuntu1
python3-novaclient 2:17.0.0-0ubuntu1