openstack mitaka evacuate fails and no clean up the evacuated instance xml in libvirt
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Confirmed
|
Low
|
Unassigned |
Bug Description
Description
===========
For example:
Theres compute Nodes: A and B and C
instance "vmtest" runs on A now.
Steps to reproduce
==================
1.nova host-evacuate A
2.Assumed instance "vmtest" evacuated failed on B with libvirt error. And found the instance xml of vmtest is still in /etc/libvirt/qemu
3.Then try again "nova host-evacuate A".If vmtest is scheduled on B again, there will throw exception: Instance instance-vmtest already exists.
Expected result
===============
When vmtest first evacuated failed on B, remove instance xml of vmtest on B
Actual result
=============
If vmtest is scheduled on B again, there will throw exception: Instance instance-vmtest already exists.
Whether it should be clean up the instance xml of vmtest on the failed evacuated node B?
Environment
===========
openstack mitaka
Logs
==========
09ed2cfc-
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
Exception during message handling: Instance instance-000024d4 already exists.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
2018-08-01 12:52:35.288 5665 ERROR oslo_messaging.
description: | updated |
description: | updated |
Changed in nova: | |
assignee: | Matt Riedemann (mriedem) → nobody |
status: | In Progress → Confirmed |
Mitaka and Newton are both end of life, but looking at current master (rocky) this is still a problem most likely:
https:/ /github. com/openstack/ nova/blob/ d4dbb42593893c1 d1ed51a127b7183 a314bcac2c/ nova/compute/ manager. py#L3102
Unless something magically started cleaning up the guest on a failed evacuate.
Can you tell where the evacuate failed? Was it during or after the driver.spawn() call here?
https:/ /github. com/openstack/ nova/blob/ d4dbb42593893c1 d1ed51a127b7183 a314bcac2c/ nova/compute/ manager. py#L2906