Nova creates duplicate Neutron ports on instance reschedule
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| OpenStack Compute (nova) |
Fix Committed
|
Medium
|
Liyingjun | ||
Bug Description
Consider this environment:
* Running stable/mitaka (latest available)
* Four hypervisors
* Two glance nodes (A and B)
* The glance nodes are storing images locally but the image files aren't in sync between both hosts
When I request a new instance, the following happens:
* Instance is scheduled to hypervisor A
* Hypervisor A checks to see if the image is available for use -- SUCCESS
* Hypervisor A calls neutron for a network port -- SUCCESS
* Hypervisor A tries to download image from glance server A -- FAILURE (glance server A doesn't have the image cached on its filesystem)
* Instance is rescheduled to hypervisor B
* Hypervisor B checks to see if the image is available for use -- SUCCESS
* Hypervisor B calls neutron for a network port -- SUCCESS
* Hypervisor B downloads an image from glance server B -- SUCCESS (glance server B has the image on its filesystem)
The instance will come up on hypervisor B with two ports attached to the instance. The second one (requested by hypervisor B) will be up and fully functional. The first port (requested by hypervisor A) will be marked as 'down' and won't be usable.
It seems like nova-compute should call neutron to say "I don't need that network port any longer since I can't get what I need to build the rest of the instance" and clean up that port. Without the cleanup, an instance can end up with a lot of ports attached and potentially waste a lot of IPv4 address space.
I wrote more details on this issue here: https:/
| summary: |
- nova doesn't clean up network ports when an image fails to download from + nova should clean up network ports when an image fails to download from glance |
| Rui Chen (kiwik-chenrui) wrote Re: nova should clean up network ports when an image fails to download from glance | #1 |
| Changed in nova: | |
| status: | New → Confirmed |
| Changed in nova: | |
| assignee: | nobody → Zhenyu Zheng (zhengzhenyu) |
| cloudbuilders (operations-8) wrote | #2 |
| Maciej Szankin (mszankin) wrote | #3 |
| summary: |
- nova should clean up network ports when an image fails to download from - glance + Nova creates duplicate Neutron ports on instance reschedule |
| Changed in nova: | |
| importance: | Undecided → Medium |
| Changed in nova: | |
| assignee: | Zhenyu Zheng (zhengzhenyu) → nobody |
| Piyush Srivastava (piyush0101) wrote | #4 |
| Changed in nova: | |
| assignee: | nobody → Liyingjun (liyingjun) |
| OpenStack Infra (hudson-openstack) wrote Fix proposed to nova (master) | #5 |
| Changed in nova: | |
| status: | Confirmed → In Progress |
| Jirayut Nimsaeng (winggundamth) wrote | #6 |
| information type: | Public → Public Security |
| information type: | Public Security → Private Security |
| information type: | Private Security → Public |
| Sean Dague (sdague) wrote | #7 |
| tags: | added: openstack-version.mitaka |
| melanie witt (melwitt) wrote | #8 |
| OpenStack Infra (hudson-openstack) wrote Change abandoned on nova (master) | #9 |
| Changed in nova: | |
| status: | In Progress → Fix Committed |
Today I face the same issue in my devstack.
stack@szxbzci0004 ~/nova (master *) $ git log -1
commit e9d503a1202fadd
Merge: 5426d95 a6ad102
Author: Jenkins <email address hidden>
Date: Thu Sep 1 03:15:49 2016 +0000
Merge "Update placement config reno"
I have two compute nodes, but one of them(A) exist RBD configure issue, so when libvirt try to launch the instance, a LibvirtError is raised, the instance is rescheduled to another compute node(B), but the linux bridge isn't cleaned up on compute node A, and the instance launch on compute node B successfully, but it allocate port again, so the instance run with two ports.
See my operation details:
http://