OpenStack Compute (nova)

nova-network cannot re-generate MAC address if collision happen

Bug #1059366 reported by MotoKen
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Dan Smith
OpenStack Compute (nova) 2013.1 "grizzly"
Folsom
Fix Released
Medium
Dan Smith
OpenStack Compute (nova) 2012.2.1
nova (Ubuntu)
Fix Released
Undecided
Unassigned
Quantal
Fix Released
Undecided
Unassigned

Bug Description

the flag "create_unique_mac_address_attempts" does not work if collision happen.

when collision occurs, it will raise DBError rather than IntegrityError.

2012-09-30 16:02:22 TRACE nova.rpc.amqp Traceback (most recent call last):
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 263, in _process_data
2012-09-30 16:02:22 TRACE nova.rpc.amqp rval = node_func(context=ctxt, **node_args)
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 270, in wrapped
2012-09-30 16:02:22 TRACE nova.rpc.amqp return func(self, context, *args, **kwargs)
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 332, in allocate_for_instance
2012-09-30 16:02:22 TRACE nova.rpc.amqp **kwargs)
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 270, in wrapped
2012-09-30 16:02:22 TRACE nova.rpc.amqp return func(self, context, *args, **kwargs)
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 922, in allocate_for_instance
2012-09-30 16:02:22 TRACE nova.rpc.amqp self._allocate_mac_addresses(context, instance_id, networks)
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 1125, in _allocate_mac_addresses
2012-09-30 16:02:22 TRACE nova.rpc.amqp self.add_virtual_interface(context, instance_id, network['id'])
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 1135, in add_virtual_interface
2012-09-30 16:02:22 TRACE nova.rpc.amqp return self.db.virtual_interface_create(context, vif)
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 469, in virtual_interface_create
2012-09-30 16:02:22 TRACE nova.rpc.amqp return IMPL.virtual_interface_create(context, values)
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/toolbox_nova/notifier/api.py", line 196, in wrapped_func
2012-09-30 16:02:22 TRACE nova.rpc.amqp retval = function(*args, **kwargs)
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 129, in wrapper
2012-09-30 16:02:22 TRACE nova.rpc.amqp return f(*args, **kwargs)
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 1136, in virtual_interface_create
2012-09-30 16:02:22 TRACE nova.rpc.amqp vif_ref.save()
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/models.py", line 58, in save
2012-09-30 16:02:22 TRACE nova.rpc.amqp session.flush()
2012-09-30 16:02:22 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 95, in _wrap
2012-09-30 16:02:22 TRACE nova.rpc.amqp raise DBError(e)
2012-09-30 16:02:22 TRACE nova.rpc.amqp DBError: (IntegrityError) (1062, "Duplicate entry 'fa:16:3e:37:7f:c4' for key 'address'") 'INSERT INTO virtual_interfaces (created_at, updated_at, deleted_at, deleted, address, network_id, instance_id, uuid) VALUES (%s, %s, %s, %s, %s, %s, %s, %s)' (datetime.datetime(2012, 9, 30, 8, 2, 22, 459356), None, None, 0, 'fa:16:3e:37:7f:c4', 2L, 3082, '1df53b1d-ca43-4664-88e6-bf041f8a606c')
2012-09-30 16:02:22 TRACE nova.rpc.amqp

Revision history for this message
Dan Smith (danms) wrote :

Confirmed in current master that db.virtual_interface_create is catching IntegrityError, instead of the raised DBError.

Changed in nova:
assignee: nobody → Dan Smith (danms)
importance: Undecided → Medium
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/14920

Changed in nova:
status: Confirmed → In Progress
Vish Ishaya (vishvananda)
tags: added: folsom-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/14920
Committed: http://github.com/openstack/nova/commit/91fb1ce5c8b8c11fb6e35d8774b7df48da0786b3
Submitter: Jenkins
Branch: master

commit 91fb1ce5c8b8c11fb6e35d8774b7df48da0786b3
Author: Dan Smith <email address hidden>
Date: Fri Oct 26 14:32:10 2012 -0700

    Fix nova-network MAC collision logic

    The exception being caught and transformed in sqlalchemy/api.py
    was incorrect, causing nova-network to miss it in the MAC
    collision retry loop.

    This patch corrects that and adds a test of the collision logic.

    Fixes bug 1059366

    Change-Id: I6bf16c2f59f06c7e84441604b057afd8dfe6d709

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/folsom)

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/14977

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/folsom)

Reviewed: https://review.openstack.org/14977
Committed: http://github.com/openstack/nova/commit/5ccd691c2089a2df8aa22ff0bdbf95095d218a00
Submitter: Jenkins
Branch: stable/folsom

commit 5ccd691c2089a2df8aa22ff0bdbf95095d218a00
Author: Dan Smith <email address hidden>
Date: Fri Oct 26 14:32:10 2012 -0700

    Fix nova-network MAC collision logic

    The exception being caught and transformed in sqlalchemy/api.py
    was incorrect, causing nova-network to miss it in the MAC
    collision retry loop.

    This patch corrects that and adds a test of the collision logic.

    Backports the fix for bug 1059366

    Change-Id: I89d17eb8606a18f102e4de0dc76c77205cafbc8d

tags: added: in-stable-folsom
Chuck Short (zulcss)
tags: removed: folsom-backport-potential
Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-1
status: Fix Committed → Fix Released
Adam Gandelman (gandelman-a)
Changed in nova (Ubuntu):
status: New → Fix Released
Changed in nova (Ubuntu Quantal):
status: New → Confirmed
Revision history for this message
Clint Byrum (clint-fewbar) wrote : Please test proposed package

Hello MotoKen, or anyone else affected,

Accepted nova into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/nova/2012.2.1+stable-20121212-a99a802e-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in nova (Ubuntu Quantal):
status: Confirmed → Fix Committed
tags: added: verification-needed
Mark McLoughlin (markmc)
tags: removed: in-stable-folsom
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (8.3 KiB)

This bug was fixed in the package nova - 2012.2.1+stable-20121212-a99a802e-0ubuntu1

---------------
nova (2012.2.1+stable-20121212-a99a802e-0ubuntu1) quantal-proposed; urgency=low

  * Ubuntu updates:
    - debian/control: Ensure novaclient is upgraded with nova,
      require python-keystoneclient >= 1:2.9.0. (LP: #1073289)
    - d/p/avoid_setuptools_git_dependency.patch: Refresh.
  * Dropped patches, applied upstream:
    - debian/patches/CVE-2012-5625.patch: [a99a802]
  * Resynchronize with stable/folsom (b55014ca) (LP: #1085255):
    - [a99a802] create_lvm_image allocates dirty blocks (LP: #1070539)
    - [670b388] RPC exchange name defaults to 'openstack' (LP: #1083944)
    - [3ede373] disassociate_floating_ip with multi_host=True fails
      (LP: #1074437)
    - [22d7c3b] libvirt imagecache should handle shared image storage
      (LP: #1075018)
    - [e787786] Detached and deleted RBD volumes remain associated with insance
      (LP: #1083818)
    - [9265eb0] live_migration missing migrate_data parameter in Hyper-V driver
      (LP: #1066513)
    - [3d99848] use_single_default_gateway does not function correctly
      (LP: #1075859)
    - [65a2d0a] resize does not migrate DHCP host information (LP: #1065440)
    - [102c76b] Nova backup image fails (LP: #1065053)
    - [48a3521] Fix config-file overrides for nova-dhcpbridge
    - [69663ee] Cloudpipe in Folsom: no such option: cnt_vpn_clients
      (LP: #1069573)
    - [6e47cc8] DisassociateAddress can cause Internal Server Error
      (LP: #1080406)
    - [22c3d7b] API calls to dis-associate an auto-assigned floating IP should
      return proper warning (LP: #1061499)
    - [bd11d15] libvirt: if exception raised during volume_detach, volume state
      is inconsistent (LP: #1057756)
    - [dcb59c3] admin can't describe all images in ec2 api (LP: #1070138)
    - [78de622] Incorrect Exception raised during Create server when metadata
      over 255 characters (LP: #1004007)
    - [c313de4] Fixed IP isn't released before updating DHCP host file
      (LP: #1078718)
    - [f4ab42d] Enabling Return Reservation ID with XML create server request
      returns no body (LP: #1061124)
    - [3db2a38] 'BackupCreate' should accept rotation parameter greater than or
      equal to zero (LP: #1071168)
    - [f7e5dde] libvirt reboot sometimes fails to reattach volumes
      (LP: #1073720)
    - [ff776d4] libvirt: detaching volume may fail while terminating other
      instances on the same host concurrently (LP: #1060836)
    - [85a8bc2] Used instance uuid rather than id in remove-fixed-ip
    - [42a85c0] Fix error on invalid delete_on_termination value
    - [6a17579] xenapi migrations fail w/ swap (LP: #1064083)
    - [97649b8] attach-time field for volumes is not updated for detach volume
      (LP: #1056122)
    - [8f6a718] libvirt: rebuild is not using kernel and ramdisk associated with
      the new image (LP: #1060925)
    - [fbe835f] live-migration and volume host assignement (LP: #1066887)
    - [c2a9150] typo prevents volume_tmp_dir flag from working (LP: #1071536)
    - [93efa21] Instances deleted during spawn leak network allocations
      (LP: #1068716)
    - [ebabd02] After restartin...

Read more...

Changed in nova (Ubuntu Quantal):
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-1 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.