OpenStack Compute (nova)

Resizing a Xen instance with attached volumes fails

Bug #1028092 reported by Chuck Thier
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Chris Behrens
OpenStack Compute (nova) 2013.1 "grizzly"
Folsom
Fix Released
Medium
Chuck Short
OpenStack Compute (nova) 2012.2.1
nova (Ubuntu)
Fix Released
Undecided
Unassigned
Quantal
Fix Released
Undecided
Unassigned

Bug Description

When a Xen instance is resized with volumes attached it goes to an ERROR state, with the following error in the logs

['SR_OPERATION_NOT_SUPPORTED', 'OpaqueRef:b96e08b6-c3ad-2e4e-f093-cdc33677b435']
  File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 204, in decorated_function
    return function(self, context, instance_uuid, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 1533, in resize_instance
    self._set_instance_error_state(context, instance_uuid)
  File "/usr/lib/python2.6/contextlib.py", line 23, in __exit__
    self.gen.next()
  File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 1528, in resize_instance
    image)
  File "/usr/lib/python2.6/dist-packages/nova/compute/manager.py", line 1503, in _resize_instance
    instance_type_ref, self._legacy_nw_info(network_info))
  File "/usr/lib/python2.6/dist-packages/nova/virt/xenapi/driver.py", line 234, in migrate_disk_and_power_off
    dest, instance_type)
  File "/usr/lib/python2.6/dist-packages/nova/virt/xenapi/vmops.py", line 742, in migrate_disk_and_power_off
    self._session, instance, vm_ref, label) as vdi_uuids:
  File "/usr/lib/python2.6/contextlib.py", line 16, in __enter__
    return self.gen.next()
  File "/usr/lib/python2.6/dist-packages/nova/virt/xenapi/vm_utils.py", line 490, in snapshot_attached_here
    session, instance, vm_ref, label)
  File "/usr/lib/python2.6/dist-packages/nova/virt/xenapi/vm_utils.py", line 512, in _create_snapshot
    template_vm_ref = session.call_xenapi('VM.snapshot', vm_ref, label)
  File "/usr/lib/python2.6/dist-packages/nova/virt/xenapi/driver.py", line 589, in call_xenapi
    return session.xenapi_request(method, args)
  File "/usr/local/lib/python2.6/dist-packages/XenAPI.py", line 133, in xenapi_request
    result = _parse_result(getattr(self, methodname)(*full_params))
  File "/usr/local/lib/python2.6/dist-packages/XenAPI.py", line 203, in _parse_result
    raise Failure(result['ErrorDescription'])

Renuka Apte (renukaapte)
Changed in nova:
assignee: nobody → Renuka Apte (renukaapte)
Renuka Apte (renukaapte)
Changed in nova:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/11906

Thierry Carrez (ttx)
Changed in cinder:
milestone: none → folsom-rc1
Revision history for this message
John Griffith (john-griffith) wrote :

I believe everything needed to fix this is localized to Nova and therefore does not affect Cinder directly.

Renuka, if I'm incorrect let me know.

Revision history for this message
Renuka Apte (renukaapte) wrote :

That is correct.

Revision history for this message
John Griffith (john-griffith) wrote :

Needed changes are only in the xen virt side of the house, nothing to do for Cinder here.

Changed in cinder:
status: New → Invalid
Thierry Carrez (ttx)
no longer affects: cinder
John Garbutt (johngarbutt)
Changed in nova:
importance: Undecided → Medium
Revision history for this message
John Garbutt (johngarbutt) wrote :

This will also affect migration, Renuka is away this week.
I will see if I can pick up this one.

tags: added: xenserver
OpenStack Infra (hudson-openstack)
Changed in nova:
assignee: Renuka Apte (renukaapte) → clayg (clay-gerrard)
Chris Behrens (cbehrens)
Changed in nova:
assignee: clayg (clay-gerrard) → Chris Behrens (cbehrens)
Vish Ishaya (vishvananda)
tags: added: folsom-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/11906
Committed: http://github.com/openstack/nova/commit/359527501604ea720b594a2ed1b32cd29fe6adbc
Submitter: Jenkins
Branch: master

commit 359527501604ea720b594a2ed1b32cd29fe6adbc
Author: Renuka Apte <email address hidden>
Date: Tue Sep 4 23:02:16 2012 -0700

    xenapi: Refactor snapshots during resize

    Currently, we use VM.snapshot for resize, which fails if we have a
    volume attached to the VM, which does not support snapshots. This change
    uses VDI.snapshot instead, for all VDIs that are not attached by nova.

    Also needed for xenapi: detaching and reattaching volumes during
    migrations and reverting of migrations.

    Fixes Bug #1028092

    Change-Id: I3e2973747135a9c33de194e38537620c397bb87e

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/folsom)

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/14758

Chuck Short (zulcss)
tags: removed: folsom-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/folsom)

Reviewed: https://review.openstack.org/14758
Committed: http://github.com/openstack/nova/commit/d5888f19c3787511b352e7d54ce2ee923ed13ae8
Submitter: Jenkins
Branch: stable/folsom

commit d5888f19c3787511b352e7d54ce2ee923ed13ae8
Author: Renuka Apte <email address hidden>
Date: Tue Sep 4 23:02:16 2012 -0700

    xenapi: Refactor snapshots during resize

    Currently, we use VM.snapshot for resize, which fails if we have a
    volume attached to the VM, which does not support snapshots. This change
    uses VDI.snapshot instead, for all VDIs that are not attached by nova.

    Also needed for xenapi: detaching and reattaching volumes during
    migrations and reverting of migrations.

    Fixes Bug #1028092

    Change-Id: I3e2973747135a9c33de194e38537620c397bb87e
    (cherry picked from commit 359527501604ea720b594a2ed1b32cd29fe6adbc)

Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-1
status: Fix Committed → Fix Released
Adam Gandelman (gandelman-a)
Changed in nova (Ubuntu):
status: New → Fix Released
Changed in nova (Ubuntu Quantal):
status: New → Confirmed
Revision history for this message
Clint Byrum (clint-fewbar) wrote : Please test proposed package

Hello Chuck, or anyone else affected,

Accepted nova into quantal-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/nova/2012.2.1+stable-20121212-a99a802e-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in nova (Ubuntu Quantal):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (8.3 KiB)

This bug was fixed in the package nova - 2012.2.1+stable-20121212-a99a802e-0ubuntu1

---------------
nova (2012.2.1+stable-20121212-a99a802e-0ubuntu1) quantal-proposed; urgency=low

  * Ubuntu updates:
    - debian/control: Ensure novaclient is upgraded with nova,
      require python-keystoneclient >= 1:2.9.0. (LP: #1073289)
    - d/p/avoid_setuptools_git_dependency.patch: Refresh.
  * Dropped patches, applied upstream:
    - debian/patches/CVE-2012-5625.patch: [a99a802]
  * Resynchronize with stable/folsom (b55014ca) (LP: #1085255):
    - [a99a802] create_lvm_image allocates dirty blocks (LP: #1070539)
    - [670b388] RPC exchange name defaults to 'openstack' (LP: #1083944)
    - [3ede373] disassociate_floating_ip with multi_host=True fails
      (LP: #1074437)
    - [22d7c3b] libvirt imagecache should handle shared image storage
      (LP: #1075018)
    - [e787786] Detached and deleted RBD volumes remain associated with insance
      (LP: #1083818)
    - [9265eb0] live_migration missing migrate_data parameter in Hyper-V driver
      (LP: #1066513)
    - [3d99848] use_single_default_gateway does not function correctly
      (LP: #1075859)
    - [65a2d0a] resize does not migrate DHCP host information (LP: #1065440)
    - [102c76b] Nova backup image fails (LP: #1065053)
    - [48a3521] Fix config-file overrides for nova-dhcpbridge
    - [69663ee] Cloudpipe in Folsom: no such option: cnt_vpn_clients
      (LP: #1069573)
    - [6e47cc8] DisassociateAddress can cause Internal Server Error
      (LP: #1080406)
    - [22c3d7b] API calls to dis-associate an auto-assigned floating IP should
      return proper warning (LP: #1061499)
    - [bd11d15] libvirt: if exception raised during volume_detach, volume state
      is inconsistent (LP: #1057756)
    - [dcb59c3] admin can't describe all images in ec2 api (LP: #1070138)
    - [78de622] Incorrect Exception raised during Create server when metadata
      over 255 characters (LP: #1004007)
    - [c313de4] Fixed IP isn't released before updating DHCP host file
      (LP: #1078718)
    - [f4ab42d] Enabling Return Reservation ID with XML create server request
      returns no body (LP: #1061124)
    - [3db2a38] 'BackupCreate' should accept rotation parameter greater than or
      equal to zero (LP: #1071168)
    - [f7e5dde] libvirt reboot sometimes fails to reattach volumes
      (LP: #1073720)
    - [ff776d4] libvirt: detaching volume may fail while terminating other
      instances on the same host concurrently (LP: #1060836)
    - [85a8bc2] Used instance uuid rather than id in remove-fixed-ip
    - [42a85c0] Fix error on invalid delete_on_termination value
    - [6a17579] xenapi migrations fail w/ swap (LP: #1064083)
    - [97649b8] attach-time field for volumes is not updated for detach volume
      (LP: #1056122)
    - [8f6a718] libvirt: rebuild is not using kernel and ramdisk associated with
      the new image (LP: #1060925)
    - [fbe835f] live-migration and volume host assignement (LP: #1066887)
    - [c2a9150] typo prevents volume_tmp_dir flag from working (LP: #1071536)
    - [93efa21] Instances deleted during spawn leak network allocations
      (LP: #1068716)
    - [ebabd02] After restartin...

Read more...

Changed in nova (Ubuntu Quantal):
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-1 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.