OpenStack Compute (nova)

newer `qemu-img info` causes in exception when finding the backing file for qcow2 images

Bug #1000261 reported by Pádraig Brady
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Pádraig Brady
OpenStack Compute (nova) 2012.2 "folsom"
Essex
Fix Released
Undecided
Unassigned
OpenStack Compute (nova) 2012.1.1
nova (Ubuntu)
Fix Released
Undecided
Unassigned
Ubuntu ubuntu-12.04.1

Bug Description

With a qemu-img updated with:
https://lists.gnu.org/archive/html/qemu-devel/2012-05/msg01468.html
you'll get this traceback on nova-compute startup

Patch coming up..

2012-05-16 14:17:26 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img info /var/lib/nova/instances/instance-00000017/disk from (pid=27483) execute /usr/lib/python2.6/site-packages/nova/utils.py:220
2012-05-16 14:17:26 CRITICAL nova [-] list index out of range
2012-05-16 14:17:26 TRACE nova Traceback (most recent call last):
2012-05-16 14:17:26 TRACE nova File "/usr/bin/nova-compute", line 49, in <module>
2012-05-16 14:17:26 TRACE nova service.wait()
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/nova/service.py", line 413, in wait
2012-05-16 14:17:26 TRACE nova _launcher.wait()
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/nova/service.py", line 131, in wait
2012-05-16 14:17:26 TRACE nova service.wait()
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 166, in wait
2012-05-16 14:17:26 TRACE nova return self._exit_event.wait()
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/eventlet/event.py", line 116, in wait
2012-05-16 14:17:26 TRACE nova return hubs.get_hub().switch()
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/eventlet/hubs/hub.py", line 177, in switch
2012-05-16 14:17:26 TRACE nova return self.greenlet.switch()
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 192, in main
2012-05-16 14:17:26 TRACE nova result = function(*args, **kwargs)
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/nova/service.py", line 101, in run_server
2012-05-16 14:17:26 TRACE nova server.start()
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/nova/service.py", line 174, in start
2012-05-16 14:17:26 TRACE nova self.manager.update_available_resource(ctxt)
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 2403, in update_available_resource
2012-05-16 14:17:26 TRACE nova self.driver.update_available_resource(context, self.host)
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/connection.py", line 1914, in update_available_resource
2012-05-16 14:17:26 TRACE nova 'disk_available_least': self.get_disk_available_least()}
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/connection.py", line 2266, in get_disk_available_least
2012-05-16 14:17:26 TRACE nova disk_infos = utils.loads(self.get_instance_disk_info(i_name))
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/connection.py", line 2236, in get_instance_disk_info
2012-05-16 14:17:26 TRACE nova backing_file = libvirt_utils.get_disk_backing_file(path)
2012-05-16 14:17:26 TRACE nova File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/utils.py", line 96, in get_disk_backing_file
2012-05-16 14:17:26 TRACE nova for i in out.split('\n') if 0 <= i.find('backing file')]
2012-05-16 14:17:26 TRACE nova IndexError: list index out of range

Related branches

lp://staging/~openstack-ubuntu-testing/nova/precise-essex-proposed
Chuck Short: Pending requested
Diff: 56 lines (+14/-4)
3 files modified
debian/changelog (+8/-0)
debian/control (+6/-3)
debian/nova-console.install (+0/-1)
lp://staging/ubuntu/precise-proposed/nova
lp://staging/ubuntu/precise-updates/nova

CVE References

Pádraig Brady (p-draigbrady)
summary: - newer `qemu=img info` causes in exception when finding the backing file
+ newer `qemu-img info` causes in exception when finding the backing file
for qcow2 images
Russell Bryant (russellb)
Changed in nova:
milestone: none → folsom-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/7485
Committed: http://github.com/openstack/nova/commit/0624b7aab0c0fe4869111ad8e302151548d6ba20
Submitter: Jenkins
Branch: master

commit 0624b7aab0c0fe4869111ad8e302151548d6ba20
Author: Pádraig Brady <email address hidden>
Date: Wed May 16 13:44:46 2012 +0100

    handle updated qemu-img info output

    Originally `qemu-img info` always output an (actual path: ...)
    even if it was duplicated with that already on the line.

     $ instances=/var/lib/nova/instances/
     $ qemu-img info $instances/instance-00000017/disk | grep 'backing'
     backing file: $instances/_base/24083... (actual path: $the_same)

    Whereas after the change referenced at:
    https://lists.gnu.org/archive/html/qemu-devel/2012-05/msg01468.html
    It suppresses a duplicate (actual path:)

     $ instances=/var/lib/nova/instances/
     $ qemu-img info $instances/instance-00000017/disk | grep 'backing'
     backing file: $instances/_base/24083...

    * nova/virt/libvirt/utils.py (get_disk_backing_file):
    Avoid an indexError exception when parsing the newer format.
    Fixes bug 1000261

    Change-Id: Ie2889b6da8a5c93e0e874e7a330529f6e6e71b0b

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/essex)

Fix proposed to branch: stable/essex
Review: https://review.openstack.org/7518

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/essex)

Reviewed: https://review.openstack.org/7518
Committed: http://github.com/openstack/nova/commit/2b3bbc49da7ea7eebbc046e746db1c085b08a425
Submitter: Jenkins
Branch: stable/essex

commit 2b3bbc49da7ea7eebbc046e746db1c085b08a425
Author: Pádraig Brady <email address hidden>
Date: Wed May 16 13:44:46 2012 +0100

    handle updated qemu-img info output

    Originally `qemu-img info` always output an (actual path: ...)
    even if it was duplicated with that already on the line.

     $ instances=/var/lib/nova/instances/
     $ qemu-img info $instances/instance-00000017/disk | grep 'backing'
     backing file: $instances/_base/24083... (actual path: $the_same)

    Whereas after the change referenced at:
    https://lists.gnu.org/archive/html/qemu-devel/2012-05/msg01468.html
    It suppresses a duplicate (actual path:)

     $ instances=/var/lib/nova/instances/
     $ qemu-img info $instances/instance-00000017/disk | grep 'backing'
     backing file: $instances/_base/24083...

    * nova/virt/libvirt/utils.py (get_disk_backing_file):
    Avoid an indexError exception when parsing the newer format.
    Fixes bug 1000261

    Change-Id: Ie2889b6da8a5c93e0e874e7a330529f6e6e71b0b
    (cherry picked from commit 0624b7aab0c0fe4869111ad8e302151548d6ba20)

tags: added: in-stable-essex
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/7556

Changed in nova:
assignee: Pádraig Brady (p-draigbrady) → Russell Bryant (russellb)
status: Fix Committed → In Progress
Russell Bryant (russellb)
Changed in nova:
assignee: Russell Bryant (russellb) → Pádraig Brady (p-draigbrady)
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/7556
Committed: http://github.com/openstack/nova/commit/8fbde4936df0c3521704dfc8629bb94579277f79
Submitter: Jenkins
Branch: master

commit 8fbde4936df0c3521704dfc8629bb94579277f79
Author: Russell Bryant <email address hidden>
Date: Thu May 17 16:15:29 2012 -0400

    Add unit test coverage for bug 1000261.

    The fix for bug 1000261 updated some parsing code to handle updated
    output from qemu-img. This test case exercises this parsing code.

    Change-Id: I5c9592478823464317cc6d521b3240913e4ded92

Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Chuck Short (zulcss)
Changed in nova (Ubuntu):
status: New → In Progress
Chuck Short (zulcss)
Changed in nova (Ubuntu):
milestone: none → ubuntu-12.04.1
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Pádraig, or anyone else affected,

Accepted nova into precise-proposed. The package will build now and be available in a few hours. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case details of your testing will help us make a better decision. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed
Revision history for this message
Adam Gandelman (gandelman-a) wrote :

Please find the attached Jenkins job results from the Ubuntu Server Team's CI
infrastructure. As part of the verification process for this bug, Nova has
been deployed and configured across multiple nodes using precise-proposed as
an installation source. After successful bring-up and configuration of the
cluster, a number of exercises and smoke tests have be invoked to ensure the
updated package did not introduce any regressions. A number of test iterations
were carried out to catch any possible transient errors.

Note the list of installed packages at the top and bottom of the report.

For records of upstream test coverage of this update, please see the
Jenkins links in the comments of the relevant upstream code-review:

https://review.openstack.org/7556

As per the provisional Micro Release Exception granted to this package by
the Technical Board, we hope this contributes toward verification of this
update.

Dave Walker (davewalker)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package nova - 2012.1+stable~20120612-3ee026e-0ubuntu1

---------------
nova (2012.1+stable~20120612-3ee026e-0ubuntu1) precise-proposed; urgency=low

  * New upstream snapshot. (LP: #1010473)
  * Dropped, superseeded by new snapshot:
    - debian/patches/upstream/0001-fix-bug-where-nova-ignores-glance-host-in-imageref.patch
    - debian/patches/upstream/0002-Stop-libvirt-test-from-deleting-instances-dir.patch
    - debian/patches/upstream/0003-Allow-unprivileged-RADOS-users-to-access-rbd-volumes.patch
    - debian/patches/upstream/0004-Fixed-bug-962840-added-a-test-case.patch
    - debian/patches/upstream/0005-Populate-image-properties-with-project_id-again.patch
    - debian/patches/upstream/0006-Use-project_id-in-ec2.cloud._format_image.patc
    - debian/patches/CVE-2012-2101.patch
    - debian/patches/CVE-2012-2654.patch
  * Resynchronize with stable/essex:
    - 3ee026e Only invoke .lower() on non-None protocols. (LP: #1010514)
    - f0a9f47 Create a utf8 version of the dns_domains table. (LP: #993663)
    - 84a43e1 Report memory correctly on Xen. (LP: #997014)
    - 8c72924 Add libvirt get_console_output tests: pty and file. (LP: #990237)
    - 4e423cd Fix Multi_Scheduler to process host capabilities. (LP: #1000403)
    - 4aea7f1 Nail pep8 dependencies to 1.0.1
    - 2b3bbc4 handle updated qemu-img info output. (LP: #1000261)
    - 2d7d51c Fix type of snapshot_id column to match db. (LP: #962615)
    - ec70c69 Generate a Changelog for Nova
    - e5e890f Fix nova.tests.test_nova_rootwrap on Fedora 17. (LP: #992916)
    - 9e9a554 Ec2 handle strings with "0x" (LP: #983206)
    - 26dc6b7 QuantumManager will start dnsmasq during startup. Fixes (LP: #977759)
    - 7028d66 Introduced flag base_dir_name. (LP: #973194)
    - 76b525a Get unit tests functional in OS X.
    - facb936 Update KillFilter to handle 'deleted' exe's. (LP: #967931)
    - 1209af4 Checks if value is string or not before decode. (LP: #952176)
    - 1209af4 Fix timeout in EC2 CloudController.create_image(). (LP: #989764)
    - 108e74b Re-add console_log from console_console_output(). (LP: #987335)
    - 48a0768 Don't leak RPC connections on timeouts or other exceptions. (LP: #968843)
    - 7c64de9 Cloudpipe tap vpn not always working. (LP: #975043)
    - 5ab5051 add libvirt_inject_key flag fix (LP: #971640)
    - 6c68ef5 Xen: Pass session to destroy_vdi. (LP: #988615)
    - 015744e Delete fixed_ips when network is deleted. (LP: #754900)
  * Add debian/scripts/changelog.sh to help generate the changelog.
  * Add debian/nova-common.docs:
    - Include changelog and README.rst
  * debian/rules: Generate a tarball from git snapshot.
  * debian/patches/fix-pep8-errors.patch: Fix pep8 errors due to pep8 upstream
    migration.
 -- Chuck Short <email address hidden> Tue, 05 Jun 2012 09:50:59 -0400

Changed in nova (Ubuntu):
status: In Progress → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: folsom-1 → 2012.2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.