VMwareAPI-Team

VMware: launching instance from instance snapshot fails

Bug #1247296 reported by Ryan Hsu
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Maithem
OpenStack Compute (nova) 2014.1 "icehouse"
Havana
Fix Released
High
Tracy Jones
OpenStack Compute (nova) 2013.2.3
VMwareAPI-Team
Fix Committed
High
Maithem

Bug Description

branch: stable/havana

When using the nova VMwareVCDriver, instances launched from instance snapshots are spawning with ERROR.

Steps to Reproduce:
(using Horizon)
1. Launch an instance using a vmdk image
2. Take a snapshot of the instance
3. Launch an instance using the snapshot

Expected result: Instance should boot up successfully
Actual result: Instance boots up with ERROR and following error is found in n-cpu.log:

 Traceback (most recent call last):
   File "/opt/stack/nova/nova/compute/manager.py", line 1407, in _spawn
     block_device_info)
   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 623, in spawn
     admin_password, network_info, block_device_info)
   File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 449, in spawn
     uploaded_vmdk_path)
   File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 387, in
     self._session._wait_for_task(instance['uuid'], vmdk_copy_task)
   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 900, in
     ret_val = done.wait()
   File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116,
     return hubs.get_hub().switch()
   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line
     return self.greenlet.switch()
 NovaException: A specified parameter was not correct.
 fileType

The error seems to occur during the CopyVirtualDisk_Task operation. Full log for context is available here: http://paste.openstack.org/show/50411/

See original description
Ryan Hsu (rhsu)
description: updated
description: updated
Revision history for this message
Gary Kotton (garyk) wrote :

Which image did you use? sparse?
Thanks
Gary

Revision history for this message
Gary Kotton (garyk) wrote :

I have confirmed the problem. The issue is when a sparse disk has been snapshotted. Steps to reprodue:

1. glance image-create --name="dos" --disk-format=vmdk --container-format=bare --is-public=true --property vmware_adaptertype="lsiLogic" --property vmware_disktype="sparse" < nostalgia-disk1.vmdk
2. nova boot --image dos --flVOR 1 Y
3. nova image-create Y YY
4. nova boot --image YY --flavor 1 X

The upload to glance should have the relevant flags set.

Changed in nova:
status: New → Confirmed
Maithem (maithem)
Changed in nova:
assignee: nobody → Maithem (maithem)
Revision history for this message
Vui Lam (vui) wrote :

Yep, this should be straightforward Maithem. The vmware_disktype property should be preallocated (or unset I guess since it is the default) when uploading to glance.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/55928

Changed in nova:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/55929

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/55930

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/55934

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/56113

Tracy Jones (tjones-i)
Changed in nova:
importance: Undecided → High
Mark McLoughlin (markmc)
tags: added: havana-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/55934
Committed: http://github.com/openstack/nova/commit/11e327f888a6dbde990d75196560083fddecf394
Submitter: Jenkins
Branch: master

commit 11e327f888a6dbde990d75196560083fddecf394
Author: Maithem <email address hidden>
Date: Mon Nov 11 16:08:02 2013 -0800

    VMware: Always upload a snapshot as a preallocated disk

    Currently, a snapshot operation always produces a preallocated
    disk type. When the snapshot is uploaded to glance, the source
    image metadata is uploaded with the new snapshot and then some
    of the metadata properties are updated. The update doesn't
    include the disk_type property, this can be problematic. If
    the original image has a sparse disk type and a snapshot is
    taken, then the snapshot which is a preallocated disk type is
    uploaded to glance with the disk_type property set to sparse,
    which creates a disk type mismatch and causes errors when a vm
    is spawned from that snapshot. This patch addresses the issue
    by correcting the image metadata used when uploading a snapshot
    to glance.

    Also, the image_upload function in vmwareapi/fake.py has been
    modified to address the change. Any function call to
    image_upload (from the unit tests) will assert that the
    metadata for the image to be uploaded must have the disk_type
    value to be preallocated.

    Closes-Bug: 1247296
    Change-Id: I71cfa8d4d1eb7911abb489153292fb0d1e614765

Changed in nova:
status: In Progress → Fix Committed
Russell Bryant (russellb)
Changed in nova:
milestone: none → icehouse-2
Tracy Jones (tjones-i)
Changed in openstack-vmwareapi-team:
importance: Undecided → High
status: New → Fix Committed
Maithem (maithem)
Changed in openstack-vmwareapi-team:
assignee: nobody → Maithem (maithem)
Thierry Carrez (ttx)
Changed in nova:
status: Fix Committed → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/74859

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/havana)

Reviewed: https://review.openstack.org/74859
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=9520b8dfc207d0183ef53e84a480270e832d6c70
Submitter: Jenkins
Branch: stable/havana

commit 9520b8dfc207d0183ef53e84a480270e832d6c70
Author: Maithem <email address hidden>
Date: Mon Nov 11 16:08:02 2013 -0800

    VMware: Always upload a snapshot as a preallocated disk

    Currently, a snapshot operation always produces a preallocated
    disk type. When the snapshot is uploaded to glance, the source
    image metadata is uploaded with the new snapshot and then some
    of the metadata properties are updated. The update doesn't
    include the disk_type property, this can be problematic. If
    the original image has a sparse disk type and a snapshot is
    taken, then the snapshot which is a preallocated disk type is
    uploaded to glance with the disk_type property set to sparse,
    which creates a disk type mismatch and causes errors when a vm
    is spawned from that snapshot. This patch addresses the issue
    by correcting the image metadata used when uploading a snapshot
    to glance.

    Also, the image_upload function in vmwareapi/fake.py has been
    modified to address the change. Any function call to
    image_upload (from the unit tests) will assert that the
    metadata for the image to be uploaded must have the disk_type
    value to be preallocated.

    Closes-Bug: 1247296
    (cherry picked from commit 11e327f888a6dbde990d75196560083fddecf394)

    Conflicts:

     nova/tests/virt/vmwareapi/test_vmwareapi.py
    Change-Id: I71cfa8d4d1eb7911abb489153292fb0d1e614765

tags: added: in-stable-havana
Thierry Carrez (ttx)
Changed in nova:
milestone: icehouse-2 → 2014.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.