Unshelve instance not working if instance is boot from volume

Bug #1404801 reported by Abhishek Kekane
22
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Pranali Deore
Juno
Fix Released
Medium
Pranali Deore

Bug Description

If instance is booted from volume, then shelving the instance sets the status as SHELVED_OFFLOADED, instance files are getting deleted properly from the base path. When you call the unshelve instance, it fails on the conductor with error "Unshelve attempted but the image_id is not provided", and instance goes in to error state.

Steps to reproduce:
-------------------

1. Log in to Horizon, create a new volume.
2. Create an Instance using newly created volume.
3. Verify instance is in active state.
$ source devstack/openrc demo demo
$ nova list
+--------------------------------------+------+--------+------------+-------------+------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+------------------+
| dae3a13b-6aa8-4794-93cd-5ab7bf90f604 | nova | ACTIVE | - | Running | private=10.0.0.3 |
+--------------------------------------+------+--------+------------+-------------+------------------+

4. Shelve the instance
$ nova shelve <instance-uuid>

5. Verify the status is SHELVED_OFFLOADED.
$ nova list
+--------------------------------------+------+-------------------+------------+-------------+------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+-------------------+------------+-------------+------------------+
| dae3a13b-6aa8-4794-93cd-5ab7bf90f604 | nova | SHELVED_OFFLOADED | - | Shutdown | private=10.0.0.3 |
+--------------------------------------+------+-------------------+------------+-------------+------------------+

6. Unshelve the instance.
$ nova unshelve <instance-uuid>

Following stack-trace logged in nova-conductor

2014-12-19 02:55:59.634 ERROR nova.conductor.manager [req-a071fbc9-1c23-4e7a-8adf-7b3d0951aadf demo demo] [instance: dae3a13b-6aa8-4794-93cd-5ab7bf90f604] Unshelve attempted but the image_id is not provided
2014-12-19 02:55:59.647 ERROR oslo.messaging.rpc.dispatcher [req-a071fbc9-1c23-4e7a-8adf-7b3d0951aadf demo demo] Exception during message handling: Error during unshelve instance dae3a13b-6aa8-4794-93cd-5ab7bf90f604: Unshelve attempted but the image_id is not provided
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 137, in _dispatch_and_reply
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher incoming.message))
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 180, in _dispatch
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 126, in _do_dispatch
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher File "/opt/stack/nova/nova/conductor/manager.py", line 727, in unshelve_instance
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher instance_id=instance.uuid, reason=reason)
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher UnshelveException: Error during unshelve instance dae3a13b-6aa8-4794-93cd-5ab7bf90f604: Unshelve attempted but the image_id is not provided
2014-12-19 02:55:59.647 TRACE oslo.messaging.rpc.dispatcher

7. Instance goes into error state.
$ nova list
+--------------------------------------+------+--------+------------+-------------+------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+------------------+
| dae3a13b-6aa8-4794-93cd-5ab7bf90f604 | nova | ERROR | unshelving | Shutdown | private=10.0.0.3 |
+--------------------------------------+------+--------+------------+-------------+------------------+

Note:
1. This issue is reproducible with admin as well as demo tenant.
2. In all cases of shelved_offload_time values (-1, 0, > 0) this issue is reproducible.

Tags: ntt
Changed in nova:
assignee: nobody → Abhishek Kekane (abhishek-kekane)
Joe Gordon (jogo)
Changed in nova:
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/144582

Changed in nova:
assignee: Abhishek Kekane (abhishek-kekane) → Pranali Deore (pranali-deore)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/144582
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c944babe99657093cc8210478deaae0142c98e96
Submitter: Jenkins
Branch: master

commit c944babe99657093cc8210478deaae0142c98e96
Author: PranaliDeore <email address hidden>
Date: Tue Dec 23 06:04:24 2014 -0800

    Unshelving a volume backed instance doesn't work

    In case of volume backed instance, snapshot is not
    taken when an instance is shelved, so shelve_image_id
    key is not set to the instance system metadata.
    If shelve_image_id is None, then it shouldn't raise UnshelveException.

    Closes-Bug: #1404801
    Change-Id: I295e3e2b2c2d640684a5416c53fa1fed7e6297e2

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/juno)

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/145725

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/juno)

Reviewed: https://review.openstack.org/145725
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=2dec248dc767ed0a9e064c3a61429387b5740bac
Submitter: Jenkins
Branch: stable/juno

commit 2dec248dc767ed0a9e064c3a61429387b5740bac
Author: PranaliDeore <email address hidden>
Date: Tue Dec 23 06:04:24 2014 -0800

    Unshelving a volume backed instance doesn't work

    In case of volume backed instance, snapshot is not
    taken when an instance is shelved, so shelve_image_id
    key is not set to the instance system metadata.
    If shelve_image_id is None, then it shouldn't raise UnshelveException.

    Note: The required unit-tests are manually added to the below path,
    as new path for unit-tests is not present in stable/juno release.
    nova/tests/conductor/test_conductor.py

    Conflicts:
            nova/conductor/manager.py
            nova/tests/unit/conductor/test_conductor.py

    Closes-Bug: #1404801
    Change-Id: I295e3e2b2c2d640684a5416c53fa1fed7e6297e2
    (cherry picked from commit c944babe99657093cc8210478deaae0142c98e96)

Thierry Carrez (ttx)
Changed in nova:
milestone: none → kilo-2
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: kilo-2 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.