OpenStack Compute (nova)

nova rebuild fails after two rebuild requests when ironic is used

Series mitaka
Bug #1564921

Bug #1564921 reported by Vladyslav Drok on 2016-04-01

This bug affects 4 people

	Status	Importance	Assigned to
Ironic	Won't Fix	Undecided	Vladyslav Drok
OpenStack Compute (nova)	Fix Released	Undecided	Vladyslav Drok
Mitaka	New	Undecided	Unassigned
Newton	Fix Committed	Undecided	Unassigned

Bug Description

First nova rebuild request passes fine, but further requests fail with the following message:

Instance b460e640-e601-4e68-b0e8-231e15201412 is already associated with a node, it cannot be associated with this other node 10c0b922-cb39-412e-849a-27e66042d4c0 (HTTP 409)", "code": 500, "details": " File \"/opt/stack/nova/nova/compute/manager.py\"

The reason for this is that nova tries to reshcedule an instance during rebuild, and in case of ironic, there can't be 2 nodes associated with the same instance_uuid.

This can be checked on devstack since change I0233f964d8f294f0ffd9edcb16b1aaf93486177f that introduced it with ironic virt driver and neutron.

Tags:

Vladyslav Drok (vdrok) on 2016-04-01

Changed in nova:
assignee:	nobody → Vladyslav Drok (vdrok)

Revision history for this message

Matt Riedemann (mriedem) wrote on 2016-04-04:

I'm not really sure what I0233f964d8f294f0ffd9edcb16b1aaf93486177f has to do with this.

tags:	added: ironic rebuild
Changed in nova:
status:	New → Confirmed

Vladyslav Drok (vdrok) on 2016-04-14

Changed in ironic:
assignee:	nobody → Vladyslav Drok (vdrok)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-04-14: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/306010

Changed in nova:
status:	Confirmed → In Progress
Changed in ironic:
status:	New → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-04-14: Fix proposed to ironic (master)

Fix proposed to branch: master
Review: https://review.openstack.org/306011

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-04-15: Change abandoned on ironic (master)

Change abandoned by Vladyslav Drok (<email address hidden>) on branch: master
Review: https://review.openstack.org/306011

Vladyslav Drok (vdrok) on 2016-04-15

Changed in ironic:
status:	In Progress → Won't Fix

Matt Riedemann (mriedem) on 2016-09-23

tags:

added: mitaka-backport-potential newton-backport-potential

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-09-24: Fix merged to nova (master)

Reviewed: https://review.openstack.org/306010
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=54b122caec1ae2418fc3e296be604d072cb5815a
Submitter: Jenkins
Branch: master

commit 54b122caec1ae2418fc3e296be604d072cb5815a
Author: Vladyslav Drok <email address hidden>
Date: Thu Apr 14 21:15:10 2016 +0300

Update instance node on rebuild only when it is recreate

    When using ironic virt driver, if scheduled_node is not specified
    in rebuild_instance compute manager method (as it happens in case
    of instance rebuild), the first ironic node is selected:

computes = ComputeNodeList.get_all_by_host(context, host, use_slave)
return computes[0]

    After the first rebuild, instance.node is updated to be this first
    ironic node, which causes subsequent rebuilds to fail, as virt driver
    tries to set instance_uuid on a newly selected ironic node and fails.

Closes-bug: #1564921
Change-Id: I2fe6e439135ba6aa4120735d030ced31081ef202

Changed in nova:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2016-11-04: Fix merged to nova (stable/newton)

Reviewed: https://review.openstack.org/375715
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=97d784b454baf3289ce2f3e1812fb94c8469b136
Submitter: Jenkins
Branch: stable/newton

commit 97d784b454baf3289ce2f3e1812fb94c8469b136
Author: Vladyslav Drok <email address hidden>
Date: Thu Apr 14 21:15:10 2016 +0300

Update instance node on rebuild only when it is recreate

    When using ironic virt driver, if scheduled_node is not specified
    in rebuild_instance compute manager method (as it happens in case
    of instance rebuild), the first ironic node is selected:

computes = ComputeNodeList.get_all_by_host(context, host, use_slave)
return computes[0]

    Closes-bug: #1564921
    Change-Id: I2fe6e439135ba6aa4120735d030ced31081ef202
    (cherry picked from commit 54b122caec1ae2418fc3e296be604d072cb5815a)