OpenStack Compute (nova)

RequestSpec.flavor is not reverted on resize revert

  1. Series ocata
  2. Bug #1785339
Bug #1785339 reported by Matt Riedemann
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Matt Riedemann
Ocata
Fix Committed
Medium
Matt Riedemann
Pike
Fix Committed
Medium
Matt Riedemann
Queens
Fix Committed
Medium
Matt Riedemann
Rocky
Fix Committed
Medium
Matt Riedemann

Bug Description

When we resize a server, we update the RequestSpec.flavor because for later move operations of the server, the RequestSpec is what gets passed to the scheduler, so naturally we need the RequestSpec.flavor to match the instance.flavor after the instance is resized. That happens here:

https://github.com/openstack/nova/blob/4c37ff72e5446c835a48d569dd5a1416fcd36c71/nova/conductor/manager.py#L313

https://github.com/openstack/nova/blob/4c37ff72e5446c835a48d569dd5a1416fcd36c71/nova/conductor/manager.py#L352

However, if the resize is reverted, we don't actually rollback the RequestSpec.flavor to the previously flavor (instance.old_flavor):

https://github.com/openstack/nova/blob/4c37ff72e5446c835a48d569dd5a1416fcd36c71/nova/compute/api.py#L3322

So consider this scenario:

1. create server with flavor 1
2. resize to server 2 (VERIFY_RESIZE status); at this point the RequestSpec.flavor is 2
3. revert the resize
4. live migrate, shelve/unshelve, evacuate or cold migrate the server - the request spec passed to the scheduler will have flavor 2 rather than flavor 1 which could cause misleading scheduling failures, e.g. if flavor 2 is bigger than flavor 1 and there is no capacity for flavor 2 on any hosts, the scheduler would raise NoValidHost.

Revision history for this message
Matt Riedemann (mriedem) wrote :

This has been broken since newton:

https://github.com/openstack/nova/commit/74ab427d4796d8a386f84a15cc49188c2a60f8f1

Revision history for this message
Matt Riedemann (mriedem) wrote :

I noticed this while going through the starlingx-staging/stx-nova diff:

https://github.com/starlingx-staging/stx-nova/commit/71acfeae0d1c59fdc77704527d763bd85a276f9a#diff-378a96ec6159d0a2f8ec7ab71bc3843bR921

Matt Riedemann (mriedem)
Changed in nova:
assignee: nobody → Matt Riedemann (mriedem)
tags: added: starlingx
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/588689

Changed in nova:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/588689
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=ef3849e2da1c94df519debcb14ba15af9bd01a60
Submitter: Zuul
Branch: master

commit ef3849e2da1c94df519debcb14ba15af9bd01a60
Author: Matt Riedemann <email address hidden>
Date: Fri Aug 3 19:11:37 2018 -0400

    Update RequestSpec.flavor on resize_revert

    Since I8abdf58a6537dd5e15a012ea37a7b48abd726579 in Newton
    we update the RequestSpec.flavor to the new flavor during
    a resize. However, if the resize is reverted, we didn't
    revert the RequestSpec.flavor to the previous flavor
    on the instance, which could cause issues later when
    moving the instance since the scheduler will get a
    RequestSpec with a flavor that doesn't match the actual
    flavor on the instance.

    This fixes the bug by updating the RequestSpec.flavor
    with instance.old_flavor on resize_revert in the API.
    Functional test wrinkles are added.

    This fix was ported from the starlingx-staging/stx-nova
    repo commit 71acfeae0.

    Change-Id: Ic6e74702f2a5b57b437f4ffdfbc86c1e34cdac7d
    Closes-Bug: #1785339

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/602403

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/602409

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/rocky)

Reviewed: https://review.openstack.org/602403
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=9125fc7caf68fcecea47a3f0ac5068cf44a97821
Submitter: Zuul
Branch: stable/rocky

commit 9125fc7caf68fcecea47a3f0ac5068cf44a97821
Author: Matt Riedemann <email address hidden>
Date: Fri Aug 3 19:11:37 2018 -0400

    Update RequestSpec.flavor on resize_revert

    Since I8abdf58a6537dd5e15a012ea37a7b48abd726579 in Newton
    we update the RequestSpec.flavor to the new flavor during
    a resize. However, if the resize is reverted, we didn't
    revert the RequestSpec.flavor to the previous flavor
    on the instance, which could cause issues later when
    moving the instance since the scheduler will get a
    RequestSpec with a flavor that doesn't match the actual
    flavor on the instance.

    This fixes the bug by updating the RequestSpec.flavor
    with instance.old_flavor on resize_revert in the API.
    Functional test wrinkles are added.

    This fix was ported from the starlingx-staging/stx-nova
    repo commit 71acfeae0.

    Change-Id: Ic6e74702f2a5b57b437f4ffdfbc86c1e34cdac7d
    Closes-Bug: #1785339
    (cherry picked from commit ef3849e2da1c94df519debcb14ba15af9bd01a60)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 18.0.1

This issue was fixed in the openstack/nova 18.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/605879

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/605880

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/queens)

Reviewed: https://review.openstack.org/602409
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=f2d2a9a7c538f87775092c6cbf3b03b463cbdc2a
Submitter: Zuul
Branch: stable/queens

commit f2d2a9a7c538f87775092c6cbf3b03b463cbdc2a
Author: Matt Riedemann <email address hidden>
Date: Fri Aug 3 19:11:37 2018 -0400

    Update RequestSpec.flavor on resize_revert

    Since I8abdf58a6537dd5e15a012ea37a7b48abd726579 in Newton
    we update the RequestSpec.flavor to the new flavor during
    a resize. However, if the resize is reverted, we didn't
    revert the RequestSpec.flavor to the previous flavor
    on the instance, which could cause issues later when
    moving the instance since the scheduler will get a
    RequestSpec with a flavor that doesn't match the actual
    flavor on the instance.

    This fixes the bug by updating the RequestSpec.flavor
    with instance.old_flavor on resize_revert in the API.
    Functional test wrinkles are added.

    This fix was ported from the starlingx-staging/stx-nova
    repo commit 71acfeae0.

    Change-Id: Ic6e74702f2a5b57b437f4ffdfbc86c1e34cdac7d
    Closes-Bug: #1785339
    (cherry picked from commit ef3849e2da1c94df519debcb14ba15af9bd01a60)
    (cherry picked from commit 9125fc7caf68fcecea47a3f0ac5068cf44a97821)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/pike)

Reviewed: https://review.openstack.org/605879
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=556079a33997958d26fb7f03946998a6b06f488c
Submitter: Zuul
Branch: stable/pike

commit 556079a33997958d26fb7f03946998a6b06f488c
Author: Matt Riedemann <email address hidden>
Date: Fri Aug 3 19:11:37 2018 -0400

    Update RequestSpec.flavor on resize_revert

    Since I8abdf58a6537dd5e15a012ea37a7b48abd726579 in Newton
    we update the RequestSpec.flavor to the new flavor during
    a resize. However, if the resize is reverted, we didn't
    revert the RequestSpec.flavor to the previous flavor
    on the instance, which could cause issues later when
    moving the instance since the scheduler will get a
    RequestSpec with a flavor that doesn't match the actual
    flavor on the instance.

    This fixes the bug by updating the RequestSpec.flavor
    with instance.old_flavor on resize_revert in the API.
    Functional test wrinkles are added.

    This fix was ported from the starlingx-staging/stx-nova
    repo commit 71acfeae0.

    Conflicts:
          nova/tests/functional/test_servers.py

    NOTE(mriedem): The conflict is due to not having change
    I40a244601e1612a87bd41cd6ad652202035a1ce4 in Pike, which
    would be difficult to backport because of change
    I89e2682c9210901cf1992dac2f9068b51f0373cd not being in
    Pike, so it's easier to just update the existing tests
    as they were in Pike.

    Change-Id: Ic6e74702f2a5b57b437f4ffdfbc86c1e34cdac7d
    Closes-Bug: #1785339
    (cherry picked from commit ef3849e2da1c94df519debcb14ba15af9bd01a60)
    (cherry picked from commit 9125fc7caf68fcecea47a3f0ac5068cf44a97821)
    (cherry picked from commit f2d2a9a7c538f87775092c6cbf3b03b463cbdc2a)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/ocata)

Reviewed: https://review.openstack.org/605880
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c4ea58eb867ee90785101663325a92f1d771842a
Submitter: Zuul
Branch: stable/ocata

commit c4ea58eb867ee90785101663325a92f1d771842a
Author: Matt Riedemann <email address hidden>
Date: Fri Aug 3 19:11:37 2018 -0400

    Update RequestSpec.flavor on resize_revert

    Since I8abdf58a6537dd5e15a012ea37a7b48abd726579 in Newton
    we update the RequestSpec.flavor to the new flavor during
    a resize. However, if the resize is reverted, we didn't
    revert the RequestSpec.flavor to the previous flavor
    on the instance, which could cause issues later when
    moving the instance since the scheduler will get a
    RequestSpec with a flavor that doesn't match the actual
    flavor on the instance.

    This fixes the bug by updating the RequestSpec.flavor
    with instance.old_flavor on resize_revert in the API.
    Functional test wrinkles are added.

    This fix was ported from the starlingx-staging/stx-nova
    repo commit 71acfeae0.

    Conflicts:
          nova/compute/api.py
          nova/tests/functional/test_servers.py
          nova/tests/unit/compute/test_compute_api.py

    NOTE(mriedem): The compute API conflict is due to not
    having change I9269ffa2b80e48db96c622d0dc0817738854f602
    in Ocata. The unit test conflict was due to mox->mock
    conversion in Pike. The functional test changes didn't
    even exist in Ocata because they were introduced in Pike
    with change If7b02bcd8d77e94c7fb42b721792c1391bc0e3b7
    and wouldn't apply to Ocata, so they are removed here.

    Change-Id: Ic6e74702f2a5b57b437f4ffdfbc86c1e34cdac7d
    Closes-Bug: #1785339
    (cherry picked from commit ef3849e2da1c94df519debcb14ba15af9bd01a60)
    (cherry picked from commit 9125fc7caf68fcecea47a3f0ac5068cf44a97821)
    (cherry picked from commit f2d2a9a7c538f87775092c6cbf3b03b463cbdc2a)
    (cherry picked from commit 556079a33997958d26fb7f03946998a6b06f488c)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 16.1.6

This issue was fixed in the openstack/nova 16.1.6 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 15.1.5

This issue was fixed in the openstack/nova 15.1.5 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 17.0.7

This issue was fixed in the openstack/nova 17.0.7 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 19.0.0.0rc1

This issue was fixed in the openstack/nova 19.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.