OpenStack Compute (nova)

Transforming the RequestSpec object into legacy dicts doesn't support the requested_destination field

  1. Series newton
  2. Bug #1702454
Bug #1702454 reported by Sylvain Bauza
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Matt Riedemann
Newton
Fix Committed
High
Tony Breeds
Ocata
Fix Committed
High
Matt Riedemann
Pike
Fix Committed
Medium
Matt Riedemann

Bug Description

We added a new field in the RequestSpec object called 'requested_destination' and we began using it for evacuations by https://review.openstack.org/#/c/315572/ (Newton)

That object was tranformed into legacy dictionaries (called "filter_properties" and "request_spec") before being rehydrated for the rebuild_instance() method in the conductor service. That said, when transforming, we were forgetting about the 'requested_destination' field in the object so that when we were calling the scheduler, we were never using that field.

That bug was fixed implicitly by https://review.openstack.org/#/c/469037/ which is now merged in master, but the issue is still there in stable branches, and if you need to use the legacy methods, you'll not have it.

As a consequence, the feature to pass a destination for evacuation is not working in Newton and Ocata. Fortunately, given we didn't transformed the object into dicts before calling the scheduler for live-migrations, it does work for that action.

A proper resolution would be to make sure that we pass the requested_destination field into 'filter_properties' so that when transforming again into an object, we set again the field.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/481116

Changed in nova:
status: Confirmed → In Progress
Sylvain Bauza (sylvain-bauza)
tags: added: pike-rc-potential
Revision history for this message
Matt Riedemann (mriedem) wrote :

Why is this tagged as pike-rc-potential, it's a latent issue in stable branches, correct?

tags: added: evacuate
removed: pike-rc-potential
Matt Riedemann (mriedem)
tags: removed: rebuild
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/490997

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/ocata)

Related fix proposed to branch: stable/ocata
Review: https://review.openstack.org/491000

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/491001

OpenStack Infra (hudson-openstack)
Changed in nova:
assignee: Sylvain Bauza (sylvain-bauza) → Matt Riedemann (mriedem)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/pike)

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/516111

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/516112

Matt Riedemann (mriedem)
Changed in nova:
assignee: Matt Riedemann (mriedem) → Sylvain Bauza (sylvain-bauza)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/newton)

Related fix proposed to branch: stable/newton
Review: https://review.openstack.org/516118

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/516119

OpenStack Infra (hudson-openstack)
Changed in nova:
assignee: Sylvain Bauza (sylvain-bauza) → Matt Riedemann (mriedem)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (master)

Reviewed: https://review.openstack.org/490997
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6e332cb7f1f1b18d117685a0a0f5bb6a8e0c8e93
Submitter: Zuul
Branch: master

commit 6e332cb7f1f1b18d117685a0a0f5bb6a8e0c8e93
Author: Sylvain Bauza <email address hidden>
Date: Wed Aug 2 19:28:13 2017 +0200

    Functional regression test for evacuate with a target

    In I6ddcaaca37fc5387c2d2e9f51c67ea9e85acb5c5 we forgot to update the
    legacy filter properties dictionary so the requested target wasn't
    passed to the scheduler when evacuating.
    Adding a functional test for verifying the behaviour.

    NOTE(sbauza): The issue has been incendentally fixed in Pike by
    I434af8e4ad991ac114dd67d66797a562d16bafe2 so the regression test just
    verifies that the expected behaviour works.
    The Newton and Ocata backports will be slightly different from that one as we
    need to verify that host3 will be preferred eventually over host2.

    Related-Bug: #1702454

    Change-Id: Id9adb10d2ef821c8b61d8f1d5dc9dd66ec7aaac8

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/481116
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=7317afd3201c4f9f21d3f37dab08423dae236b17
Submitter: Zuul
Branch: master

commit 7317afd3201c4f9f21d3f37dab08423dae236b17
Author: Sylvain Bauza <email address hidden>
Date: Thu Jul 6 17:17:31 2017 +0200

    Pass requested_destination in filter_properties

    When we added the requested_destination field for the RequestSpec object
    in Newton, we forgot to pass it to the legacy dictionary when wanting to
    use scheduler methods not yet supporting the NovaObject.
    As a consequence, when we were transforming the RequestSpec object into a
    tuple of (request_spec, filter_props) dicts and then rehydrating a new
    RequestSpec object using those dicts, the newly created object was not
    keeping that requested_destination field from the original.

    Change-Id: Iba0b88172e9a3bfd4f216dd364d70f7e01c60ee2
    Closes-Bug: #1702454

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/ocata)

Reviewed: https://review.openstack.org/491000
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0b672420bceaf5a6e264a9362eeb6e29badd09d8
Submitter: Zuul
Branch: stable/ocata

commit 0b672420bceaf5a6e264a9362eeb6e29badd09d8
Author: Sylvain Bauza <email address hidden>
Date: Wed Aug 2 19:28:13 2017 +0200

    Functional regression test for evacuate with a target

    In I6ddcaaca37fc5387c2d2e9f51c67ea9e85acb5c5 we forgot to update the
    legacy filter properties dictionary so the requested target wasn't
    passed to the scheduler when evacuating.
    Adding a functional test for verifying the behaviour.

    NOTE(sbauza): The issue has been incendentally fixed in Pike by
    I434af8e4ad991ac114dd67d66797a562d16bafe2 so the regression test just
    verifies that the expected behaviour works.
    That specific backport is consequently different from the master change
    as the final target is different from the one we expected.

    Related-Bug: #1702454

    Change-Id: Id9adb10d2ef821c8b61d8f1d5dc9dd66ec7aaac8
    (cherry picked from commit e0e2e065a495b4fa9ebdec987c935e3c83118c46)
    (cherry picked from commit b575a2645137d0e8f1f622ee45fca89605d2fdb9)

tags: added: in-stable-ocata
tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/pike)

Reviewed: https://review.openstack.org/516111
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=de27ede60dd66f7c4b76fe39c63c0305424a7903
Submitter: Zuul
Branch: stable/pike

commit de27ede60dd66f7c4b76fe39c63c0305424a7903
Author: Sylvain Bauza <email address hidden>
Date: Wed Aug 2 19:28:13 2017 +0200

    Functional regression test for evacuate with a target

    In I6ddcaaca37fc5387c2d2e9f51c67ea9e85acb5c5 we forgot to update the
    legacy filter properties dictionary so the requested target wasn't
    passed to the scheduler when evacuating.
    Adding a functional test for verifying the behaviour.

    NOTE(sbauza): The issue has been incendentally fixed in Pike by
    I434af8e4ad991ac114dd67d66797a562d16bafe2 so the regression test just
    verifies that the expected behaviour works.
    The Newton and Ocata backports will be slightly different from that one as we
    need to verify that host3 will be preferred eventually over host2.

    Related-Bug: #1702454

    Change-Id: Id9adb10d2ef821c8b61d8f1d5dc9dd66ec7aaac8
    (cherry picked from commit e0e2e065a495b4fa9ebdec987c935e3c83118c46)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/pike)

Reviewed: https://review.openstack.org/516112
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0c0023d0759f54f5bd99d9ed29c2a2ccb3965cba
Submitter: Zuul
Branch: stable/pike

commit 0c0023d0759f54f5bd99d9ed29c2a2ccb3965cba
Author: Sylvain Bauza <email address hidden>
Date: Thu Jul 6 17:17:31 2017 +0200

    Pass requested_destination in filter_properties

    When we added the requested_destination field for the RequestSpec object
    in Newton, we forgot to pass it to the legacy dictionary when wanting to
    use scheduler methods not yet supporting the NovaObject.
    As a consequence, when we were transforming the RequestSpec object into a
    tuple of (request_spec, filter_props) dicts and then rehydrating a new
    RequestSpec object using those dicts, the newly created object was not
    keeping that requested_destination field from the original.

    Change-Id: Iba0b88172e9a3bfd4f216dd364d70f7e01c60ee2
    Closes-Bug: #1702454
    (cherry picked from commit 69bef428bd555bb31f43db6ca9c21db8aeb9007e)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/ocata)

Reviewed: https://review.openstack.org/491001
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=fe8acf003e8e4d26f02ec38034c4715dfb9888ba
Submitter: Zuul
Branch: stable/ocata

commit fe8acf003e8e4d26f02ec38034c4715dfb9888ba
Author: Sylvain Bauza <email address hidden>
Date: Thu Jul 6 17:17:31 2017 +0200

    Pass requested_destination in filter_properties

    When we added the requested_destination field for the RequestSpec object
    in Newton, we forgot to pass it to the legacy dictionary when wanting to
    use scheduler methods not yet supporting the NovaObject.
    As a consequence, when we were transforming the RequestSpec object into a
    tuple of (request_spec, filter_props) dicts and then rehydrating a new
    RequestSpec object using those dicts, the newly created object was not
    keeping that requested_destination field from the original.

    Conflicts:
          nova/tests/unit/objects/test_request_spec.py

    NOTE(mriedem): Conflict is due to d288132dca7cc76dfc6679eda17bb8fcc62577de
    not being in Ocata, but it's not needed either.

    Change-Id: Iba0b88172e9a3bfd4f216dd364d70f7e01c60ee2
    Closes-Bug: #1702454
    (cherry picked from commit 69bef428bd555bb31f43db6ca9c21db8aeb9007e)
    (cherry picked from commit bceb176fb00104fb0e39d6f1f3183f616c2b90f7)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/newton)

Reviewed: https://review.openstack.org/516118
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=32094105285daa7c420189f73025dd24437843bb
Submitter: Zuul
Branch: stable/newton

commit 32094105285daa7c420189f73025dd24437843bb
Author: Sylvain Bauza <email address hidden>
Date: Wed Aug 2 19:28:13 2017 +0200

    Functional regression test for evacuate with a target

    In I6ddcaaca37fc5387c2d2e9f51c67ea9e85acb5c5 we forgot to update the
    legacy filter properties dictionary so the requested target wasn't
    passed to the scheduler when evacuating.
    Adding a functional test for verifying the behaviour.

    NOTE(sbauza): The issue has been incendentally fixed in Pike by
    I434af8e4ad991ac114dd67d66797a562d16bafe2 so the regression test just
    verifies that the expected behaviour works.
    That specific backport is consequently different from the master change
    as the final target is different from the one we expected.

    Related-Bug: #1702454

    NOTE(mriedem): There are a few changes needed in newton to make
    the test work, like there is no PlacementFixture, and some of the
    configs changed.

    Change-Id: Id9adb10d2ef821c8b61d8f1d5dc9dd66ec7aaac8
    (cherry picked from commit e0e2e065a495b4fa9ebdec987c935e3c83118c46)
    (cherry picked from commit b575a2645137d0e8f1f622ee45fca89605d2fdb9)
    (cherry picked from commit 2f9ce628a527c5cffb6cde5b9ef6490c0d4af60c)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 15.0.8

This issue was fixed in the openstack/nova 15.0.8 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 16.0.3

This issue was fixed in the openstack/nova 16.0.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/newton)

Reviewed: https://review.openstack.org/516119
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6ce8bcab7cd73e27adfd26fceee7ec7c397bfa9b
Submitter: Zuul
Branch: stable/newton

commit 6ce8bcab7cd73e27adfd26fceee7ec7c397bfa9b
Author: Sylvain Bauza <email address hidden>
Date: Thu Jul 6 17:17:31 2017 +0200

    Pass requested_destination in filter_properties

    When we added the requested_destination field for the RequestSpec object
    in Newton, we forgot to pass it to the legacy dictionary when wanting to
    use scheduler methods not yet supporting the NovaObject.
    As a consequence, when we were transforming the RequestSpec object into a
    tuple of (request_spec, filter_props) dicts and then rehydrating a new
    RequestSpec object using those dicts, the newly created object was not
    keeping that requested_destination field from the original.

    Conflicts:
          nova/objects/request_spec.py

    NOTE(mriedem): The conflict is due to not having change
    b26ef568077f1bd9af8076c362bad06da3ec7e43 in Newton.

    Conflicts:
          nova/tests/unit/objects/test_request_spec.py

    NOTE(tonyb): Due to having Icb418f2be575bb2ba82756fdeb67b24a28950746

    Depends-On: I344d8fdded9b7d5385fcb41b699f1352acb4cda7

    Change-Id: Iba0b88172e9a3bfd4f216dd364d70f7e01c60ee2
    Closes-Bug: #1702454
    (cherry picked from commit 69bef428bd555bb31f43db6ca9c21db8aeb9007e)
    (cherry picked from commit bceb176fb00104fb0e39d6f1f3183f616c2b90f7)
    (cherry picked from commit 841121a95d76b10e6738156b55ee8ebc648054b4)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 14.0.10

This issue was fixed in the openstack/nova 14.0.10 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 17.0.0.0b2

This issue was fixed in the openstack/nova 17.0.0.0b2 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.