OpenStack Compute (nova)

openstack server list --deleted --limit -1 hangs

  1. Series ocata
  2. Bug #1849409
Bug #1849409 reported by Keith Berger
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Matt Riedemann
Ocata
Fix Committed
High
Matt Riedemann
Pike
Fix Released
High
Matt Riedemann
Queens
Fix Released
High
Matt Riedemann
Rocky
Fix Committed
High
Matt Riedemann
Stein
Fix Committed
High
Matt Riedemann
Train
Fix Committed
High
Matt Riedemann

Bug Description

OpenStack Rocky:

When running `openstack server list --deleted --limit -1 hangs` it will hang and not return.

A debug output is found in this pastebin

http://paste.openstack.org/show/785497/

and a direct curl is shown here

http://paste.openstack.org/show/785488/

It seems to be related to the marker as discussed here.

http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-10-22.log.html#t2019-10-22T20:35:11

after the initial collection of 18 servers, for some reason it than tried to grab with

https://192.168.23.35:8774/v2.1/428982d4248a419a933668b6a4dd14a0/servers/dtail?deleted=True&marker=8aab9854-af8e-4e98-840a-192f15ae01f9

where the marker is the 18th deleted server in the list. it then loops on this until you breakout.

Revision history for this message
Matt Riedemann (mriedem) wrote :

I've got the fix started here: https://review.opendev.org/#/c/690429/

Changed in nova:
assignee: nobody → Matt Riedemann (mriedem)
status: New → Triaged
importance: Undecided → High
tags: added: api
OpenStack Infra (hudson-openstack)
Changed in nova:
status: Triaged → In Progress
Revision history for this message
Matt Riedemann (mriedem) wrote :

The regression was introduced with https://review.opendev.org/#/q/I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab which started in Queens and was backported to Newton. We can only fix upstream going back to Ocata though since the newton branch is end of life upstream.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (master)

Reviewed: https://review.opendev.org/690428
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=45c2752f2ce08b314012eff044b01aa7d626b43d
Submitter: Zuul
Branch: master

commit 45c2752f2ce08b314012eff044b01aa7d626b43d
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/690710

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/690711

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/stein)

Related fix proposed to branch: stable/stein
Review: https://review.opendev.org/690715

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/stein)

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/690716

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/rocky)

Related fix proposed to branch: stable/rocky
Review: https://review.opendev.org/690720

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/690721

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/queens)

Related fix proposed to branch: stable/queens
Review: https://review.opendev.org/690724

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.opendev.org/690725

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/pike)

Related fix proposed to branch: stable/pike
Review: https://review.opendev.org/690734

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.opendev.org/690735

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/ocata)

Related fix proposed to branch: stable/ocata
Review: https://review.opendev.org/690746

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.opendev.org/690747

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.opendev.org/690429
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=df03499843aa7fd6089bd4d07b9d0eb5a8c14b47
Submitter: Zuul
Branch: master

commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/train)

Reviewed: https://review.opendev.org/690710
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=727d942b2830fb6970d99507f2b5eb1a28df01b2
Submitter: Zuul
Branch: stable/train

commit 727d942b2830fb6970d99507f2b5eb1a28df01b2
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/train)

Reviewed: https://review.opendev.org/690711
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=03a2508362ecf50463d0659142312a30d0fb91f3
Submitter: Zuul
Branch: stable/train

commit 03a2508362ecf50463d0659142312a30d0fb91f3
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/stein)

Reviewed: https://review.opendev.org/690715
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=47caaccd4a03660d7df144f2eadd821d36baeaa8
Submitter: Zuul
Branch: stable/stein

commit 47caaccd4a03660d7df144f2eadd821d36baeaa8
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)

tags: added: in-stable-stein
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/stein)

Reviewed: https://review.opendev.org/690716
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6038455e1dafeaf8ccfdd4be9e993beea3c9fb45
Submitter: Zuul
Branch: stable/stein

commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 20.0.1

This issue was fixed in the openstack/nova 20.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/rocky)

Reviewed: https://review.opendev.org/690720
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=08337cccb060d0b3cad388004c1f802d5d053813
Submitter: Zuul
Branch: stable/rocky

commit 08337cccb060d0b3cad388004c1f802d5d053813
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    NOTE(mriedem): In this backport we get the PlacementFixture from
    the usual nova_fixtures import since change
    Idaed39629095f86d24a54334c699a26c218c6593 is not in Rocky. Also,
    test_concurrent_request_server_group_members_over_quota leaks deleted
    servers from cell0 for some reason in Rocky which can make this test
    intermittently fail. This is not a problem after Rocky for some reason.
    To avoid intermittent failures due to that leak, the test is modified
    to filter on the name of the server created in the test only.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)
    (cherry picked from commit 47caaccd4a03660d7df144f2eadd821d36baeaa8)

tags: added: in-stable-rocky
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/rocky)

Reviewed: https://review.opendev.org/690721
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c84846dbf93588e5420b3565d163c0393443e6d6
Submitter: Zuul
Branch: stable/rocky

commit c84846dbf93588e5420b3565d163c0393443e6d6
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Conflicts:

      nova/tests/functional/regressions/test_bug_1849409.py

    NOTE(mriedem): The conflict is due to the Rocky-specific change
    to the functional test in I324193129acb6ac739133c7e76920762a8987a84
    due to test_concurrent_request_server_group_members_over_quota leaking
    deleted servers for some reason.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)
    (cherry picked from commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/queens)

Reviewed: https://review.opendev.org/690724
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=f03f5075e3751ccd03a60999eab1d7f4bf7c4f02
Submitter: Zuul
Branch: stable/queens

commit f03f5075e3751ccd03a60999eab1d7f4bf7c4f02
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    NOTE(mriedem): In this backport the test is modified to create the
    server without networking to avoid a NetworkAmbiguous error
    which is probably due to some missing NeutronFixture sugar added
    in Rocky.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)
    (cherry picked from commit 47caaccd4a03660d7df144f2eadd821d36baeaa8)
    (cherry picked from commit 08337cccb060d0b3cad388004c1f802d5d053813)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/queens)

Reviewed: https://review.opendev.org/690725
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=8c18293f93a0ef957010198ae3ef2f336364b97f
Submitter: Zuul
Branch: stable/queens

commit 8c18293f93a0ef957010198ae3ef2f336364b97f
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)
    (cherry picked from commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45)
    (cherry picked from commit c84846dbf93588e5420b3565d163c0393443e6d6)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/pike)

Reviewed: https://review.opendev.org/690734
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e2ba87cb7b1cfd4226cfd8028436ecf87ad8a2be
Submitter: Zuul
Branch: stable/pike

commit e2ba87cb7b1cfd4226cfd8028436ecf87ad8a2be
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)
    (cherry picked from commit 47caaccd4a03660d7df144f2eadd821d36baeaa8)
    (cherry picked from commit 08337cccb060d0b3cad388004c1f802d5d053813)
    (cherry picked from commit f03f5075e3751ccd03a60999eab1d7f4bf7c4f02)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/pike)

Reviewed: https://review.opendev.org/690735
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b277ec154f2a175c4cab609b89fd8d3c7de4d0e9
Submitter: Zuul
Branch: stable/pike

commit b277ec154f2a175c4cab609b89fd8d3c7de4d0e9
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)
    (cherry picked from commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45)
    (cherry picked from commit c84846dbf93588e5420b3565d163c0393443e6d6)
    (cherry picked from commit 8c18293f93a0ef957010198ae3ef2f336364b97f)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 19.1.0

This issue was fixed in the openstack/nova 19.1.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 18.3.0

This issue was fixed in the openstack/nova 18.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/ocata)

Reviewed: https://review.opendev.org/690746
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b6c7456face630086a0eb7fd1c8335d42ab0456a
Submitter: Zuul
Branch: stable/ocata

commit b6c7456face630086a0eb7fd1c8335d42ab0456a
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:53:20 2019 -0400

    Add functional regression test for bug 1849409

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This adds a functional recreate test for the regression which will
    be updated when the bug is fixed.

    NOTE(mriedem): In this backport the test is modified to disable the
    DiskFilter since we're using Placement for filtering on DISK_GB.
    Also, _wait_until_deleted is moved to InstanceHelperMixin since
    If7b02bcd8d77e94c7fb42b721792c1391bc0e3b7 is not in Ocata.

    Change-Id: I324193129acb6ac739133c7e76920762a8987a84
    Related-Bug: #1849409
    (cherry picked from commit 45c2752f2ce08b314012eff044b01aa7d626b43d)
    (cherry picked from commit 727d942b2830fb6970d99507f2b5eb1a28df01b2)
    (cherry picked from commit 47caaccd4a03660d7df144f2eadd821d36baeaa8)
    (cherry picked from commit 08337cccb060d0b3cad388004c1f802d5d053813)
    (cherry picked from commit f03f5075e3751ccd03a60999eab1d7f4bf7c4f02)
    (cherry picked from commit e2ba87cb7b1cfd4226cfd8028436ecf87ad8a2be)

tags: added: in-stable-ocata
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/ocata)

Reviewed: https://review.opendev.org/690747
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b91592595141d7766c00255ffff04a25ca81c834
Submitter: Zuul
Branch: stable/ocata

commit b91592595141d7766c00255ffff04a25ca81c834
Author: Matt Riedemann <email address hidden>
Date: Tue Oct 22 17:59:32 2019 -0400

    Fix listing deleted servers with a marker

    Change I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab in Queens,
    which was backported through to Newton, introduced a regression
    when listing deleted servers with a marker because it assumes
    that if BuildRequestList.get_by_filters does not raise
    MarkerNotFound that the marker was found among the build requests
    and does not account for that get_by_filters method short-circuiting
    if filtering servers with deleted/cleaned/limit=0. The API code
    then nulls out the marker which means you'll continue to get the
    marker instance back in the results even though you shouldn't,
    and that can cause an infinite loop in some client-side tooling like
    nova's CLI:

      nova list --deleted --limit -1

    This fixes the bug by raising MarkerNotFound from
    BuildRequestList.get_by_filters if we have a marker but are
    short-circuiting and returning early from the method based on
    limit or filters.

    Change-Id: Ic2b19c2aa06b3059ab0344b6ac56ffd62b3f755d
    Closes-Bug: #1849409
    (cherry picked from commit df03499843aa7fd6089bd4d07b9d0eb5a8c14b47)
    (cherry picked from commit 03a2508362ecf50463d0659142312a30d0fb91f3)
    (cherry picked from commit 6038455e1dafeaf8ccfdd4be9e993beea3c9fb45)
    (cherry picked from commit c84846dbf93588e5420b3565d163c0393443e6d6)
    (cherry picked from commit 8c18293f93a0ef957010198ae3ef2f336364b97f)
    (cherry picked from commit b277ec154f2a175c4cab609b89fd8d3c7de4d0e9)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova ocata-eol

This issue was fixed in the openstack/nova ocata-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova pike-eol

This issue was fixed in the openstack/nova pike-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova queens-eol

This issue was fixed in the openstack/nova queens-eol release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.