OpenStack Compute (nova)

Create servergroup failed with unexpected error

  1. Series rocky
  2. Bug #1894966
Bug #1894966 reported by Ethan Lynn
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Low
Stephen Finucane
Ocata
Confirmed
Undecided
Unassigned
Pike
Confirmed
Undecided
Unassigned
Queens
Confirmed
Undecided
Unassigned
Rocky
Confirmed
Undecided
Unassigned
Stein
Confirmed
Undecided
Unassigned
Train
Fix Released
Undecided
Stephen Finucane
Ussuri
Fix Released
Undecided
Stephen Finucane

Bug Description

Create a serverGroup with policies set to [] will cause nova api return unexpected error.

Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.307 9 DEBUG nova.api.openstack.wsgi [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Action: 'create', calling method: <function Controller.__getattribute__.<locals>.version_select at 0x7f0600e5d680>, body: {"server_group": {"name": "tpp-test-archetype-app-sandbox-vlyo", "policies": []}} _process_stack /usr/lib/python3.7/site-packages/nova/api/openstack/wsgi.py:520
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.336 9 DEBUG nova.quota [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Getting quotas for project 9425beb60c8746518fd77513191623ea. Resources: {'server_groups'} _get_quotas /usr/lib/python3.7/site-packages/nova/quota.py:390
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.340 9 DEBUG nova.quota [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Getting quotas for user 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 and project 9425beb60c8746518fd77513191623ea. Resources: {'server_groups'} _get_quotas /usr/lib/python3.7/site-packages/nova/quota.py:382
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Unexpected exception in API method: IndexError: list index out of range
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/openstack/wsgi.py", line 671, in wrapped
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/validation/__init__.py", line 110, in wrapper
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi return func(*args, **kwargs)
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.7/site-packages/nova/api/openstack/compute/server_groups.py", line 201, in create
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup(context, policy=policies[0])
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi IndexError: list index out of range
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.351 9 ERROR nova.api.openstack.wsgi
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.354 9 INFO nova.api.openstack.wsgi [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: <class 'IndexError'>
Aug 25 15:16:16 controller-qf47h5jn62 nova-osapi[752]: 2020-08-25 15:16:16.355 9 DEBUG nova.api.openstack.wsgi [req-d2edaa0f-bc9f-42ac-ba85-1273d25f3799 451d8d24f6428a3cef7409695abb6cbd3a1c4d2ef2b459e1fcdf75a848be8294 9425beb60c8746518fd77513191623ea - default default] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.

Sylvain Bauza (sylvain-bauza)
tags: added: api
Revision history for this message
Sylvain Bauza (sylvain-bauza) wrote :

Which release are you running with and do you specify a specific API microversion ?

Our latest release (Ussuri) includes API validation on the policies field to be a list of either 'affinity' or 'anti-affinity' enums on v2.1 and more on later microversions.

https://github.com/openstack/nova/blob/stable/ussuri/nova/api/openstack/compute/schemas/server_groups.py#L28-L39

Changed in nova:
status: New → Incomplete
Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

I can reproduce the problem on the latest master

stack@aio:/opt/stack/nova$ curl -g -i -X POST http://192.168.121.129/compute/v2.1/os-server-groups -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: $TOKEN" -H "X-OpenStack-Nova-API-Version: 2.1" -d '{"server_group": {"name": "my-grp", "policies": []}}'
HTTP/1.1 500 Internal Server Error
Date: Tue, 15 Sep 2020 10:34:08 GMT
Server: Apache/2.4.29 (Ubuntu)
OpenStack-API-Version: compute 2.1
X-OpenStack-Nova-API-Version: 2.1
Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version
Content-Type: application/json; charset=UTF-8
Content-Length: 184
x-openstack-request-id: req-6651c4cd-bae4-4011-8975-cb00460d36d3
x-compute-request-id: req-6651c4cd-bae4-4011-8975-cb00460d36d3
Connection: close

{"computeFault": {"code": 500, "message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'IndexError'>"}}stack@aio:/opt/stack/nova$

Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

The above reproduction holds for microversion 2.1 - 2.63. In microversion 2.64 the API has changed, the policies field was renamed to policy and the type changed from a list to a single string. So the problem does not appear with microversion 2.64 or above

Balazs Gibizer (balazs-gibizer)
Changed in nova:
status: Incomplete → Confirmed
importance: Undecided → Low
Revision history for this message
Ghanshyam Mann (ghanshyammann) wrote :

yeah in 2.64 microversion request has been changed from 'policies' as array to single items as string
- https://review.opendev.org/#/c/567534/36/nova/api/openstack/compute/rest_api_version_history.rst

empty 'policies' was anyways wrong request which has been fixed in 2.64 by converting the array type to string field. What we can do is to improve the error code from 500 to 400 but do we need to improve the error code for older microversion? or should ask to use new microversion which already fixed the issue.

Revision history for this message
Brin Zhang (zhangbailin) wrote :

Agree improve the error code from 500 to 400, and IMO we also need to improve the error code for older microversion, it's friendly for users.

Revision history for this message
Brin Zhang (zhangbailin) wrote :

This is happened in microversion 2.1~2.63, there is no much benefits to improve error code for older microversion.

A good way is to change the docs.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.opendev.org/752133

Changed in nova:
assignee: nobody → Brin Zhang (zhangbailin)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/752210

Changed in nova:
assignee: Brin Zhang (zhangbailin) → Stephen Finucane (stephenfinucane)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.opendev.org/752211

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (master)

Reviewed: https://review.opendev.org/752210
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=2c66962c7a40d8ef4fab54324e06edcdec1bd716
Submitter: Zuul
Branch: master

commit 2c66962c7a40d8ef4fab54324e06edcdec1bd716
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:52:17 2020 +0100

    tests: Add regression test for bug 1894966

    You must specify the 'policies' field. Currently, not doing so will
    result in a HTTP 500 error code. This should be a 4xx error. Add a test
    to demonstrate the bug before we provide a fix.

    Change-Id: I72e85855f621d3a51cd58d14247abd302dcd958b
    Signed-off-by: Stephen Finucane <email address hidden>
    Related-Bug: #1894966

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Brin Zhang (<email address hidden>) on branch: master
Review: https://review.opendev.org/752133

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/ussuri)

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/752371

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.opendev.org/752211
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=32c43fc8017ee89d4e6cdf79086d87735a00f0c0
Submitter: Zuul
Branch: master

commit 32c43fc8017ee89d4e6cdf79086d87735a00f0c0
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:59:49 2020 +0100

    api: Set min, maxItems for server_group.policies field

    As noted inline, the 'policies' field may be a list but it expects one
    of two items.

    Change-Id: I34c68df1e6330dab1524aa0abec733610211a407
    Signed-off-by: Stephen Finucane <email address hidden>
    Closes-Bug: #1894966

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/752702

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (stable/train)

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/752706

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/train)

Fix proposed to branch: stable/train
Review: https://review.opendev.org/752707

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/ussuri)

Reviewed: https://review.opendev.org/752371
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=94d24e3e8d04488abdebd4969daf98b780125297
Submitter: Zuul
Branch: stable/ussuri

commit 94d24e3e8d04488abdebd4969daf98b780125297
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:52:17 2020 +0100

    tests: Add regression test for bug 1894966

    You must specify the 'policies' field. Currently, not doing so will
    result in a HTTP 500 error code. This should be a 4xx error. Add a test
    to demonstrate the bug before we provide a fix.

    Change-Id: I72e85855f621d3a51cd58d14247abd302dcd958b
    Signed-off-by: Stephen Finucane <email address hidden>
    Related-Bug: #1894966
    (cherry picked from commit 2c66962c7a40d8ef4fab54324e06edcdec1bd716)

tags: added: in-stable-ussuri
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/ussuri)

Reviewed: https://review.opendev.org/752702
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=781210bd598c3e0ee9bd6a7db5d25688b5fc0131
Submitter: Zuul
Branch: stable/ussuri

commit 781210bd598c3e0ee9bd6a7db5d25688b5fc0131
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:59:49 2020 +0100

    api: Set min, maxItems for server_group.policies field

    As noted inline, the 'policies' field may be a list but it expects one
    of two items.

    Change-Id: I34c68df1e6330dab1524aa0abec733610211a407
    Signed-off-by: Stephen Finucane <email address hidden>
    Closes-Bug: #1894966
    (cherry picked from commit 32c43fc8017ee89d4e6cdf79086d87735a00f0c0)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (stable/train)

Reviewed: https://review.opendev.org/752706
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=cf6db29168c4ee9a34f3b3ebdd1deae31f95f203
Submitter: Zuul
Branch: stable/train

commit cf6db29168c4ee9a34f3b3ebdd1deae31f95f203
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:52:17 2020 +0100

    tests: Add regression test for bug 1894966

    You must specify the 'policies' field. Currently, not doing so will
    result in a HTTP 500 error code. This should be a 4xx error. Add a test
    to demonstrate the bug before we provide a fix.

    Changes:
      nova/tests/functional/regressions/test_bug_1894966.py

    NOTE(stephenfin): Need to update 'super' call to Python 2-compatible
    variant.

    Change-Id: I72e85855f621d3a51cd58d14247abd302dcd958b
    Signed-off-by: Stephen Finucane <email address hidden>
    Related-Bug: #1894966
    (cherry picked from commit 2c66962c7a40d8ef4fab54324e06edcdec1bd716)
    (cherry picked from commit 94d24e3e8d04488abdebd4969daf98b780125297)

tags: added: in-stable-train
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/train)

Reviewed: https://review.opendev.org/752707
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1634d3f59ace0206131992e31ee2d4b64123d7e8
Submitter: Zuul
Branch: stable/train

commit 1634d3f59ace0206131992e31ee2d4b64123d7e8
Author: Stephen Finucane <email address hidden>
Date: Wed Sep 16 10:59:49 2020 +0100

    api: Set min, maxItems for server_group.policies field

    As noted inline, the 'policies' field may be a list but it expects one
    of two items.

    Change-Id: I34c68df1e6330dab1524aa0abec733610211a407
    Signed-off-by: Stephen Finucane <email address hidden>
    Closes-Bug: #1894966
    (cherry picked from commit 32c43fc8017ee89d4e6cdf79086d87735a00f0c0)
    (cherry picked from commit 781210bd598c3e0ee9bd6a7db5d25688b5fc0131)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (stable/stein)

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/stein
Review: https://review.opendev.org/c/openstack/nova/+/772720
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by "Elod Illes <email address hidden>" on branch: stable/stein
Review: https://review.opendev.org/c/openstack/nova/+/772721
Reason: This branch transitioned to End of Life for this project, open patches needs to be closed to be able to delete the branch.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.