[SRU] exceptions.from_response with webob 1.6.0 results in "AttributeError: 'unicode' object has no attribute 'get'"

I think we can confirm this, we're currently testing webob 1.5.1 in the gate but the upper-constraints proposal to include webob 1.6.0 is failing on the same thing:

http://logs.openstack.org/49/288849/15/check/gate-tempest-dsvm-full/4722c5a/logs/devstacklog.txt.gz#_2016-03-18_07_06_03_369

The problem is in python-novaclient.

Looks like this is the breaking change in webob 1.6.0:

https://github.com/Pylons/webob/pull/139

As you can see in the response body:

RESP BODY: {"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

The error message has html in it.

Also wondering about https://github.com/Pylons/webob/pull/230

The failure is here:

https://github.com/openstack/python-novaclient/blob/3.3.0/novaclient/exceptions.py#L298

Apparently our assumptions about the body there are wrong now.

So given the error debug above, the body is this:

{"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

If you take that python-novaclient code, you get:

error = body[list(body)[0]]

>>> body[list(body)[0]]
'The resource could not be found.<br /><br />\n\n\n'

So error is going to be a str rather than a dict. I'm not sure what this was before.

From what I can tell, this should also be a problem in openstackclient:

https://github.com/openstack/python-openstackclient/blob/2.2.0/openstackclient/common/exceptions.py#L125

And cinderclient:

https://github.com/openstack/python-cinderclient/blob/master/cinderclient/exceptions.py#L183

Unless it has something to do with the content-type on the request/response?

Yeah, this is the issue:

https://github.com/Pylons/webob/commit/87c8749a57c1ff2442db2d74d9fb86935b7b201e

Which was written for glance, and of course glanceclient handles this properly:

https://github.com/openstack/python-glanceclient/blob/2.0.0/glanceclient/exc.py#L157

If you downgrade to WebOb 1.5.1 the problem will not go away until nova-api is restarted, so there is some involvement from the nova api server also.

I've opened an issue against webob:

https://github.com/Pylons/webob/issues/235

I doubt they are going to change it since it's fixing a long standing behavior issue, but it was a backward incompatible change and probably should have gone out in a major version 2.0 release.

But we'll probably need to cap webob<1.6.0 until the various clients are working with this.

Here is what a 404 response looks like with novaclient and webob 1.5.1:

DEBUG (session:248) REQ: curl -g -i -X GET http://9.5.126.108:8774/v2.1/e4f68bae118f46fa986e3fd54cbacb1e/flavors/test -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.25" -H "X-Auth-Token: {SHA1}8b87e089d79ad483b178bdf673e9639ffd86d529"
DEBUG (connectionpool:387) "GET /v2.1/e4f68bae118f46fa986e3fd54cbacb1e/flavors/test HTTP/1.1" 404 77
DEBUG (session:277) RESP: [404] Content-Length: 77 X-Compute-Request-Id: req-d9df03b0-4150-4b53-8157-7560ccf39f75 Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.25 Date: Fri, 18 Mar 2016 22:30:46 GMT Content-Type: application/json; charset=UTF-8
RESP BODY: {"itemNotFound": {"message": "Flavor test could not be found.", "code": 404}}

So the body is a dict of dicts, where I guess the error message keys to the body, which is how the code expected it to work in novaclient.

>>> body = {"itemNotFound": {"message": "Flavor test could not be found.", "code
": 404}}
>>> body[list(body)[0]]
{'message': 'Flavor test could not be found.', 'code': 404}
>>>

But now we get something like:

{"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"}

Note that even with webob 1.5.1 the content-type in the response is application/json, so it doesn't really help us to key off that to handle this. We basically can't trust the format until we require at least webob>=1.6.0.

Fix proposed to branch: master
Review: https://review.openstack.org/294850

python-novaclient patch is here: https://review.openstack.org/#/c/294850/

Looks like the apiclient in oslo-incubator handled this awhile ago:

https://github.com/openstack/oslo-incubator/commit/fd8dc0c3ffe38fe64a3c841d87ea820045221770

Which is now used in keystoneauth. And python-keystoneclient is using that code from keystoneauth.

https://bugs.launchpad.net/networking-midonet/+bug/1558923 seems like the same issue

Hey all,

Sorry about the breakage, but I have a couple of questions, especially since Nova's Fault() should be doing the right thing (but clearly it is not):

See:

https://github.com/Pylons/webob/issues/235#issuecomment-198643200

If someone can help me understand the flow on the nova API side then I can make sure that it is a bug in WebOb, because according to my reading WebOb shouldn't even be hitting the code path that generates it's internal JSON response.

I have run all tests on the nova codebase using tox -e py27,functional with WebOb 1.6.0 and I can not replicate any failure's. Please provide a stacktrace that shows WebOb's exception's generate_response function being called (which is what is causing the wrong JSON to be returned).

Referring to #17:

https://github.com/Pylons/webob/blob/1.6-branch/webob/exc.py#L325

This is the function that has to be called to generate the wrong JSON response. With the Fault wrapping that nova does, I can't figure out how it would ever be called. See the Github issue link: https://github.com/Pylons/webob/issues/235 for more information on what I have tried and the analysis.

Bert, thanks for the offer to help.

So Nova uses this FaultWrapper middleware (https://github.com/openstack/nova/blob/3b2895b960630934f827918d50577f9167110cab/nova/api/openstack/__init__.py#L106) to intercept all exceptions and problems so it can re-wrap them in its Fault class. I wonder if something changed that stopped calling the Fault instance. It seems like we're expecting all of the work that is causing this to happen in the Fault's __call__ (https://github.com/openstack/nova/blob/2bc8128d7793cc72ca2e146de3a092e1fef5033b/nova/api/openstack/wsgi.py#L1061) method and I wonder if something stopped calling it.

Note also that the FaultWrapper middleware *returns* the exception instead of re-raising the exception wrapped in the Fault. Could either of those (or both) be related to this problem?

For what it's wroth, the paste file looks like this:

https://github.com/openstack/nova/blob/2bc8128d7793cc72ca2e146de3a092e1fef5033b/etc/nova/api-paste.ini#L45 which uses the faultwrap filter defined here: https://github.com/openstack/nova/blob/2bc8128d7793cc72ca2e146de3a092e1fef5033b/etc/nova/api-paste.ini#L62

Nova return fault at here https://github.com/openstack/nova/blob/stable/mitaka/nova/wsgi.py#L460 for any resource which didn't register to the route mapper

If we try to request with '-H "Accept: text/plain"' explicitly and with webob 1.6.0, we will get same response as before"

curl -g -i -X GET http://192.168.2.176:8774/v2.1/b90b53ed87d74e19806da34dbaa056c9/ser -H "User-Agent: python-novaclient" -H "Accept: text/plain" -H "X-Auth-Token: 3d30d32d4c3f4c36b268656f070a7e4a"HTTP/1.1 404 Not Found
Content-Length: 52
Content-Type: text/plain; charset=UTF-8
X-Compute-Request-Id: req-12de9dc1-ff6a-4f77-81a3-079a1d4c0221
Date: Tue, 22 Mar 2016 09:58:47 GMT

404 Not Found

The resource could not be found.

@Matt, I think we will return the same response even we upgrade webob to 1.6 in the case of "GET /v2.1/e4f68bae118f46fa986e3fd54cbacb1e/flavors/test", due to this is registered resource. The response changed only for the case like "GET /v2.1/b90b53ed87d74e19806da34dbaa056c9/not_registered_resource"

Change abandoned by YAMAMOTO Takashi (<email address hidden>) on branch: master
Review: https://review.openstack.org/294527
Reason: turned out to be unnecessary

midonet v2015.06 fix: https://review.gerrithub.io/#/c/269940/

Related fix proposed to branch: master
Review: https://review.openstack.org/298086

Change abandoned by Wenzhi Yu (wenzhi_yu@163.com) on branch: master
Review: https://review.openstack.org/298086
Reason: 'from_response' is never called in code base, should remove it.

Related fix proposed to branch: master
Review: https://review.openstack.org/298617

Reviewed: https://review.openstack.org/298617
Committed: https://git.openstack.org/cgit/openstack/python-openstackclient/commit/?id=848d5312fffa358a789a46a8657af3a92cb3ef6f
Submitter: Jenkins
Branch: master

commit 848d5312fffa358a789a46a8657af3a92cb3ef6f
Author: Wenzhi Yu <wenzhi_yu@163.com>
Date: Tue Mar 29 17:20:05 2016 +0800

    Remove unused method 'from_response'

    'openstackclient.common.exceptions.from_response' method is never called
    in openstackclient code base, so we should remove it.

    Change-Id: I04254a4e66863942e6c273d77bbd66ce2ce7804c
    Related-Bug: #1559072

Reviewed: https://review.openstack.org/294850
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=fa377e7fca44e81247639efebd7cae5cb377e57e
Submitter: Jenkins
Branch: master

commit fa377e7fca44e81247639efebd7cae5cb377e57e
Author: Matt Riedemann <email address hidden>
Date: Fri Mar 18 19:17:47 2016 -0400

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.

    Change-Id: If653a247d842786d2824b4b3a5c0cde1383ed7ab
    Closes-Bug: #1559072

all relevant patches are merged

This issue was fixed in the openstack/python-novaclient 3.4.0 release.

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/307021

Change abandoned by Alexis Lee (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/307021
Reason: cool, thank you

Status changed to 'Confirmed' because the bug affects multiple users.

Fixes in 3.4.0; Marking yakkety task as fix released.

Unable to reproduce on Xenial; Xenial has webob 1.5.1 so I don't think its likely to be impacted.

Re-opening Xenial task - accessing a cloud using webob 1.6.0 with older novaclient from 16.04 will fail with this problem; bearing this in mind we might want to consider resurrecting the stable/mitaka novaclient fix as well.

In trying to create a nova flavor from a Xenial machine against a Xenial-Newton deployment, I see:

00:25:25.978 + nova flavor-create m1.cirros 6 64 1 1
00:25:27.740 ERROR (AttributeError): 'unicode' object has no attribute 'get'

jenkins@juju-osci1-machine-8:~/temp$ apt-cache policy python-novaclient
python-novaclient:
  Installed: 2:3.3.1-2
  Candidate: 2:3.3.1-2
  Version table:
 *** 2:3.3.1-2 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status

I just tested python-novaclient 2:3.3.1-2 on xenial with the abandoned upstream patch successfully.

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/378671

I've re-proposed the stable/mitaka patch for this since this is an issue when the stable/mitaka python-novaclient is used with newton+ nova deployments.

Hello Phil, or anyone else affected,

Accepted python-novaclient into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-novaclient/2:3.3.1-2ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Change abandoned by Corey Bryant (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/378671
Reason: Abandoned in favor of: https://review.openstack.org/307021

Reviewed: https://review.openstack.org/307021
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=70c35e6e3d2aed9f37d5101dafd22c5905ec8583
Submitter: Jenkins
Branch: stable/mitaka

commit 70c35e6e3d2aed9f37d5101dafd22c5905ec8583
Author: Matt Riedemann <email address hidden>
Date: Fri Mar 18 19:17:47 2016 -0400

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.

    Change-Id: If653a247d842786d2824b4b3a5c0cde1383ed7ab
    Closes-Bug: #1559072

This has been fixed in webob 1.6.1:

http://docs.webob.org/en/stable/changes.html

Fix tested successfully on xenial-proposed and mitaka-proposed.

This bug was fixed in the package python-novaclient - 2:3.3.1-2ubuntu1

---------------
python-novaclient (2:3.3.1-2ubuntu1) xenial; urgency=medium

  * d/p/enable-1.6.0-webob.patch: Cherry pick from stable/mitaka gerrit review
    to enable use of python-novaclient with newton+ nova deployments that have
    python-webob >= 1.6.0 (LP: #1559072).

 -- Corey Bryant <email address hidden> Wed, 28 Sep 2016 10:44:36 -0400

The verification of the Stable Release Update for python-novaclient has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

cinderclient patch: https://review.openstack.org/#/c/407422/

Reviewed: https://review.openstack.org/407422
Committed: https://git.openstack.org/cgit/openstack/python-cinderclient/commit/?id=19befa69651a41534c7d25b1a0d3a0cd407afb44
Submitter: Jenkins
Branch: master

commit 19befa69651a41534c7d25b1a0d3a0cd407afb44
Author: Akira KAMIO <email address hidden>
Date: Tue Dec 6 14:10:38 2016 +0900

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.Handle error response for
    webob>=1.6.0

    Change-Id: I7d589415aa024588faf77c8234ac026110f6c3cd
    Closes-Bug: #1559072

This issue was fixed in the openstack/python-cinderclient 1.10.0 release.

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/462204

A comment about the new review 462204: I requested a backport of the cinderclient patch (407422) to Newton. We recently upgraded to Newton, and a user is now unable to delete Heat stacks that contain Cinder volumes that have already been removed. Applying the cinderclient patch fixes this problem.

Reviewed: https://review.openstack.org/462204
Committed: https://git.openstack.org/cgit/openstack/python-cinderclient/commit/?id=b20d9fd6563c1ef295e8a260742f2b3e0b8ee327
Submitter: Jenkins
Branch: stable/newton

commit b20d9fd6563c1ef295e8a260742f2b3e0b8ee327
Author: Akira KAMIO <email address hidden>
Date: Tue Dec 6 14:10:38 2016 +0900

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:

    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.Handle error response for
    webob>=1.6.0

    Change-Id: I7d589415aa024588faf77c8234ac026110f6c3cd
    Closes-Bug: #1559072
    (cherry picked from commit 19befa69651a41534c7d25b1a0d3a0cd407afb44)

Openstack: stable/ocata
Ubuntu 16.04.02 Xenial Xerus

$ openstack server list
+--------------------------------------+--------+-----------+
| ID | Name | Status |
+--------------------------------------+--------+-----------+
| 54b2465f-6a5d-467e-8815-42c436852ba6 | jex-n1 | ACTIVE |
| a0442c11-dc5b-4d8f-9e77-a83b59782b7c | jex-n2 | SUSPENDED |
+--------------------------------------+--------+-----------+

$ openstack server rebuild jex-n1
'unicode' object has no attribute 'get'

====== --debug -vvvv =======
GET call to compute for http://controller:8774/v2.1/servers?name=jex-n1 used request id req-def51157-4adb-4c37-9ccf-77a8124fa939
REQ: curl -g -i -X GET http://controller:8774/v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}7970e0bb2968bd69cc6f49026ba350e55da7b105"
http://controller:8774 "GET /v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6 HTTP/1.1" 200 1782
RESP: [200] Content-Length: 1782 Content-Type: application/json Openstack-Api-Version: compute 2.1 X-Openstack-Nova-Api-Version: 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-4e02c928-6dbc-4f79-9cc9-1ca0ad8e69e3 Date: Mon, 05 Jun 2017 10:44:16 GMT
RESP BODY: {"server": {"OS-EXT-STS:task_state": null, "addresses": {"e-net": [{"OS-EXT-IPS-MAC:mac_addr": "d0:1c:a0:04:a4:5d", "version": 4, "addr": "51.255.0.248", "OS-EXT-IPS:type": "fixed"}], "jex-net": [{"OS-EXT-IPS-MAC:mac_addr": "d0:1c:a0:74:f8:57", "version": 4, "addr": "1.1.1.10", "OS-EXT-IPS:type": "fixed"}], "dummy-net": [{"OS-EXT-IPS-MAC:mac_addr": "d0:1c:a0:8c:ae:f2", "version": 4, "addr": "25.0.0.7", "OS-EXT-IPS:type": "fixed"}]}, "links": [{"href": "http://controller:8774/v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6", "rel": "self"}, {"href": "http://controller:8774/servers/54b2465f-6a5d-467e-8815-42c436852ba6", "rel": "bookmark"}], "image": "", "OS-EXT-STS:vm_state": "active", "OS-EXT-SRV-ATTR:instance_name": "instance-0000016f", "OS-SRV-USG:launched_at": "2017-06-05T08:53:58.000000", "flavor": {"id": "d0ff4bc5-df38-4f20-8908-afc516d594e6", "links": [{"href": "http://controller:8774/flavors/d0ff4bc5-df38-4f20-8908-afc516d594e6", "rel": "bookmark"}]}, "id": "54b2465f-6a5d-467e-8815-42c436852ba6", "security_groups": [{"name": "default"}, {"name": "jex-esg"}], "user_id": "0d01892c43b0498198c4716d510c6667", "OS-DCF:diskConfig": "MANUAL", "accessIPv4": "", "accessIPv6": "", "progress": 0, "OS-EXT-STS:power_state": 1, "OS-EXT-AZ:availability_zone": "nova", "metadata": {}, "status": "ACTIVE", "updated": "2017-06-05T08:53:59Z", "hostId": "4c5f24a68ef88f97beb99c3e34d5ea6cba6d3fabe3dc5a6487ed8415", "OS-EXT-SRV-ATTR:host": "ardbeg", "OS-SRV-USG:terminated_at": null, "key_name": null, "OS-EXT-SRV-ATTR:hypervisor_hostname": "compute0.cloud.local", "name": "jex-n1", "created": "2017-06-05T08:53:42Z", "tenant_id": "d8051a3ff3ad4c4bb380f828992b8178", "os-extended-volumes:volumes_attached": [{"id": "d76e23e8-c6cb-462e-9ac0-81b2d733fc77"}], "config_drive": ""}}
GET call to compute for http://controller:8774/v2.1/servers/54b2465f-6a5d-467e-8815-42c436852ba6 used re...

This issue was fixed in the openstack/python-novaclient 3.3.3 release.

yakkety is EOL

Status changed to 'Confirmed' because the bug affects multiple users.

I've uploaded python-cinderclient to the xenial review queue and directly to the newton cloud archive staging PPA.

Hello Phil, or anyone else affected,

Accepted python-cinderclient into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-cinderclient/1:1.6.0-2ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Phil, or anyone else affected,

Accepted python-cinderclient into newton-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:newton-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-newton-needed to verification-newton-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-newton-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I tested the package on Newton. It LGTM. Thank you

Hello Phil, or anyone else affected,

Accepted python-cinderclient into mitaka-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:mitaka-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-mitaka-needed to verification-mitaka-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-mitaka-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

@zioproto

Please could you update the bug tags inline with the acceptance comments when you've completed testing.

The verification of the Stable Release Update for python-cinderclient has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package python-cinderclient - 1:1.9.0-0ubuntu1~cloud2
---------------

 python-cinderclient (1:1.9.0-0ubuntu1~cloud2) xenial-newton; urgency=medium
 .
   * No change. Upload new version as last version was rejected (LP: #1559072).

I've verified that python-cinderclient is fixed on xenial-proposed: http://paste.ubuntu.com/25283545/

Regression tests have also passed successfully.

I've also verified against the same cloud (with web>=1.6.0) via a trusty machine with the mitaka-proposed cloud archive: http://paste.ubuntu.com/25283602/

This bug was fixed in the package python-cinderclient - 1:1.6.0-2ubuntu1

---------------
python-cinderclient (1:1.6.0-2ubuntu1) xenial; urgency=medium

  * d/p/review-462204.patch: Handle error response for webob>=1.6.0
    (LP: #1559072).

 -- Saverio Proto <email address hidden> Mon, 26 Jun 2017 11:55:31 +0000

The verification of the Stable Release Update for python-cinderclient has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package python-cinderclient - 1:1.6.0-2ubuntu1~cloud0
---------------

 python-cinderclient (1:1.6.0-2ubuntu1~cloud0) trusty-mitaka; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 python-cinderclient (1:1.6.0-2ubuntu1) xenial; urgency=medium
 .
   * d/p/review-462204.patch: Handle error response for webob>=1.6.0
     (LP: #1559072).

Fix proposed to branch: master
Review: https://review.openstack.org/541170

Reviewed: https://review.openstack.org/541170
Committed: https://git.openstack.org/cgit/openstack/python-troveclient/commit/?id=0b28b04ebbfaed981f4e00e69e4c7a8e0ed75395
Submitter: Zuul
Branch: master

commit 0b28b04ebbfaed981f4e00e69e4c7a8e0ed75395
Author: Zhao Chao <email address hidden>
Date: Tue Feb 6 15:31:42 2018 +0800

    Handle error response for webob>=1.6.0

    WebOb change https://github.com/Pylons/webob/pull/230 changed
    the way in which the error response body is formatted such that
    it's no longer a nested dict. So we have to handle both the
    old convention of an error message key to the response body error
    dict and the new way with just the error body dict.

    This was reported upstream:
    https://github.com/Pylons/webob/issues/235

    But given this was apparently implemented as a long-overdue change
    in WebOb the behavior is not likely to change.

    Change-Id: Id8bfde6277640de48768bbe5a5e4252b97253ed2
    Closes-Bug: #1559072
    Co-Authored-By: Matt Riedemann <email address hidden>
    Signed-off-by: Zhao Chao <email address hidden>

This issue was fixed in the openstack/python-troveclient 2.15.0 release.