Barbican

Investigate and fix sporadic JSONDecodeError in the functional tests

Bug #1407767 reported by John Vrbanac
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Barbican
Fix Released
High
Steve Heyman
Barbican 2015.1.0 "kilo"

Bug Description

We keep having a number of these errors kick up on various CR's which ends up failing our devstack gate jobs.

Traceback (most recent call last):
       File "/opt/stack/new/barbican/functionaltests/api/v1/functional/test_containers.py", line 76, in setUp
         self.secret_ref_1 = self._create_a_secret()
       File "/opt/stack/new/barbican/functionaltests/api/v1/functional/test_containers.py", line 104, in _create_a_secret
         resp, secret_ref = self.secret_behaviors.create_secret(secret_model)
       File "/opt/stack/new/barbican/functionaltests/api/v1/behaviors/secret_behaviors.py", line 35, in create_secret
         returned_data = resp.json()
       File "/usr/lib/python2.7/dist-packages/requests/models.py", line 741, in json
         return json.loads(self.text, **kwargs)
       File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 488, in loads
         return _default_decoder.decode(s)
       File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 370, in decode
         obj, end = self.raw_decode(s)
       File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 389, in raw_decode
         return self.scan_once(s, idx=_w(s, idx).end())
     JSONDecodeError: Expecting value: line 1 column 1 (char 0)

More Example errors:
http://paste.openstack.org/show/155624/

Two recent CR's that have exhibited this problem:
https://review.openstack.org/#/c/142209/3
https://review.openstack.org/#/c/141535/

John Vrbanac (john.vrbanac)
Changed in barbican:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to barbican (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/145591

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to barbican (master)

Reviewed: https://review.openstack.org/145591
Committed: https://git.openstack.org/cgit/openstack/barbican/commit/?id=2e2e9077ba4ea8a9debabd3914194255d168d7b4
Submitter: Jenkins
Branch: master

commit 2e2e9077ba4ea8a9debabd3914194255d168d7b4
Author: John Vrbanac <email address hidden>
Date: Wed Jan 7 15:10:15 2015 -0600

    Adding error handling to help debug devstack issue

    We keep running into sporadic JSONDecodeError exceptions being
    raised on our functional tests. Adding a little error handling
    to give us more information and gracefully handle the exception.

    Change-Id: Ib6bf8fdcceedd6f67f9d7b047f330538b5002289
    Related-Bug: #1407767

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to barbican (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/146088

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on barbican (master)

Change abandoned by Juan Antonio Osorio Robles (<email address hidden>) on branch: master
Review: https://review.openstack.org/146088

Steve Heyman (sheyman)
Changed in barbican:
assignee: nobody → Steve Heyman (sheyman)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to barbican (master)

Reviewed: https://review.openstack.org/146608
Committed: https://git.openstack.org/cgit/openstack/barbican/commit/?id=08726233da35481c540a4a79ef1b988ab989e0dd
Submitter: Jenkins
Branch: master

commit 08726233da35481c540a4a79ef1b988ab989e0dd
Author: Steve Heyman <email address hidden>
Date: Mon Jan 12 12:50:38 2015 -0600

    Resolve intermittent HTTP 404 in devstack gate

    An intermittent 404 would occur because of the way that
    Tempest keystone v3 authprovider was resolving base URLs.
    Barbican wasn't passing in a region code, and the tempest
    v3 auth provider was picking the "first" entry in the
    entrypoint list, which was often the Barbican admin URL
    (port 9312). Trying to use that port for normal Barbican
    ReST calls resulted in the http 404. This fix adds the
    region to the devstack config and uses that on the filter
    that Barbican gives to Tempest to find the correct
    Barbican endpoint.

    Change-Id: Ib6dd5aa79198463b5db2541d85de8e67b400212b
    Closes-Bug: #1407767

Changed in barbican:
status: In Progress → Fix Committed
Douglas Mendizábal (dougmendizabal)
Changed in barbican:
milestone: none → kilo-2
Thierry Carrez (ttx)
Changed in barbican:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in barbican:
milestone: kilo-2 → 2015.1.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.