traceback due to malformed ceph request when upgrading from ~15.04 to 17.11

Bug #1753613 reported by Paul Collins
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Charm Helpers
Triaged
Medium
Unassigned
OpenStack Glance Charm
Triaged
Medium
Unassigned

Bug Description

Upgrading glance from ~15.04 (r113, revision-id: <email address hidden>) to 17.11 yielded the following traceback:

2018-02-07 22:54:37 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/config-changed", line 328, in ceph_changed
2018-02-07 22:54:37 INFO config-changed if is_request_complete(get_ceph_request()):
2018-02-07 22:54:37 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/charmhelpers/contrib/storage/linux/ceph.py", line 1270, in is_request_complete
2018-02-07 22:54:37 INFO config-changed states = get_request_states(request, relation=relation)
2018-02-07 22:54:37 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/charmhelpers/contrib/storage/linux/ceph.py", line 1231, in get_request_states
2018-02-07 22:54:37 INFO config-changed previous_request = get_previous_request(rid)
2018-02-07 22:54:37 INFO config-changed File "/var/lib/juju/agents/unit-glance-0/charm/hooks/charmhelpers/contrib/storage/linux/ceph.py", line 1211, in get_previous_request
2018-02-07 22:54:37 INFO config-changed request_id=request_data['request-id'])
2018-02-07 22:54:37 INFO config-changed KeyError: 'request-id'

From examining the code it seems that the ~15.04 version of glance has a very early version of the ceph code. Later versions added the request-id field but did not bump api-version, and also apparently cannot cope when it's missing.

The following workaround (adjusted to match the actual request as needed) and juju resolved --retry completed the upgrade.

juju run --unit glance/0 'relation-set -r $(relation-ids ceph) broker_req="{\"api-version\": 1, \"request-id\": \"'$(uuid -v1)'\", \"ops\": [{\"replicas\": 3, \"name\": \"glance\", \"op\": \"create-pool\"}]}"'

Paul Collins (pjdc)
tags: added: canonical-is-ps45-1711-upgrade
Revision history for this message
James Page (james-page) wrote :

This should be fixable with some opportunistic reading of request-id.

no longer affects: charms.ceph
tags: added: upgrade
Changed in charm-helpers:
status: New → Triaged
Changed in charm-glance:
status: New → Triaged
Changed in charm-helpers:
importance: Undecided → Medium
Changed in charm-glance:
importance: Undecided → Medium
tags: added: charm-upgrade
removed: upgrade
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.