Key rotation performed regardless of lead unit being paused or not

Bug #1787719 reported by Frode Nordahl
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Keystone Charm
Fix Released
High
Frode Nordahl

Bug Description

As can be seen in the artifacts from a test failure here:
https://openstack-ci-reports.ubuntu.com/artifacts/test_charm_pipeline_amulet_full/openstack/charm-keystone/593266/1/1825/index.html

Traceback (most recent call last):
  File "/var/lib/jenkins/checkout/0/keystone/.tox/func/lib/python3.5/site-packages/zaza/charm_tests/keystone/tests.py", line 130, in test_key_distribution_and_rotation
    pprint.pformat(unit_repo)))
zaza.utilities.exceptions.KeystoneKeyRepositoryError: expect: "OrderedDict([('/etc/keystone/credential-keys/',
              OrderedDict([('0',
                            'uNMjfqs7nPdMHeAYCeUoNcuH-jcd5KlL9eltGk-QiUI='),
                           ('1',
                            'JaUJG9mXt__ahzNLtinhquoqygIDZwWo-BSaSJczih0=')])),
             ('/etc/keystone/fernet-keys/',
              OrderedDict([('0',
                            'HmfrbKuHm_PTrBKvlEhGHPrao9BG5giYpmAOrfQXSpY='),
                           ('10',
                            'uEqakkPg-zd-ZLmw0KotdsnJ2oqwhJlGJ5GU6Scisgw='),
                           ('11',
                            'gVUe-laRj7GP9fltRgjIDVMJeua7N_wwUFDJrguwtMI=')]))])" actual(keystone/2): "OrderedDict([('/etc/keystone/credential-keys/',
              OrderedDict([('0',
                            'uNMjfqs7nPdMHeAYCeUoNcuH-jcd5KlL9eltGk-QiUI='),
                           ('1',
                            'JaUJG9mXt__ahzNLtinhquoqygIDZwWo-BSaSJczih0=')])),
             ('/etc/keystone/fernet-keys/',
              OrderedDict([('0',
                            'HmfrbKuHm_PTrBKvlEhGHPrao9BG5giYpmAOrfQXSpY='),
                           ('10', ''),
                           ('11',
                            'gVUe-laRj7GP9fltRgjIDVMJeua7N_wwUFDJrguwtMI=')]))])"

If you examine the juju crashdump you will see that the keys are fully rotated and written to disk at a later time, so our test has caught it in the middle of a rotation.

There are two possible solutions to this intermittent test failure:
1. Pause the lead unit before checking that rotation has taken place.
2. Only list key file names and take existence of keys with higher index than what is created on initial setup as proof of successful rotation.

While investigating these options I came across the fact that the charm currently will rotate keys regardless of it being paused, and that in itself is a bug, thus the filing of this bug report.

Frode Nordahl (fnordahl)
Changed in charm-keystone:
milestone: none → 18.08
status: New → Triaged
importance: Undecided → High
Revision history for this message
Frode Nordahl (fnordahl) wrote :
Changed in charm-keystone:
assignee: nobody → Frode Nordahl (fnordahl)
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to charm-keystone (master)

Reviewed: https://review.openstack.org/593532
Committed: https://git.openstack.org/cgit/openstack/charm-keystone/commit/?id=93db01848530f8e160509ecc78ea9520626df21f
Submitter: Zuul
Branch: master

commit 93db01848530f8e160509ecc78ea9520626df21f
Author: Frode Nordahl <email address hidden>
Date: Mon Aug 20 09:37:09 2018 +0200

    Do not rotate keys when lead unit is paused

    Closes-Bug: #1787719
    Change-Id: I0557803e90d8ec52271f01e5e7276d2db8338ce2

Changed in charm-keystone:
status: In Progress → Fix Committed
David Ames (thedac)
Changed in charm-keystone:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.