When removing a cinder-ceph storage backend relation we ended up with a env var override for the cinder service that referenced the non-existing ceph authx key from cinder-ceph
Detail:
1. Pre-existing cinder service configured for rbd backend
2. Added a cinder-ceph service foo-storage and relate to cinder
3. Removed relation cinder - cinder-ceph
This reconfigured the original rbd backend in cinder.conf, however it left an override for ceph authx behind:
When removing a cinder-ceph storage backend relation we ended up with a env var override for the cinder service that referenced the non-existing ceph authx key from cinder-ceph
Detail:
1. Pre-existing cinder service configured for rbd backend
2. Added a cinder-ceph service foo-storage and relate to cinder
3. Removed relation cinder - cinder-ceph
This reconfigured the original rbd backend in cinder.conf, however it left an override for ceph authx behind:
cat /etc/init/ cinder- volume. override
env CEPH_ARGS="--id foo-storage"
As the foo-storage key doesn't exist at this point the cinder-volume service errored out with (redacted):
2018-05-04 14:22:19.576 4010160 ERROR cinder. volume. manager [req-fd6ff4fd- f041-47c7- a2da-144ea353fe f9 8bb46780e82b40c 594476a7609b1b8 8f 5f182a7b537a4aa 7b40832ed4bbbfb ed - - -] Driver initialize connection failed (error: Unexpected error while running command. ceph.client. foo-storage. keyring: (2) No such file or directory\ n2018-05- 04 14:22:19.561560 7ff990749700 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication\ n2018-05- 04 14:22:19.5
Command: ceph mon dump --format=json --id cinder --cluster ceph
Exit code: 1
Stdout: u''
Stderr: u'2018-05-04 14:22:19.561548 7ff990749700 -1 auth: unable to find a keyring on /etc/ceph/
61562 7ff990749700 0 librados: client.foo-storage initialization error (2) No such file or directory\nError connecting to cluster: ObjectNotFound\n').
Afaict those env settings are done on ceph-relation- changed in set_ceph_ env_variables( )