If the certificates-relation-joined hook runs before vault has been configured
then the hook fails. This is because the charm tries to access the running
vault service but at this point vault is not configured or running. This
regression appears to have been introduced by 1159e547
( https://review.opendev.org/c/openstack/charm-vault/+/828885 ). This patch
seems to incorrectly gate on the `certificates.available` flag. Despite the
name, `certificates.available` only indicates that certificate have
been requested i.e. it means "a certificate is available to be processed" it
does not mean that vault is ready.
Note that in the bundle the relation between vault-mysql-router and
mysql-innodb-cluster is missing. This simulates the situation where
a `certificates-relation-joined` fires before vault has been setup
because the initial configuration of vault is gated on
`shared-db.available` flag being set.
This bug can present itself in subtly different ways that may initially
appear like the db-router/shared-db relations are at fault. In the
output below vault/0 and vault/2 are both hitting this bug and in the
case of vault/0 the bug was hit before the unit sent its db access request
to vault-mysql-router/2 which is why vault-mysql-router/2 is reporting it
has missing data.
Unit Workload Message
vault/0 error hook failed: "certificates-relation-joined"
vault-mysql-router/2 waiting shared-db' incomplete, Waiting for proxied DB creation from cluster
vault/1* blocked Vault needs to be initialized
vault-mysql-router/1 active Unit is ready
vault/2 error hook failed: "certificates-relation-joined"
vault-mysql-router/0* active Unit is ready
If the certificates- relation- joined hook runs before vault has been configured /review. opendev. org/c/openstack /charm- vault/+ /828885 ). This patch available` flag. Despite the available` only indicates that certificate have
then the hook fails. This is because the charm tries to access the running
vault service but at this point vault is not configured or running. This
regression appears to have been introduced by 1159e547
( https:/
seems to incorrectly gate on the `certificates.
name, `certificates.
been requested i.e. it means "a certificate is available to be processed" it
does not mean that vault is ready.
The issue can be reproduced with this bundle:
series: focal
applications:
keystone- mysql-router: mysql-router:
charm: ch:mysql-router
channel: latest/edge
vault-
charm: ch:mysql-router
channel: latest/edge
mysql- innodb- cluster: innodb- cluster
charm: ch:mysql-
constraints: mem=3072M
num_units: 3
channel: latest/edge
vault:
num_units: 3
charm: ch:vault
channel: latest/edge
keystone: password: openstack
charm: ch:keystone
num_units: 1
options:
admin-
channel: latest/edge
relations: mysql-router: shared- db'
- - 'vault:shared-db'
- 'vault-
- - 'keystone: shared- db' mysql-router: shared- db' mysql-router: db-router' innodb- cluster: db-router'
- 'keystone-
- - 'keystone-
- 'mysql-
- - 'vault: certificates' certificates'
- 'keystone:
Note that in the bundle the relation between vault-mysql-router and cluster is missing. This simulates the situation where relation- joined` fires before vault has been setup db.available` flag being set.
mysql-innodb-
a `certificates-
because the initial configuration of vault is gated on
`shared-
This bug can present itself in subtly different ways that may initially router/ 2 which is why vault-mysql- router/ 2 is reporting it
appear like the db-router/shared-db relations are at fault. In the
output below vault/0 and vault/2 are both hitting this bug and in the
case of vault/0 the bug was hit before the unit sent its db access request
to vault-mysql-
has missing data.
Unit Workload Message relation- joined" mysql-router/ 2 waiting shared-db' incomplete, Waiting for proxied
DB creation from cluster mysql-router/ 1 active Unit is ready relation- joined" mysql-router/ 0* active Unit is ready
vault/0 error hook failed: "certificates-
vault-
vault/1* blocked Vault needs to be initialized
vault-
vault/2 error hook failed: "certificates-
vault-