Comment 5 for bug 1896542

Looking at test run 07adb558-f37c-4474-9f97-267f5b243827...

Output from `juju show-unit vault/1` indicates that kubeapi-load-balancer/0 has updated the requested sans on the relation (with "santest.example.com" being the new one):

kubeapi-load-balancer/0:
  in-scope: true
  data:
    certificate_name: 4b5072b4-9720-4878-ab36-5292e4df2904
    common_name: 10.246.166.212
    egress-subnets: 10.246.169.65/32
    ingress-address: 10.246.169.65
    private-address: 10.246.169.65
    sans: '["10.246.166.212", "10.246.169.65", "juju-04a9d1-0-lxd-1", "juju-04a9d1-0-lxd-1.nosilo.lab1.solutionsqa", "santest.example.com"]'

Vault decides to reuse a cached certificate, instead of issuing a new one:

2023-04-04 04:49:56 INFO unit.vault/1.juju-log server.go:316 certificates:57: Processing certificate request from kubeapi-load-balancer_0 for 10.246.166.212
2023-04-04 04:49:56 INFO unit.vault/1.juju-log server.go:316 certificates:57: Reusing certificate for unit 'kubeapi-load-balancer_0' and CN '10.246.166.212' from cache.

Since kubeapi-load-balancer and kubernetes-control-plane never receive new certificates from vault, they continue to serve with the old one, which does not have santest.example.com in its SAN list:

X509v3 Subject Alternative Name:
    DNS:10.246.166.212, DNS:juju-04a9d1-0-lxd-1, DNS:juju-04a9d1-0-lxd-1.nosilo.lab1.solutionsqa, IP Address:10.246.166.212, IP Address:10.246.169.65

This caching behavior in vault appears to be new, as the change that introduced it was merged just 2 months ago[1]. I would call this a bug in the vault charm's handling of updated SANs on the relation.

[1]: https://opendev.org/openstack/charm-vault/commit/15ab73ea723d158f9105f0d603515e5d3ddaa78c