Comment 6 for bug 1848392

The info pertinent to this bug is strafed across several locations, so I'll attempt to bring them together here.

The issue was raised on Discourse here:
https://discourse.jujucharms.com/t/incorrect-binding-used-for-cmr/2232

It came up during testing of this patch:
https://github.com/openstack-charmers/charm-interface-vault-kv/pull/6

There is a fundamental issue here around cross-model relations in that when a relation is established, the only guarantee is that the controllers can communicate. There is no explicit contract or recording of data that declares a network route between the two *units*.

The old version of the charm logic works in some cases. I think this is when:
- There is only one subnet in play on either end and those subnets happen to be routable to each other.
- The relation endpoint is bound to a space with a subnet(s) that are routable.
- The relation endpoint is unbound and the unit's public address happens to be in a routable subnet.

I believe the new charm logic (using network_get) would work under the same circumstances.

It fails here, because the secrets endpoint is not bound to a space. When calling network_get in the context of a relation, the question being asked is "as a unit, what do I know about my networking in this context?". When the relation endpoint is not bound to a space, the unit's default public address is returned. In this case it is the local cloud address that the other end knows nothing about.

The Juju team has discussed some options that might help here like adding parameters to the offer/consume commands such as an explicit space that would be used for communication, but at present they are just ideas.

Options to make this work with the current Juju primitives are:

1) Bind the "secrets" endpoint to the "external" space, so that there is an explicit statement that the external subnet is the one used in the relation. This is undesirable when Vault is servicing both the under and over clouds, as it means the under-cloud uses the unit's virtual IP.

2) Modify the charm with another relation endpoint for use by external consumers, and bind each of the endpoints to the appropriate space.

I'm sure this with need further discussion, so if I have anything wrong in my assumptions, or if any of this is unclear, please contact me via any of the usual channels.