kube-api-endpoint relation between control-plane and worker creates SPOF in OpenStack integration

Bug #1974280 reported by Nobuto Murata
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Charmed Kubernetes Bundles
Fix Released
High
George Kraft

Bug Description

In the current documentation and the overlay bundle, there is a relation for OpenStack integration as:
  - ['kubernetes-control-plane:kube-api-endpoint', 'kubernetes-worker:kube-api-endpoint']

https://ubuntu.com/kubernetes/docs/openstack-integration#api-server-load-balancer
https://raw.githubusercontent.com/charmed-kubernetes/bundle/main/overlays/openstack-lb-overlay.yaml

By doing so, worker nodes connect the control plane using a private IP of the control-plane node, which by-passes the OpenStack load balancer. When the control plane node is down, worker nodes cannot connect to other remaining and living control plane nodes.

By removing the relation, worker nodes will have the OpenStack LB's address in /root/.kube/config.

Can we revisit the previous changes and agree what's the expected config for the OpenStack integration?

https://github.com/charmed-kubernetes/bundle/pull/768
https://github.com/charmed-kubernetes/kubernetes-docs/pull/371

Revision history for this message
Nobuto Murata (nobuto) wrote :
Revision history for this message
Nobuto Murata (nobuto) wrote :

192.168.151.72 - external IP address of OpenStack LB (Octavia Amphora) as expected
10.5.5.14 - internal IP address of kubernetes-control-plane/0 as SPOF

$ juju run --unit kubernetes-worker/0 -- cat /root/.kube/config | grep server:
    server: https://192.168.151.72:443

$ juju add-relation kubernetes-worker:kube-api-endpoint kubernetes-control-plane:kube-api-endpoint
$ juju-wait -w

$ juju run --unit kubernetes-worker/0 -- cat /root/.kube/config | grep server:
    server: https://10.5.5.14:6443

$ juju remove-relation kubernetes-worker:kube-api-endpoint kubernetes-control-plane:kube-api-endpoint
$ juju-wait -w

$ juju run --unit kubernetes-worker/0 -- cat /root/.kube/config | grep server:
    server: https://192.168.151.72:443

Revision history for this message
Nobuto Murata (nobuto) wrote :
Revision history for this message
Nobuto Murata (nobuto) wrote :

The following comment looks like confirming the idea.

https://bugs.launchpad.net/charm-openstack-integrator/+bug/1959720/comments/13
> I can confirm that it is fixed in Charmed Kubernetes 1.22. After you
> upgrade to 1.22, make sure you remove the deprecated relation between
> kubernetes-worker:kube-api-endpoint and
> kubernetes-master:kube-api-endpoint. That will allow kubernetes-worker
> to get the API endpoint from the kube-control relation instead, which
> uses the loadbalanced API endpoint.

Revision history for this message
Nobuto Murata (nobuto) wrote :

Subscribing ~field-high to get a confirmation that removing the relation is the right way to the issue before applying that to the production system.

Revision history for this message
George Kraft (cynerva) wrote :

Yes, since Charmed Kubernetes 1.22, we recommend removing the relation between kubernetes-worker:kube-api-endpoint and kubernetes-control-plane:kube-api-endpoint. That will allow kubernetes-worker to obtain the proper loadbalanced address from the kube-control relation instead.

We will get the documentation updated.

Revision history for this message
George Kraft (cynerva) wrote :
Changed in charmed-kubernetes-bundles:
status: New → In Progress
assignee: nobody → George Kraft (cynerva)
importance: Undecided → High
milestone: none → 1.24+ck1
George Kraft (cynerva)
Changed in charmed-kubernetes-bundles:
status: In Progress → Fix Committed
Adam Dyess (addyess)
Changed in charmed-kubernetes-bundles:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.