stale relation data for sdn-ip affects kubelet clusterDNS
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Kubernetes Control Plane Charm |
Triaged
|
High
|
Unassigned |
Bug Description
We observed a problem where kubernetes-
Consider the scenario where k-c-p/0 is the leader and discovers the cluster dns service ip to be x.y.z.119. It transmits this data over the kube-control relation, which is eventually consumed by the kubernetes-worker units and written to /root/cdk/
...
clusterDNS:
- x.y.z.119
...
This data originates from the send_cluster_
Which is gated by the `cdk-addons.
https:/
If leadership changes to k-c-p/1 and the dns service IP changes, k8s worker units will see both the previous and current IPs on the relation and prefer the old leader's value for sdn-ip (k-c-p/0 unit name < k-c-p/1). This will lead to a mis-configuration of the k8s worker kubelet service.
There are a few ways to fix this:
- adjust charms.reactive to detect when a leader is sending data over a relation and prefer that
- clear relation data keys from k-c-p units on leadership change
- fire send_cluster_
Option 3 feels the safest to implement to ensure dns info is consistent for all k-c-p units regardless of leadership.
Changed in charm-kubernetes-master: | |
status: | New → Triaged |
importance: | Undecided → High |
Changed in charm-kubernetes-master: | |
milestone: | none → 1.28 |
Changed in charm-kubernetes-master: | |
milestone: | 1.28+ck1 → 1.29 |
It seems likely that this bug will be fixed with the re-write in the ops framework.