@nobuto Thanks for the added details regarding "option redispatch". When I observed this, my CLI was doing pretty well with the API and getting consistent results.
It seems I've unearthed a different bug!
I see in my kubelet logs an attempt to directly talk to a single kubernetes-master unit's IP to query the API from the kubelet.
So, ultimately, the failure mode I experienced seems to actually be related to which IP is provided to the kubernetes-worker over the relation. I'm going to gather additional information now.
@nobuto Thanks for the added details regarding "option redispatch". When I observed this, my CLI was doing pretty well with the API and getting consistent results.
It seems I've unearthed a different bug!
I see in my kubelet logs an attempt to directly talk to a single kubernetes-master unit's IP to query the API from the kubelet.
2022-02- 01T15:17: 39Z kubelet. daemon[ 3830984] : E0201 15:17:39.710823 3830984 controller.go:178] failed to update node lease, error: Put "https:/ /192.168. 200.123: 6443/apis/ coordination. k8s.io/ v1/namespaces/ kube-node- lease/leases/ juju-e06743- kubernetes- 0?timeout= 10s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
So, ultimately, the failure mode I experienced seems to actually be related to which IP is provided to the kubernetes-worker over the relation. I'm going to gather additional information now.