kubelet kube-proxy does not start

Bug #1850176 reported by Ashley Lai
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Kubernetes Worker Charm
Triaged
High
Unassigned

Bug Description

We have a test run that timed out waiting for kube-proxy to start in kuernetes worker.

kubernetes-worker/1 waiting idle 12 10.245.162.33 Waiting for kubelet,kube-proxy to start.

Tags: cdo-qa
Revision history for this message
Ashley Lai (alai) wrote :
Revision history for this message
Ashley Lai (alai) wrote :
Revision history for this message
Ashley Lai (alai) wrote :
Revision history for this message
Mike Wilson (knobby) wrote :

Same as was reported in https://bugs.launchpad.net/charm-kubernetes-master/+bug/1849349, but this has much more information about the deployment. Thanks, I'll take a look.

Changed in charm-kubernetes-worker:
assignee: nobody → Mike Wilson (knobby)
Revision history for this message
Mike Wilson (knobby) wrote :

I've tried reproducing with this bundle on cloud instances without any luck so far. I'm going to alter the bundle to fit on my bare metal cluster and attempt to reproduce it there.

Changed in charm-kubernetes-worker:
assignee: Mike Wilson (knobby) → nobody
George Kraft (cynerva)
Changed in charm-kubernetes-worker:
importance: Undecided → High
status: New → Triaged
Revision history for this message
Alexander Balderson (asbalderson) wrote :

We had this come up on a run recently while testing 1.19
one unit failed to start the kube-proxy service, and systemctl shows that its failing because it detected swap:

Sep 27 00:57:16 armaldo systemd[1]: snap.kubelet.daemon.service: Main process exited, code=exited, status=255/EXCEPTION
Sep 27 00:57:16 armaldo systemd[1]: snap.kubelet.daemon.service: Failed with result 'exit-code'.
Sep 27 00:57:17 armaldo systemd[1]: snap.kubelet.daemon.service: Scheduled restart job, restart counter is at 2673.
Sep 27 00:57:17 armaldo systemd[1]: Stopped Service for snap application kubelet.daemon.
Sep 27 00:57:17 armaldo systemd[1]: Started Service for snap application kubelet.daemon.
Sep 27 00:57:17 armaldo kubelet.daemon[898183]: I0927 06:57:17.391577 898183 server.go:411] Version: v1.19.0
Sep 27 00:57:17 armaldo kubelet.daemon[898183]: W0927 06:57:17.392116 898183 server.go:553] standalone mode, no API client
Sep 27 00:57:22 armaldo kubelet.daemon[898183]: W0927 06:57:22.416926 898183 nvidia.go:61] NVIDIA GPU metrics will not be available: no NVIDIA devices found
Sep 27 00:57:22 armaldo kubelet.daemon[898183]: W0927 06:57:22.429746 898183 server.go:468] No api server defined - no events will be sent to API server.
Sep 27 00:57:22 armaldo kubelet.daemon[898183]: I0927 06:57:22.429765 898183 server.go:640] --cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /
Sep 27 00:57:22 armaldo kubelet.daemon[898183]: F0927 06:57:22.430001 898183 server.go:265] failed to run Kubelet: running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: [Filename Type Size Used Priority /swap.img >
Sep 27 00:57:22 armaldo kubelet.daemon[898183]: goroutine 1 [running]:

https://solutions.qa.canonical.com/kubernetes/testRun/6abf39fa-5ce5-4522-8a47-dd89129bba2b

All the machines being deployed in this deployment have the same specs and are setup by maas using buckets. It seems odd that one unit would end up with a swap.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.