microk8s: coredns does not default to host resolver

Bug #2018354 reported by James Page
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Snap
Triaged
High
Hemanth Nakkina

Bug Description

We enable the dns addon in K8S as part of the MicroK8S deployment steps; this is enabled without any configuration for an upstream resolver - however the coredns configmap ends up with the google DNS servers populated.

As a result containers can't resolve external services in environments where these resolvers are not addressable.

James Page (james-page)
Changed in snap-openstack:
importance: Undecided → High
Revision history for this message
James Page (james-page) wrote :

microk8s v1.26.4 5222 1.26-strict/stable canonical✓ -

Revision history for this message
James Page (james-page) wrote :

microk8s active 1 microk8s latest/edge 37 no

Revision history for this message
Hemanth Nakkina (hemanth-n) wrote :
Changed in snap-openstack:
assignee: nobody → Hemanth Nakkina (hemanth-n)
status: New → Fix Committed
Revision history for this message
James Page (james-page) wrote :

I think this is actually an automatic behaviour in microk8s itself - the dns addon attempts to figure out the host resolver and if that fails it falls back to Google DNS.

Revision history for this message
James Page (james-page) wrote :

$ sudo microk8s enable dns
Infer repository core for addon dns
Enabling DNS
No valid resolv.conf file could be found
Falling back to 8.8.8.8 8.8.4.4 as upstream nameservers
Applying manifest
serviceaccount/coredns created
configmap/coredns created
deployment.apps/coredns created
service/kube-dns created
clusterrole.rbac.authorization.k8s.io/coredns created
clusterrolebinding.rbac.authorization.k8s.io/coredns created
Restarting kubelet
DNS is enabled

Changed in snap-openstack:
status: Fix Committed → Confirmed
Revision history for this message
James Page (james-page) wrote (last edit ):

digging a bit further - the dns addon enable script looks at the kubelet args for a resolver - if that's not found, it uses a python script to find a resolve.conf file in:

DEFAULT_RESOLV_CONFS = [
    "/etc/resolv.conf",
    "/run/systemd/resolve/resolv.conf",
]

however the snap is strictly confined and can't read these locations - so falls back to the google servers.

https://github.com/canonical/microk8s-core-addons/blob/main/addons/dns/enable

Revision history for this message
James Page (james-page) wrote :

Potential resolutions - switch to classic confinement for microk8s, revert the commit that prompts for a DNS resolver as part of the bootstrap process.

Revision history for this message
Hemanth Nakkina (hemanth-n) wrote :

It is problem only in LXD provider based VMs when IPv6 address is allocated.
I dont see issues when openstack snap is used with VMs launched on openstack ipv4 network

https://github.com/canonical/microk8s/blob/master/scripts/find-resolv-conf.py#L24 resulted in ValueError for IPv6 for python 3.6.9 (microk8s 1.26/stable-strict snap uses python 3.6.9)

find-resolv-conf.py works great with 3.10.x and IPv6.

Resolution is to disable ipv6 on lxd

James Page (james-page)
Changed in snap-openstack:
status: Confirmed → Triaged
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.