juju 3.2.3 fails to bootstrap on microk8s due to failure to verify certificate

Bug #2034599 reported by Bas de Bruijne
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Harry Pidcock

Bug Description

In test run https://solutions.qa.canonical.com/testruns/d3584e68-704a-4951-b98b-17e56b48cbf6/, the bootstrapping of a juju controller on a snapped microk8s fails with the following message:

2023-09-06-01:59:31 root DEBUG [localhost]: juju add-k8s snapped-microk8s_cloud --client --storage microk8s-hostpath
2023-09-06-01:59:34 fce.juju_snapped-microk8s_controller INFO Finished step: juju_snapped-microk8s_controller:snapped_microk8s_setup_controller. Time elapsed (hh:mm:ss.ms): 0:00:02.677265
2023-09-06-01:59:34 fce.juju_snapped-microk8s_controller INFO Starting step: juju_snapped-microk8s_controller:bootstrap
2023-09-06-01:59:34 root DEBUG ['juju', 'bootstrap', '--config', 'controller-service-type=loadbalancer', '--bootstrap-constraints', 'arch=amd64 ', '--config', 'bootstrap-timeout=1800', '--model-default', '/home/ubuntu/project/generated/juju_snapped-microk8s_controller/model_defaults.yaml', 'snapped-microk8s_cloud', 'popocatepetl']
2023-09-06-01:59:34 root DEBUG [localhost]: juju bootstrap --config controller-service-type=loadbalancer --bootstrap-constraints 'arch=amd64 ' --config bootstrap-timeout=1800 --model-default /home/ubuntu/project/generated/juju_snapped-microk8s_controller/model_defaults.yaml snapped-microk8s_cloud popocatepetl
2023-09-06-01:59:34 root DEBUG Creating Juju controller "popocatepetl" on snapped-microk8s_cloud/localhost
2023-09-06-01:59:34 root DEBUG Bootstrap to Kubernetes cluster identified as microk8s/localhost
2023-09-06-01:59:35 root DEBUG Creating k8s resources for controller "controller-popocatepetl"
2023-09-06-02:00:04 root DEBUG Downloading images
2023-09-06-02:01:47 root DEBUG Starting controller pod
2023-09-06-02:01:47 root DEBUG Bootstrap agent now started
2023-09-06-02:01:47 root DEBUG Contacting Juju controller at to verify accessibility...
2023-09-06-02:01:51 root DEBUG ERROR unable to contact api server after 0 attempts: unable to connect to API: tls: failed to verify certificate: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "juju-ca")
2023-09-06-02:07:46 root ERROR [localhost] Command failed: juju bootstrap --config controller-service-type=loadbalancer --bootstrap-constraints 'arch=amd64 ' --config bootstrap-timeout=1800 --model-default /home/ubuntu/project/generated/juju_snapped-microk8s_controller/model_defaults.yaml snapped-microk8s_cloud popocatepetl

The juju controller can't be accessed because the certificate can't be verified. Did something around the certificates change in juju 3.2.3?

More logs and configs can be found here: https://oil-jenkins.canonical.com/artifacts/d3584e68-704a-4951-b98b-17e56b48cbf6/index.html

Revision history for this message
Bas de Bruijne (basdbruijne) wrote :

We are not hitting this issue consistently with 3.2.3, there are multiple runs that pass the microk8s bootstrapping.

tags: added: cdo-qa foundations-engine
Thomas Miller (tlmiller)
Changed in juju:
importance: Undecided → High
assignee: nobody → Thomas Miller (tlmiller)
assignee: Thomas Miller (tlmiller) → nobody
assignee: nobody → Harry Pidcock (hpidcock)
Revision history for this message
Alexander Balderson (asbalderson) wrote :
Changed in juju:
status: New → Triaged
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.