OpenStack External cloud provider configuration does not reference correct OpenStack certificate path

Bug #1840318 reported by Ed Stewart
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Kubernetes Control Plane Charm
Fix Released
High
Cory Johns

Bug Description

Testing OpenStack Rocky with Kubernetes 1.14 using edge charms to test external cloud provider support for Octavia load balancers.

Using edge charm versions kubernetes-worker-570 and kubernetes-master-723.

With these charms and with an SSL enabled OpenStack, the cluster doesn't really form - the openstack-cloud-controller-manager pods all stay in CrashLoopBackOff

This is happening because the cloud controller manager can't find the openstack ca cert:
I0815 07:55:37.035804 1 serving.go:319] Generated self-signed cert in-memory
W0815 07:55:37.432173 1 client_config.go:541] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0815 07:55:37.436745 1 controllermanager.go:117] Version: v0.0.0-master+$Format:%h$
W0815 07:55:37.436774 1 plugins.go:118] WARNING: openstack built-in cloud provider is now deprecated. Please use 'external' cloud provider for openstack: https://github.com/kubernetes/cloud-provider-openstack
I0815 07:55:37.437128 1 openstack.go:236] New openstack client created failed with config
F0815 07:55:37.437143 1 controllermanager.go:121] Cloud provider could not be initialized: could not init cloud provider "openstack": open /etc/kubernetes/openstack-ca.cert: no such file or directory

The issue that the cloud-config secret contains the following path to the CA cert:
[Global]
auth-url = https://auth.xxxx.customerb.internal:5000/v3
...
ca-file = /etc/kubernetes/openstack-ca.cer

whereas the pod config is mounting the cloud-config secret at /etc/config, therefore I think the ca-file will actually be at /etc/config/endpoint-ca.cert (different path and filename)

Manually editing the embedded cloud.conf file in the cloud-config secret (generated by the kubernetes-common layer of the charm) to specify the ca path to ca-file = /etc/config/endpoint-ca.cert instead of ca-file = /etc/kubernetes/openstack-ca.cert brings up the openstack-cloud-controller-manager pods and enables the rest of the cluster to come up.

The issue appears to be this line: https://github.com/charmed-kubernetes/layer-kubernetes-common/blame/master/lib/charms/layer/kubernetes_common.py#L451

Tags: atos
Changed in charm-kubernetes-master:
assignee: nobody → Cory Johns (johnsca)
status: New → Triaged
importance: Undecided → High
Revision history for this message
Ed Stewart (emcs2) wrote :
Changed in charm-kubernetes-master:
milestone: none → 1.15+ck2
status: Triaged → Fix Committed
Revision history for this message
Cory Johns (johnsca) wrote :

This is currently available in the edge channel of the charmed-kubernetes bundle (specifically cs:~containers/kubernetes-master-730)

Changed in charm-kubernetes-master:
milestone: 1.15+ck2 → 1.16
Changed in charm-kubernetes-master:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.