Application status could indicate service failures

Bug #1868367 reported by Barry Price
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Kubernetes Control Plane Charm
Triaged
High
Unassigned
Openstack Integrator Charm
Triaged
High
Unassigned

Bug Description

After deploying the integrator to a cluster, 'juju status' suggests all is well:

App Version Status Scale Charm Store Rev OS Notes
openstack-integrator train active 1 openstack-integrator local 0 ubuntu

Unit Workload Agent Machine Public address Ports Message
openstack-integrator/0* active idle 13 10.15.210.18 Ready

And the storage class certainly shows up:

$ kubectl get sc --all-namespaces
NAME PROVISIONER AGE
cdk-cinder csi-cinderplugin 9d

But on closer inspection:

$ kubectl get po --all-namespaces | grep csi-cinder
kube-system csi-cinder-controllerplugin-0 2/4 CrashLoopBackOff 1300 45h
kube-system csi-cinder-nodeplugin-p6lzs 1/2 CrashLoopBackOff 3391 9d
kube-system csi-cinder-nodeplugin-schgg 1/2 CrashLoopBackOff 2622 9d

See LP:1868368 for the above issue itself, but is it possible (and if so, is it a good idea) for the charm to notice that the pods haven't started, and adjust application status to e.g. "blocked"?

Tags: canonical-is
Barry Price (barryprice)
description: updated
tags: added: canonical-is
George Kraft (cynerva)
Changed in charm-kubernetes-master:
importance: Undecided → High
Changed in charm-openstack-integrator:
importance: Undecided → High
Changed in charm-kubernetes-master:
status: New → Triaged
Changed in charm-openstack-integrator:
status: New → Triaged
Revision history for this message
Martin Kalcok (martin-kalcok) wrote :

Pods mentioned in the description (along with some other k8s resources) are currently "managed" by cdk-addons.
When "openstack-integrator" joins with "kubernetes-master", the k8s-master executes "cdk-addons" with parameter "openstack-enabled=true" which deploys appropriate k8s resources. However, there's no mechanism that would let "kubernetes-master" unit know which resources were created so it can not monitor if they were created successfully.

There's currently an effort to make cdk-addons obsolete and replace them with standalone charms. I think that instead of inventing a mechanism that would let "kubernetes-master" know which kubernetes resources were created by cdk-addons, we could instead make it a responsibility of "openstack-integrator" to create and monitor these resources.
In practice, this would involve moving k8s resource templates from cdk-addons to "openstack-integrator" and using k8s API to directly create resources from the charm.

There's only one drawback to this approach, we would have to require that the "openstack-integrator" is a subordinate charm of the "kubernetes-master" so that it can have direct access to the kube config, and use kubernetes API directly.

What do you all think about this approach?

Revision history for this message
George Kraft (cynerva) wrote :

Neither kubernetes-master nor openstack-integrator should be responsible for creating or monitoring the openstack-cloud-controller-manager and csi-cinder pods. The plan is to create new standalone k8s charms for those instead, with support for cross-model relations to openstack-integrator or kubernetes-master as needed. The new charms would be responsible for monitoring their own workloads.

I think the correct answer here is to mark this issue as "won't fix" since the pods in question are about to be deprecated. What do you think?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.