Sometimes CK model deployment gets stuck with etcd and calico colocated on the same machine

Hi team.

We're facing intermittent issue with certain Charmed Kubernetes deployments getting stuck: see the attached juju status output sample.

First of all, I'll give a few observations:
- In this specific case etcd and calico charms are co-located on the same machines (if etcd units were placed in a lxd containers there would be no calico);
- Sometimes calico charm tries to access etcd cluster before it was actually initialized (race condition?);
- calicoctl doesn't have a timeout in case something goes wrong: https://github.com/projectcalico/calico/issues/5266
- Calico charm may take machine-wide juju lock while calling calicoctl

We suppose that if all those factors come together there is a chance for the deployment to become stuck like in the aforementioned sample: calico charm calls calicoctl to save some calico data, such as pool configuration into etcd cluster before the cluster has been initialized, which causes calicoctl to hang. As the charm calls calicoctl taking juju machine lock, this causes the whole machine to freeze waiting for calicoctl to terminate, and that never happens because of the calicoctl issue. Because calico runs on all the K8s nodes, it seems like the whole model gets stuck.

I'm not sure if this is a calico or etcd charm bug, filing on calico-charm project initially. Please feel free to reassign it to the proper project.