Comment 1 for bug 1933355

I'm about to wipe out this stack and switch over to Kolla/Kayobe or something actually FOSS, so won't be able to get any debug data after i wipe the hosts.
However, the permissions errors are also showing up in the journal of the relevant snap service in the lxd container unit:
```
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 etcd[326634]: Go Version: go1.13.10
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 etcd.etcd[326634]: [WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 etcd[326634]: Go OS/Arch: linux/amd64
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 etcd[326634]: setting maximum number of CPUs to 16, total number of available CPUs is 16
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 etcd[326634]: failed to detect default host (operation not permitted)
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 etcd[326634]: the server is already initialized as member before, starting as etcd member...
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 etcd[326634]: peerTLS: cert = /var/snap/etcd/common/server.crt, key = /var/snap/etcd/common/server.key, trusted-ca = /var/snap/etcd/common/ca.crt, client-cert-auth = true, crl-file =
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 etcd[326634]: listen tcp 0.0.0.0:2380: socket: permission denied
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 systemd[1]: snap.etcd.etcd.service: Main process exited, code=exited, status=1/FAILURE
Jun 23 21:42:33 juju-4e82f9-0-lxd-1 systemd[1]: snap.etcd.etcd.service: Failed with result 'exit-code'.
Jun 23 21:42:44 juju-4e82f9-0-lxd-1 systemd[1]: snap.etcd.etcd.service: Scheduled restart job, restart counter is at 5432.
Jun 23 21:42:44 juju-4e82f9-0-lxd-1 systemd[1]: Stopped Service for snap application etcd.etcd.
Jun 23 21:42:44 juju-4e82f9-0-lxd-1 systemd[1]: Started Service for snap application etcd.etcd.
Jun 23 21:42:44 juju-4e82f9-0-lxd-1 etcd.etcd[326745]: Running as system with data in /var/snap/etcd/230
Jun 23 21:42:44 juju-4e82f9-0-lxd-1 etcd.etcd[326745]: Configuration from /var/snap/etcd/common/etcd.conf.yml
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd.etcd[326745]: [WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: Loading server configuration from "/var/snap/etcd/common/etcd.conf.yml". Other configuration command line flags and environment variables will be ignored if provided.
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: etcd Version: 3.4.5
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd.etcd[326745]: [WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: Git SHA: Not provided (use ./build instead of go build)
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: Go Version: go1.13.10
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: Go OS/Arch: linux/amd64
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: setting maximum number of CPUs to 16, total number of available CPUs is 16
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: failed to detect default host (operation not permitted)
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: the server is already initialized as member before, starting as etcd member...
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: peerTLS: cert = /var/snap/etcd/common/server.crt, key = /var/snap/etcd/common/server.key, trusted-ca = /var/snap/etcd/common/ca.crt, client-cert-auth = true, crl-file =
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 etcd[326745]: listen tcp 0.0.0.0:2380: socket: permission denied
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 systemd[1]: snap.etcd.etcd.service: Main process exited, code=exited, status=1/FAILURE
Jun 23 21:42:45 juju-4e82f9-0-lxd-1 systemd[1]: snap.etcd.etcd.service: Failed with result 'exit-code'.
```
Some proprietary snap confinement nonsense from what i can see, but it broke absolutely everything in the stack above, so way too fragile for production use like this (sometime a year down the line this might happen again killing vault, sql, ceph key management, and all services depending on those things).
Removed and reinstalled the snap now that the snap store is back up (was down yesterday evening, returning empty responses despite the service status page's claims that outages were resolved earlier, furthering my resolve to avoid reliance on Canonical's proprietary pieces) - no dice, same results, can't bind sockets, stack is toast.

From an architectural perspective, the whole idea of packaging (debs, in Ubuntu's case) is to have data (including executable binaries and code) delivered to the user's system. Snaps are not that - they are a non-standard runtime context controlled by a single company with commercial intent, and are not interchangeable with Open Source packages on any level.