Comment 2 for bug 1840933

Revision history for this message
Jay Kuri (jk0ne) wrote : Re: etcd snapshots aren't saving other namespaces

I did not run the action with keys-version=v3. I was unaware of that option (it's not in the docs FYI)

I did some investigation yesterday and I found a few things.

The missing db file seems to be a bug in pre 3.1.8 etcd itself (we have 3.0.17 in the affected environment)

https://github.com/etcd-io/etcd/issues/8331

I also did some testing with snapshots created via the documented process. If you shut down etcd manually on the unit and touch the member/snap/db file as mentioned in the above ticket, when etcd is restarted it does allow the process to proceed.

Also - when the snapshot is restored, the cluster memberships are restored with it. This causes another etcd crash if restoring to a different environment (or a rebuild of the existing one) where the etcd units have changed.

Testing with snapshots from a different environment (with the above workarounds) I was able to get the data restored and verify the k8s data was present.