Looking at the logs I think the issue is around not having an upgrade step form 1.20.x to 1.22.x which filters out any lxcbr0 addresses from the API hosts/ports. 10.0.3.1 was among the list of API hostsPorts along with quite a lot of IPv6 link-local addresses:
So: 1) to fix the issue some mongo surgery is needed (to salvage the environment) - drop all 10.0.3.0/24 and fe80::/64 addresses from the hostPorts (stateServers?) collections; 2) to make sure it doesn't happen again - do an upgrade step, run on the DatabaseMaster which gets current API hostPorts converts them to []network.Address for each server (network.HostsWithoutPorts?) and calls network.FilterLXCAddresses() on these slices, and sets the filtered api hosts/ports back in mongo.
Looking at the logs I think the issue is around not having an upgrade step form 1.20.x to 1.22.x which filters out any lxcbr0 addresses from the API hosts/ports. 10.0.3.1 was among the list of API hostsPorts along with quite a lot of IPv6 link-local addresses:
https:/ /pastebin. canonical. com/133025/
So: 1) to fix the issue some mongo surgery is needed (to salvage the environment) - drop all 10.0.3.0/24 and fe80::/64 addresses from the hostPorts (stateServers?) collections; 2) to make sure it doesn't happen again - do an upgrade step, run on the DatabaseMaster which gets current API hostPorts converts them to []network.Address for each server (network. HostsWithoutPor ts?) and calls network. FilterLXCAddres ses() on these slices, and sets the filtered api hosts/ports back in mongo.