Comment 31 for bug 1815101

Revision history for this message
Rafael David Tinoco (rafaeldtinoco) wrote : Re: [Bug 1815101] Re: [master] Restarting systemd-networkd breaks keepalived, heartbeat, corosync, pacemaker (interface aliases are restarted)

Balint, based on your input...

> thanks for the fixes in Eoan. Unfortunately we have a product based on
> disco and cannot move forward at this time. Being a networking shop,
> this issue has a serious effect on us and we would like to avoid moving
> to something like ifupdown2 within our stable branch.

So, Disco is EOL as it is not a LTS version, that is why it did not
get a fix (as the fix is very close to the one done in Eoan). Since
its unsupported by the community, it's up to you backport the Eoan
fixes to Disco if you'd like... you can even create a PPA for your
product and distribute along.

> For our users the real impact of the bug is not that that the interface
> that we are currently reconfiguring is suffering a downtime, but the
> fact that _all_ interfaces have their aliases removed if networkd is
> restarted. The proposed KeepConfiguration solution kind of beats the
> purpose of reconfiguring the interfaces, as old addresses are kept and
> need to be handled manually. Also it interferes with how DHCP works. I
> believe this might be an issue for others as well.

We are following systemd-networkd upstream decisions here. The option
"dhcp" only exists for CERTAIN scenarios (when root disk depends on
that connection, for iSCSI and/or NFS/ROOT for example). It is
explicitly said in the documentation:

"""
Takes a boolean or one of "static", "dhcp-on-stop", "dhcp". When
"static", systemd-networkd will not drop static addresses and routes
on starting up process. When set to "dhcp-on-stop", systemd-networkd
will not drop addresses and routes on stopping the daemon. When
"dhcp", the addresses and routes provided by a DHCP server will never
be dropped even if the DHCP lease expires. This is contrary to the
DHCP specification, but may be the best choice if, e.g., the root
filesystem relies on this connection. The setting "dhcp" implies
"dhcp-on-stop", and "yes" implies "dhcp" and "static". Defaults to
"no".
"""

and it is a question of choice: to have a window of opportunity for
duplicate IPs - in cases where there is no dynamic IP mapping to that
mac address - but possibly maintain the connection instead of causing
uninterruptable I/Os trying to shutdown a machine, for example. I
particularly don't like this option but it is not the default one and
was meant for a specific purpose.

>
> >From our point of view the ideal solution would be a combination of the
> keepalived patch that detects VIP removal and systemd version 244 that
> already supports "networkctl reconfigure" and "networkctl reload".

networkctl reconfigure/reload is a new functionality and won't be
added to previous already released versions as this is against SRU
guidelines. Systemd 244.2-1ubuntu1 is being included in 20.04, our
NEXT LTS version.

Like said before, you can try backporting systemd 244 to disco, or
bionic, if you are willing to support it on your own as it was already
EOL for community support. You should follow:
https://packaging.ubuntu.com/html/backports.html if you would like to
do that.

For the keepalived patches, they could be backported to Eoan, maybe
Bionic and Xenial depending on the amount of work. But then I would
need a practical example of why the systemd-networkd fix is no good in
most used scenarios.

> Is there any chance that v244 is backported to bionic? It is already
> included in focal and debian stable backports, but unfortunately I am
> not familiar enough with systemd development to tell what the impact of
> this would be.

Problem with backports is that they are unsupported even on supported
releases. I wouldn't be able to guarantee functionalities or fix it in
a constant basis. You can do it on your own and have it in a PPA of
your product, for example.

As systemd nowadays include networkd, udev management, sysV runtime
generators, tmpfiles creation, sockets creation, cgroups integration
for the process slices, etc etc... it is very very risky to backport
systemd to have "just" those 2 functionalities.

>
> As for keepalived, in bug #1819074 there was an ongoing investigation on
> the patch, that implements the keepalived transition on removing the
> VIP. We have traced back this functionality to this patch:
>
> https://github.com/acassen/keepalived/commit/0b1528c76d3fe8d1c5765841df86c59570a036da
>
> It was born before v1.3.6 was released, so we hope that it is self-
> contained enough for a backport if v2.0 of keepalived is not included in
> bionic-backports.

Let me check keepalived fix more closely and see what can be done for
the previous releases. As we are close to freeze date for our next LTS
release, it is unlikely that I do it before 2 weeks from now (as our
focus is in the development version entirely and I still need to fix
netplan to support the networkd KeepConfiguration functionality).

Lets keep talking.. I'll first patch netplan and go back with other
releases to check what can be done for them.

For now I would *strongly* recommend that in previous releases,
whoever wants to use HA related resource managers, to stick with
ifupdown / resolvconf / net-tools / bridge-utils / vlan package
combination as it works for is trying to be accomplished here.