Duplicate default routes on VMs with multiple NICs

Ouch, that seems like a nasty bug.

I think the "netplan way" here would be to find a systemd-networkd and/or networkmanager config option, and expose that in the config file. I'll have a look and see if there's one that fits the bill.

Regards,
Daniel

Hi,

It looks like we'd need to implement support for: (from systemd.network)

UseDNS=
UseNTP=
UseHostname=
UseDomains=
UseRoutes=

to get you at least some of the functionality you had.

I wonder if in the shorter term we could implement route metric support as discussed in LP: #1771834. That seems to contemplate a similar scenario (multiple interfaces with DHCP). Would that be a workable stop-gap for you?

Regards,
Daniel

Hi Daniel,
for the moment I reverted to static configuration for all but the primary NIC.
It is currently the only reliable setup, as systemd DHCP seems to add redundand default routes intermittently when UseRoutes is disabled.
Not ideal, but the reqired stopgap.
Cheers!Michael
-------- Ursprüngliche Nachricht --------Von: Daniel Axtens <email address hidden> Datum: 23.07.18 05:01 (GMT+01:00) An: <email address hidden> Betreff: [Bug 1776228] Re: Duplicate default routes on VMs with multiple NICs
Hi,

It looks like we'd need to implement support for: (from systemd.network)

UseDNS=
UseNTP=
UseHostname=
UseDomains=
UseRoutes=

to get you at least some of the functionality you had.

I wonder if in the shorter term we could implement route metric support
as discussed in LP: #1771834. That seems to contemplate a similar
scenario (multiple interfaces with DHCP). Would that be a workable stop-
gap for you?

Regards,
Daniel

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1776228

Title:
  Duplicate default routes on VMs with multiple NICs

Status in netplan:
  New

Bug description:
  I'm facing issues with OpenStack VMs built from Ubuntu 18.04 cloud
  image and configured with multiple NICs. I have a solution working on
  16.04, which doesn't do any more since the introduction of netplan.

  Requirements:

   - Multiple interfaces connected to different networks.
   - All NICs configured via DHCP to ensure IP address, but also parameters like MTU adapt to the environment automatically.
   - Default route is supposed to use one specific NIC and network.

  The solution on 16.04 was customization of /etc/dhcp/dhclient.conf,
  such that all unique parameters are retrieved from only that
  interface. Example with ens3 being the intended authoritative
  interface:

    option rfc3442-classless-static-routes code 121 = array of unsigned integer 8;
    send host-name = gethostname();
    request subnet-mask, broadcast-address, time-offset,
            netbios-name-servers, netbios-scope, interface-mtu,
            rfc3442-classless-static-routes;
    interface "ens3" { request routers, domain-name, domain-name-servers, domain-search, host-name,
                       dhcp6.name-servers, dhcp6.domain-search, dhcp6.fqdn, dhcp6.sntp-servers, ntp-servers; }

  This does no longer work with netplan. I couldn't figure any netplan
  properties to configure DHCP client to that granularity. And
  /etc/dhcp/dhclient.conf no longer seems to be honored. As I can see in
  VM startup console log, each interface is getting default route
  assigned and the system remains unresponsive most of the times.

  How am I supposed to solve this on a netplan managed system?

To manage notifications about this bug go to:
https://bugs.launchpad.net/netplan/+bug/1776228/+subscriptions

Any news or updates on this?

Updated bug since this has been landed in Disco already, but still is in progress for SRU.

Hello Michael, or anyone else affected,

Accepted netplan.io into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.96-0ubuntu0.18.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Michael, or anyone else affected,

Accepted netplan.io into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.96-0ubuntu0.18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Works fine with now with netplan.io 0.96-0ubuntu0.18.04.1!

The issue https://github.com/systemd/systemd/issues/9289 remains (as expected) with the current systemd 237-3ubuntu10.15, but is not directly caused by netplan. See also

https://bugs.launchpad.net/netplan/+bug/1759014/comments/35

Also verified cosmic. Worked out of the box with netplan.io 0.96-0ubuntu0.18.10.1, systemd 239-7ubuntu10.10, and a netplan configuration:

network:
    version: 2
    ethernets:
        ens3:
            dhcp4: true
            match:
                macaddress: fa:16:3e:02:43:e7
            set-name: ens3
        ens4:
            dhcp4: true
            dhcp4-overrides:
                use-routes: false
                use-dns: false
            match:
                macaddress: fa:16:3e:62:33:09
            set-name: ens4

Also confirms that systemd release v237 is responsible for NIC status detection failure in LTS, and that an upgrade to v239 would fix it there.

Thanks!

Hello Michael, or anyone else affected,

Accepted netplan.io into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.96-0ubuntu0.18.10.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Michael, or anyone else affected,

Accepted netplan.io into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.96-0ubuntu0.18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

netplan.io (0.96-0ubuntu0.18.10.2 works fine as netplan.io 0.96-0ubuntu0.18.10.1 did. What changes did make a new test necessary?

netplan.io 0.96-0ubuntu0.18.04.2 works fine as 0.96-0ubuntu0.18.04.1 did.

Subsequent hickup up of systemd 237-3ubuntu10.15 also remains as expected.

Hello Michael, or anyone else affected,

Accepted netplan.io into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.96-0ubuntu0.18.04.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

netplan.io 0.96-0ubuntu0.18.04.3 with systemd 237-3ubuntu10.19:

network:
    version: 2
    ethernets:
        ens3:
            dhcp4: true
            match:
                macaddress: fa:16:3e:28:d1:28
            set-name: ens3
        ens4:
            dhcp4: true
            dhcp4-overrides:
                use-routes: false
                use-dns: false
                route-metric: 3333
            match:
                macaddress: fa:16:3e:d7:41:1b
            set-name: ens4

Works fine, except the known issue with systemd addressed with https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1823730.

This bug was fixed in the package netplan.io - 0.96-0ubuntu0.18.10.2

---------------
netplan.io (0.96-0ubuntu0.18.10.2) cosmic; urgency=medium

  * d/p/0001-Partially-revert-the-change-for-enabling-systemd-net.patch:
    Partially revert changes to networkd jobs ordering: leave systemd-networkd
    enabled in multi-user.target instead of network-online.target, as in some
    minimal setups there might be no job requiring network-online.target,
    whereas the natural target to reach for booting is multi-user.target.
    (LP: #1821867)

netplan.io (0.96-0ubuntu0.18.10.1) cosmic; urgency=medium

  * Backport netplan.io 0.96 to 18.10.
  * debian/patches/glib_changes.patch: Patch tests to work again on older GLib.
  * debian/control: adjust Depends for cosmic / re-add nplan package.

netplan.io (0.96-0ubuntu1) disco; urgency=medium

  * New upstream release 0.96.
    - Moved netplan-wpa@ services to earlier at boot (LP: #1819014)
    - Restart services when unconfiguring (LP: #1811868)
    - Use the .nmconnection extension for NM keyfiles (LP: #1817655)
    - Fixed integration tests runner to correctly report failures
    - Enforce integrity for use-routes in networkd backend.
    - Ensure terminal state at end of test (LP: #1817660)
    - Various small test fixes.
    - Fix typos in documentation.
  * debian/control: Update Maintainer for ubuntu upload.

netplan.io (0.95-2) unstable; urgency=medium

  * Set Priority to optional (Closes: #920327).

netplan.io (0.95-1) unstable; urgency=medium

  * New upstream release.
  * Update autopkgtests from the upstream.
  * Add debian/watch following GitHub releases.
  * Add Homepage (Closes: #917233).

netplan.io (0.95) disco; urgency=medium

  * New upstream release:
    - Added support for WPA Enterprise / 802.1x authentication (LP: #1739578)
    - Added support for setting up IP tunnels; supporting the types: ipip,
      gretap, VTI, ISATAP (NetworkManager only), sit, gre, ipip6 and ip6ip6.
      + Fixes sit (ipv6) tunnels using Hurricane Electric (LP: #1799487)
    - Add support to override networkd UseMTU setting (LP: #1807273)
    - Generate output files in dependency order
    - Refactored unit and integration tests, along with various cleanups.
    - Add DHCP overrides to control route usage and default metric for DHCP
      routes. (LP: #1776228)
    - Mitigate against bad matching on devices behind bonds then they share
      the same MAC from a physical interface. (LP: #1804861)
    - Added snapcraft.yaml.

netplan.io (0.90.1) disco; urgency=medium

  * Do not assume /etc/network exists in postinst, as netbase 5.5 no longer
    creates it.
  * netplan/cli/commands/ip.py: fix a flake.

netplan.io (0.90) disco; urgency=medium

  * New upstream release:
    - build: fixes for building on RPM-based distros
    - build: code prettiness changes (make indentation consistent)
    - Fix device name-changes detection (LP: #1770082)
    - Add support for IPv6 Privacy Extensions (LP: #1750392)
    - Add dhcp{4,6}-overrides to control DNS, NTP, hostname updates via DHCP
      (LP: #1759014)
    - Clarify MAC and MTU setting requirements (LP: #1800668)
    - Various documentation fixes (LP: #1800669)
    -...

The verification of the Stable Release Update for netplan.io has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

A possible SRU regression has been reported against netplan.io 0.96-0ubuntu0.18.10.2 in LP: #1825206. This version has been rolled back to -proposed while the investigation is ongoing.

The systemd issue on Bionic has been resolved with systemd 237-3ubuntu10.21: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1804478

Hello Michael, or anyone else affected,

Accepted netplan.io into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.96-0ubuntu0.18.10.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Michael, or anyone else affected,

Accepted netplan.io into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.96-0ubuntu0.18.04.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Verified netplan.io 0.96-0ubuntu0.18.04.4 using the configuration from https://bugs.launchpad.net/netplan/+bug/1776228/comments/16. Routes are configured as expected.

Verified netplan.io 0.96-0ubuntu0.18.10.3 using

network:
    version: 2
    ethernets:
        ens3:
            dhcp4: true
            match:
                macaddress: fa:16:3e:09:85:54
            set-name: ens3
        ens4:
            dhcp4: true
            dhcp4-overrides:
                use-routes: false
                use-dns: false
                route-metric: 3333
            match:
                macaddress: fa:16:3e:56:2f:d6
            set-name: ens4

Routes are configured as expected.

This bug was fixed in the package netplan.io - 0.96-0ubuntu0.18.04.4

---------------
netplan.io (0.96-0ubuntu0.18.04.4) bionic; urgency=medium

  * debian/patches/git_revert_explicit_renderer_def_ebc212a.patch: revert
    commit ebc212a: make renderer values explicit at the end of each parsing
    pass; it breaks "default" renderer behavior when multiple files may set
    a global renderer and expect the last to take effect globally.
    (LP: #1825206)
  * debian/patches/git_reorg_netdef_validation_181b583.patch: correct the
    fallout from the above change: validate netdefs in a single pass at the
    very end of parsing, once we know which is the applicable renderer. This
    makes sure tunnels get validated correctly.

netplan.io (0.96-0ubuntu0.18.04.3) bionic; urgency=medium

  * debian/patches/disable-networkd-tunnels-ipip-gre.patch: disable IPIP and
    GRE tunnel tests; those appear to be broken because neither the kernel nor
    networkd bring up the device automatically as in other releases.

netplan.io (0.96-0ubuntu0.18.04.2) bionic; urgency=medium

  * d/p/0001-Partially-revert-the-change-for-enabling-systemd-net.patch:
    Partially revert changes to networkd jobs ordering: leave systemd-networkd
    enabled in multi-user.target instead of network-online.target, as in some
    minimal setups there might be no job requiring network-online.target,
    whereas the natural target to reach for booting is multi-user.target.
    (LP: #1821867)

netplan.io (0.96-0ubuntu0.18.04.1) bionic; urgency=medium

  * Backport netplan.io 0.96 to 18.04.
  * debian/patches/glib_changes.patch: Patch tests to work again on older GLib.
  * debian/control: adjust Depends for bionic / re-add nplan package.

netplan.io (0.96-0ubuntu1) disco; urgency=medium

  * New upstream release 0.96.
    - Moved netplan-wpa@ services to earlier at boot (LP: #1819014)
    - Restart services when unconfiguring (LP: #1811868)
    - Use the .nmconnection extension for NM keyfiles (LP: #1817655)
    - Fixed integration tests runner to correctly report failures
    - Enforce integrity for use-routes in networkd backend.
    - Ensure terminal state at end of test (LP: #1817660)
    - Various small test fixes.
    - Fix typos in documentation.
  * debian/control: Update Maintainer for ubuntu upload.

netplan.io (0.95-2) unstable; urgency=medium

  * Set Priority to optional (Closes: #920327).

netplan.io (0.95-1) unstable; urgency=medium

  * New upstream release.
  * Update autopkgtests from the upstream.
  * Add debian/watch following GitHub releases.
  * Add Homepage (Closes: #917233).

netplan.io (0.95) disco; urgency=medium

  * New upstream release:
    - Added support for WPA Enterprise / 802.1x authentication (LP: #1739578)
    - Added support for setting up IP tunnels; supporting the types: ipip,
      gretap, VTI, ISATAP (NetworkManager only), sit, gre, ipip6 and ip6ip6.
      + Fixes sit (ipv6) tunnels using Hurricane Electric (LP: #1799487)
    - Add support to override networkd UseMTU setting (LP: #1807273)
    - Generate output files in dependency order
    - Refactored unit and integration tests, along with various cleanups.
    - Add DHCP overrides to control ...

This bug was fixed in the package netplan.io - 0.96-0ubuntu0.18.10.3

---------------
netplan.io (0.96-0ubuntu0.18.10.3) cosmic; urgency=medium

  * debian/patches/git_revert_explicit_renderer_def_ebc212a.patch: revert
    commit ebc212a: make renderer values explicit at the end of each parsing
    pass; it breaks "default" renderer behavior when multiple files may set
    a global renderer and expect the last to take effect globally.
    (LP: #1825206)
  * debian/patches/git_reorg_netdef_validation_181b583.patch: correct the
    fallout from the above change: validate netdefs in a single pass at the
    very end of parsing, once we know which is the applicable renderer. This
    makes sure tunnels get validated correctly.

netplan.io (0.96-0ubuntu0.18.10.2) cosmic; urgency=medium

  * d/p/0001-Partially-revert-the-change-for-enabling-systemd-net.patch:
    Partially revert changes to networkd jobs ordering: leave systemd-networkd
    enabled in multi-user.target instead of network-online.target, as in some
    minimal setups there might be no job requiring network-online.target,
    whereas the natural target to reach for booting is multi-user.target.
    (LP: #1821867)

netplan.io (0.96-0ubuntu0.18.10.1) cosmic; urgency=medium

  * Backport netplan.io 0.96 to 18.10.
  * debian/patches/glib_changes.patch: Patch tests to work again on older GLib.
  * debian/control: adjust Depends for cosmic / re-add nplan package.

netplan.io (0.96-0ubuntu1) disco; urgency=medium

  * New upstream release 0.96.
    - Moved netplan-wpa@ services to earlier at boot (LP: #1819014)
    - Restart services when unconfiguring (LP: #1811868)
    - Use the .nmconnection extension for NM keyfiles (LP: #1817655)
    - Fixed integration tests runner to correctly report failures
    - Enforce integrity for use-routes in networkd backend.
    - Ensure terminal state at end of test (LP: #1817660)
    - Various small test fixes.
    - Fix typos in documentation.
  * debian/control: Update Maintainer for ubuntu upload.

netplan.io (0.95-2) unstable; urgency=medium

  * Set Priority to optional (Closes: #920327).

netplan.io (0.95-1) unstable; urgency=medium

  * New upstream release.
  * Update autopkgtests from the upstream.
  * Add debian/watch following GitHub releases.
  * Add Homepage (Closes: #917233).

netplan.io (0.95) disco; urgency=medium

  * New upstream release:
    - Added support for WPA Enterprise / 802.1x authentication (LP: #1739578)
    - Added support for setting up IP tunnels; supporting the types: ipip,
      gretap, VTI, ISATAP (NetworkManager only), sit, gre, ipip6 and ip6ip6.
      + Fixes sit (ipv6) tunnels using Hurricane Electric (LP: #1799487)
    - Add support to override networkd UseMTU setting (LP: #1807273)
    - Generate output files in dependency order
    - Refactored unit and integration tests, along with various cleanups.
    - Add DHCP overrides to control route usage and default metric for DHCP
      routes. (LP: #1776228)
    - Mitigate against bad matching on devices behind bonds then they share
      the same MAC from a physical interface. (LP: #1804861)
    - Added snapcraft.yaml.

netplan.io (0.90.1) disco; urgency=medium

  * Do ...

I have same problem in ubuntu 20 , having more than default routes

netplan dchp over-ride works but I would like to propose to change the default behaviour not to allow more than a default gateway , or choose different metrics

Thanks a lot
BR