interface not removed

This is a known issue and has been the case for a long time: http://manpages.ubuntu.com/manpages/focal/man8/netplan-apply.8.html#known%20issues

Would the workaround described in the manpage (i.e. calling "ip link delete dev br54" manually) work for you?

I'll check what I can do about it inside the netplan CLI & DBus API.

I can confirm this happens using normal netplan set/netplan apply on an current Ubuntu Impish installation, using netplan 0.103:

root@ii:~# ip l | grep br54
root@ii:~# netplan get
network:
  ethernets:
    eth0:
      dhcp4: true
  version: 2
root@ii:~# netplan set network.bridges.br54.dhcp4=true
root@ii:~# netplan apply
root@ii:~# netplan get
network:
  bridges:
    br54:
      dhcp4: true
  ethernets:
    eth0:
      dhcp4: true
  version: 2
root@ii:~# ip l | grep br54
4: br54: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
root@ii:~# netplan set network.bridges.br54.dhcp4=null
root@ii:~# netplan apply
root@ii:~# netplan get
network:
  ethernets:
    eth0:
      dhcp4: true
  version: 2
root@ii:~# ip l | grep br54
4: br54: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000

The problem here is that netplan is stateless and does not know which interfaces were defined before we called 'netplan apply'. We would somehow need to pass the "YAML state before we modified it" into netplan apply, so that it can calculate the delta of old/new interfaces defined in there.

Actually, it seems to be working if the "NetworkManager" renderer is being used, as NM is cleaning up after itself, but systemd-networkd is not: https://github.com/systemd/systemd/issues/6600

In the DBus API Config/Set/Try/Apply case, we could probably store the old netplan state somewhere and make 'netplan apply' aware of it, so that it can calculate the delta and clean up after systemd-networkd.

Should be fixed in: https://github.com/canonical/netplan/pull/231

This bug was fixed in the package netplan.io - 0.103-0ubuntu6

---------------
netplan.io (0.103-0ubuntu6) impish; urgency=medium

  * Add d/p/0005-Implement-YAML-state-tracking-and-use-it-in-the-DBus.patch:
    Allow to pass a state to netplan apply/try so it can cleanup unused
    virtual network interfaces after itself. Make use of this functionality
    inside the DBus Config.Try()/Apply() API and the 'netplan try' CLI.
    (LP: #1943120)

 -- Lukas Märdian <email address hidden> Mon, 27 Sep 2021 16:47:30 +0200

Hello Michael, or anyone else affected,

Accepted netplan.io into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.103-0ubuntu5~21.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Michael, or anyone else affected,

Accepted netplan.io into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/netplan.io/0.103-0ubuntu5~20.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted netplan.io (0.103-0ubuntu5~21.04.2) for hirsute have finished running.
The following regressions have been reported in tests triggered by the package:

netplan.io/0.103-0ubuntu5~21.04.2 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/hirsute/update_excuses.html#netplan.io

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

I Tested netplan.io 0.103-0ubuntu5~21.04.2 inside a fresh Hirsute LXD container:
$ lxc launch ubuntu-daily:hirsute hh
$ lxc exec hh bash

=> Enabled hirsute-proposed & installed/upgraded netplan.io

root@hh:~# dpkg -l | grep netplan
ii libnetplan0:amd64 0.103-0ubuntu5~21.04.2
ii netplan.io 0.103-0ubuntu5~21.04.2

root@hh:~# netplan set network.bridges.br54.dhcp4=false
root@hh:~# netplan get # check that br54 is present
network:
  bridges:
    br54:
      dhcp4: false
  ethernets:
    eth0:
      dhcp4: true
root@hh:~# netplan apply
root@hh:~# ip link show br54 && echo OK
3: br54: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether 7a:32:79:fc:e6:a2 brd ff:ff:ff:ff:ff:ff
OK
root@hh:~# mkdir -p /tmp/np_backup/etc/ && cp -r /etc/netplan /tmp/np_backup/etc/
root@hh:~# netplan set network.bridges.br54=null
root@hh:~# netplan get # check that br54 is NOT present
network:
  ethernets:
    eth0:
      dhcp4: true
root@hh:~# netplan apply --state /tmp/np_backup
root@hh:~# ip link show br54 && echo ERROR
Device "br54" does not exist.

In accordance with https://wiki.ubuntu.com/NetplanUpdates I'm also attaching the full autopkgtest logs for those builds:
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/hirsute_amd64.log
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/hirsute_arm64.log
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/hirsute_armhf.log
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/hirsute_ppc64el.log
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/hirsute_s390x.log

The mentioned autopkgtest regression is gone after a retry.

I Tested netplan.io 0.103-0ubuntu5~20.04.2 inside a fresh Focal LXD container:
$ lxc launch ubuntu-daily:focal ff
$ lxc exec ff bash

=> Enabled focal-proposed & installed/upgraded netplan.io

root@ff:~# dpkg -l | grep netplan
ii libnetplan0:amd64 0.103-0ubuntu5~20.04.2
ii netplan.io 0.103-0ubuntu5~20.04.2
root@ff:~# netplan set network.bridges.br54.dhcp4=false
root@ff:~# netplan get # check that br54 is present
network:
  bridges:
    br54:
      dhcp4: false
  ethernets:
    eth0:
      dhcp4: true
root@ff:~# netplan apply
root@ff:~# ip link show br54 && echo OK
3: br54: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 06:0f:64:90:38:53 brd ff:ff:ff:ff:ff:ff
OK
root@ff:~# mkdir -p /tmp/np_backup/etc/ && cp -r /etc/netplan /tmp/np_backup/etc/
root@ff:~# netplan set network.bridges.br54=null
root@ff:~# netplan get # check that br54 is NOT present
network:
  ethernets:
    eth0:
      dhcp4: true
netplan apply --state /tmp/np_backup
root@ff:~# ip link show br54 && echo ERROR
Device "br54" does not exist.

In accordance with https://wiki.ubuntu.com/NetplanUpdates I'm also attaching the full autopkgtest logs for those builds:
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/focal_amd64.log
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/focal_arm64.log
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/focal_armhf.log
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/focal_ppc64el.log
https://git.launchpad.net/~slyon/+git/files/tree/LP1943120/focal_s390x.log

This bug was fixed in the package netplan.io - 0.103-0ubuntu5~21.04.2

---------------
netplan.io (0.103-0ubuntu5~21.04.2) hirsute; urgency=medium

  * Backport patches from impish:
    + Add d/p/0006-netplan-set-make-it-possible-to-unset-a-whole-devtyp.patch:
      Fix unset of a devtype subtree, e.g. "netplan set network.ethernets=null"
      (LP: #1942930)
    + Add d/p/0005-Implement-YAML-state-tracking-and-use-it-in-the-DBus.patch:
      Allow to pass a state to netplan apply/try so it can cleanup unused
      virtual network interfaces after itself. Make use of this functionality
      inside the DBus Config.Try()/Apply() API and the 'netplan try' CLI.
      (LP: #1943120)

 -- Simon Chopin <email address hidden> Wed, 06 Oct 2021 12:57:35 +0200

The verification of the Stable Release Update for netplan.io has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package netplan.io - 0.103-0ubuntu5~20.04.2

---------------
netplan.io (0.103-0ubuntu5~20.04.2) focal; urgency=medium

  * Backport patches from impish:
    + Add d/p/0006-netplan-set-make-it-possible-to-unset-a-whole-devtyp.patch:
      Fix unset of a devtype subtree, e.g. "netplan set network.ethernets=null"
      (LP: #1942930)
    + Add d/p/0005-Implement-YAML-state-tracking-and-use-it-in-the-DBus.patch:
      Allow to pass a state to netplan apply/try so it can cleanup unused
      virtual network interfaces after itself. Make use of this functionality
      inside the DBus Config.Try()/Apply() API and the 'netplan try' CLI.
      (LP: #1943120)

 -- Simon Chopin <email address hidden> Wed, 06 Oct 2021 12:57:35 +0200