System with DNS server in /etc/network/interfaces has bogus systemd-resolved config after upgrade to 22.04

Bug #1981103 reported by Uwe Schindler
42
This bug affects 8 people
Affects Status Importance Assigned to Milestone
ifupdown (Ubuntu)
Fix Released
Medium
Heitor Alves de Siqueira
Focal
Fix Released
Low
Heitor Alves de Siqueira
Jammy
In Progress
Low
Heitor Alves de Siqueira
Lunar
In Progress
Low
Heitor Alves de Siqueira

Bug Description

[ Impact ]
ifupdown configurations will stop having a working DNS server on upgrade

[ Test Plan ]
Upgrading from an Ubuntu 18.04 or earlier release, bounce a network interface with configured DNS server:

root@pangaea-pm:~# ifdown ens13; ifup ens13
/etc/network/if-down.d/resolved: 12: mystatedir: not found
/etc/network/if-up.d/resolved: 12: mystatedir: not found
/etc/network/if-up.d/resolved: 71: DNS: not found
/etc/network/if-up.d/resolved: 1: /run/network/ifupdown-inet-ens13: DNS=134.102.20.20 134.102.200.14: not found
/etc/network/if-up.d/resolved: 2: /run/network/ifupdown-inet-ens13: DOMAINS=marum.de: not found
Failed to parse DNS server address: DNS
Failed to set DNS configuration: Invalid argument

This should result in no errors, and in a valid network configuration with working DNS resolution.
It's likely that existing netplan configuration needs to be removed and migrated to classic ifupdown scripts before running the test above.

[ Where problems could occur ]
Although we're only fixing the resolved scripts, more complex network setups might see regressions in DNS resolution for some cases. We should pay special attention to bridges or virtual interfaces that have been configured through ifupdown, as well as validate the other scripts under /etc/network/if-up.d/

[ Other Info ]
Description: Ubuntu 22.04 LTS
Release: 22.04

ifupdown:
  Installed: 0.8.36+nmu1ubuntu3
  Candidate: 0.8.36+nmu1ubuntu3
  Version table:
 *** 0.8.36+nmu1ubuntu3 500
        500 http://de.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages
        100 /var/lib/dpkg/status

After upgrading a server with classic ifupdown configuration after reboot the machine had no valid dns servers anymore.

The problem is that the state file created by ifupdown using /etc/network/if-up.d/resolved looks like this:

root@pangaea-pm:~# cat /run/network/ifupdown-inet-ens13
"DNS"="134.102.20.20 134.102.200.14"
"DOMAINS"="marum.de"

The script later sources this file and causes the following errors, easy to see when you execute this:

root@pangaea-pm:~# ifdown ens13; ifup ens13
/etc/network/if-down.d/resolved: 12: mystatedir: not found
/etc/network/if-up.d/resolved: 12: mystatedir: not found
/etc/network/if-up.d/resolved: 71: DNS: not found
/etc/network/if-up.d/resolved: 1: /run/network/ifupdown-inet-ens13: DNS=134.102.20.20 134.102.200.14: not found
/etc/network/if-up.d/resolved: 2: /run/network/ifupdown-inet-ens13: DOMAINS=marum.de: not found
Failed to parse DNS server address: DNS
Failed to set DNS configuration: Invalid argument

This happened to me on three different servers, so this is a serious bug and should be fixed before 22.04 upgrades are allowed for everybody. Most servers provided by data centers like Hetzner (Germany) are configure like that. After a do-release-upgrade you have no working DNS anymore, unless you disable systemd-resolved. I don't want to use netplan, so changing to this is no option.

The fix is easy - remove the quotes in the script on the left side "$DNS" => $DNS; same for DOMAINS:

    if [ -n "$NEW_DNS" ]; then
        cat <<EOF >"$mystatedir/ifupdown-${ADDRFAM}-$interface"
$DNS="$NEW_DNS"
EOF
        if [ -n "$NEW_DOMAINS" ]; then
            cat <<EOF >>"$mystatedir/ifupdown-${ADDRFAM}-$interface"
$DOMAINS="$NEW_DOMAINS"
EOF
        fi
    fi

Revision history for this message
Uwe Schindler (uwe-thetaphi) wrote :

One addition, line 71 in script also looks strange:

    if ! cmp --silent "$oldstate" "$newstate" 2>/dev/null; then
        DNS DNS6 DOMAINS DOMAINS6 DEFAULT_ROUTE

This causes:
/etc/network/if-up.d/resolved: 71: DNS: not found

If you are on it, please fix this, too.

Revision history for this message
Uwe Schindler (uwe-thetaphi) wrote :

I think this should unset the variables, then error goes away and it makes sense!

    if ! cmp --silent "$oldstate" "$newstate" 2>/dev/null; then
        unset DNS DNS6 DOMAINS DOMAINS6 DEFAULT_ROUTE

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ifupdown (Ubuntu):
status: New → Confirmed
Revision history for this message
Chris Nicolson (morphiusfaydal) wrote :

This is affecting me too. It's only happening on systems that started life as Ubuntu 18.04 and earlier - any server that was installed as 20.04 is upgrading to 22.04 without problem. Once I make the same change as Uwe, systemd-resolved is getting configured correctly.

Revision history for this message
Neil (broomfn) wrote :

+1, can I simply remove ifupdown (e.g. apt remove ifupdown), do I still need it with 22.04.1?

Revision history for this message
Uwe Schindler (uwe-thetaphi) wrote :

If you changed away from /etc/network/interfaces file to netplan.io configuration (/etc/netplan/...) then you won't need ifupdown. There is a configuartion tool, with simple configs it works: See this info how to convert your network config. But be sure to have physical access to the machine's console: https://gist.github.com/mss/7a8e048dd51e5ef928039f1450ba8f31

I did this for my systems and removed and purged ifupdown. For some configs it may be very complex, especially if you have many tunnels, bridges or other special stuff in /etc/network/interfaces. But for simple single ethernet/wifi device it is easy to migrate using the above GIST.

Revision history for this message
Paul Wright (paulwright76) wrote :

The same error also happened in the ifdown script for resolved. Weird.

Revision history for this message
Paul Wright (paulwright76) wrote :

Deleted my incorrect comment about the use of "local", that doesn't work. https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/1907878 has the correct fix.

Lukas Märdian (slyon)
tags: added: foundations-triage-discuss
Changed in ifupdown (Ubuntu):
status: Confirmed → Triaged
importance: Undecided → Medium
Revision history for this message
David Kowis (dkowis) wrote :

https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/1910273

This one seems to have the full patch repair in the bug description. None of the other bugs I've found relating to this have the full solution.

Revision history for this message
Uwe Schindler (uwe-thetaphi) wrote : Re: [Bug 1981103] Re: System with DNS server in /etc/network/interfaces has bogus systemd-resolved config after upgrade to 22.04

*This* bug has the full information in the first post including all needed to fix the problem. Actually there are more problems in the script regarding syntax, so more issues exist, leading to unwanted behavior.

Uwe

Am 22. Oktober 2022 22:05:59 MESZ schrieb David Kowis <email address hidden>:
>https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/1910273
>
>This one seems to have the full patch repair in the bug description.
>None of the other bugs I've found relating to this have the full
>solution.
>
>--
>You received this bug notification because you are subscribed to the bug
>report.
>https://bugs.launchpad.net/bugs/1981103
>
>Title:
> System with DNS server in /etc/network/interfaces has bogus systemd-
> resolved config after upgrade to 22.04
>
>Status in ifupdown package in Ubuntu:
> Triaged
>
>Bug description:
> Description: Ubuntu 22.04 LTS
> Release: 22.04
>
> ifupdown:
> Installed: 0.8.36+nmu1ubuntu3
> Candidate: 0.8.36+nmu1ubuntu3
> Version table:
> *** 0.8.36+nmu1ubuntu3 500
> 500 http://de.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages
> 100 /var/lib/dpkg/status
>
> After upgrading a server with classic ifupdown configuration after
> reboot the machine had no valid dns servers anymore.
>
> The problem is that the state file created by ifupdown using
> /etc/network/if-up.d/resolved looks like this:
>
> root@pangaea-pm:~# cat /run/network/ifupdown-inet-ens13
> "DNS"="134.102.20.20 134.102.200.14"
> "DOMAINS"="marum.de"
>
> The script later sources this file and causes the following errors,
> easy to see when you execute this:
>
> root@pangaea-pm:~# ifdown ens13; ifup ens13
> /etc/network/if-down.d/resolved: 12: mystatedir: not found
> /etc/network/if-up.d/resolved: 12: mystatedir: not found
> /etc/network/if-up.d/resolved: 71: DNS: not found
> /etc/network/if-up.d/resolved: 1: /run/network/ifupdown-inet-ens13: DNS=134.102.20.20 134.102.200.14: not found
> /etc/network/if-up.d/resolved: 2: /run/network/ifupdown-inet-ens13: DOMAINS=marum.de: not found
> Failed to parse DNS server address: DNS
> Failed to set DNS configuration: Invalid argument
>
> This happened to me on three different servers, so this is a serious
> bug and should be fixed before 22.04 upgrades are allowed for
> everybody. Most servers provided by data centers like Hetzner
> (Germany) are configure like that. After a do-release-upgrade you have
> no working DNS anymore, unless you disable systemd-resolved. I don't
> want to use netplan, so changing to this is no option.
>
> The fix is easy - remove the quotes in the script on the left side
> "$DNS" => $DNS; same for DOMAINS:
>
> if [ -n "$NEW_DNS" ]; then
> cat <<EOF >"$mystatedir/ifupdown-${ADDRFAM}-$interface"
> $DNS="$NEW_DNS"
> EOF
> if [ -n "$NEW_DOMAINS" ]; then
> cat <<EOF >>"$mystatedir/ifupdown-${ADDRFAM}-$interface"
> $DOMAINS="$NEW_DOMAINS"
> EOF
> fi
> fi
>
>To manage notifications about this bug go to:
>https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/1981103/+subscriptions
>

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

Revision history for this message
Lukas Märdian (slyon) wrote (last edit ):

Thank you for describing the solution in this bug report.

Could somebody come up with a patch/debdiff, so that any MOTU could test & sponsor the ifupdown fix into universe?

tags: removed: foundations-triage-discuss
Changed in ifupdown (Ubuntu):
assignee: nobody → Heitor Alves de Siqueira (halves)
status: Triaged → In Progress
Revision history for this message
Heitor Alves de Siqueira (halves) wrote :
Revision history for this message
Heitor Alves de Siqueira (halves) wrote :

I've attached a debdiff with the patches for fixing this one. I've subscribed ubuntu-sponsors to kindly request sponsorship for mantic.

Thanks!

Revision history for this message
Benjamin Drung (bdrung) wrote :

Thanks. The 0.8.41 Debian package contains this fix already (so no forwarding needed). So uploaded to mantic (included some trailing space removals).

Changed in ifupdown (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ifupdown - 0.8.36+nmu1ubuntu5

---------------
ifupdown (0.8.36+nmu1ubuntu5) mantic; urgency=medium

  [ Uwe Kleine-König ]
  * Fix if-up.d/resolved hook to properly work with nameservers and search
    domains (LP: #1981103)

 -- Heitor Alves de Siqueira <email address hidden> Wed, 06 Sep 2023 13:23:34 +0000

Changed in ifupdown (Ubuntu):
status: Fix Committed → Fix Released
description: updated
Changed in ifupdown (Ubuntu Focal):
assignee: nobody → Heitor Alves de Siqueira (halves)
Changed in ifupdown (Ubuntu Jammy):
assignee: nobody → Heitor Alves de Siqueira (halves)
Changed in ifupdown (Ubuntu Lunar):
assignee: nobody → Heitor Alves de Siqueira (halves)
Changed in ifupdown (Ubuntu Focal):
status: New → In Progress
Changed in ifupdown (Ubuntu Jammy):
status: New → In Progress
Changed in ifupdown (Ubuntu Lunar):
status: New → In Progress
Changed in ifupdown (Ubuntu Focal):
importance: Undecided → Low
Changed in ifupdown (Ubuntu Lunar):
importance: Undecided → Critical
Changed in ifupdown (Ubuntu Jammy):
importance: Undecided → Low
Changed in ifupdown (Ubuntu Lunar):
importance: Critical → Low
Revision history for this message
Heitor Alves de Siqueira (halves) wrote :

Focal isn't affected by this, so marking it as Fix Released.

Changed in ifupdown (Ubuntu Focal):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.