[SRU] dnsmasq doesn't start on boot because its interface isn't up yet

Bug #1531184 reported by Simon Déziel
162
This bug affects 33 people
Affects Status Importance Assigned to Milestone
One Hundred Papercuts
Confirmed
High
Unassigned
dnsmasq (Debian)
New
Unknown
dnsmasq (Ubuntu)
Confirmed
Low
Unassigned
Xenial
Won't Fix
Low
Unassigned
Bionic
Confirmed
Low
Unassigned

Bug Description

[Impact]
dnsmasq will fail to respond on network devices that weren't up when its
service started, thus not binding as expected.

[Test Case]
TBD

[Regression Potential]
The fix is just configuring the order of service startup, so is unlikely to create any regressions. Things to watch would be service related misbehaviors and general availability of the dnsmasq functionality.

[Fix]
Straightforward packaging fix to the service to make it delay startup
until after the network is online.

https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=774970;filename=774970-network-online.debdiff;msg=22

[Discussion]

[Original Report]
My dnsmasq instance uses "interface=br-vz0" and the interface br-vz0 is managed manually in /etc/network/interfaces.

During boot, dnsmasq is started before br-vz0 is created and this causes dnsmasq to exit:

Jan 5 08:56:16 simon-laptop dnsmasq[1008]: dnsmasq: unknown interface br-vz0
Jan 5 08:56:16 simon-laptop dnsmasq[1008]: unknown interface br-vz0
Jan 5 08:56:16 simon-laptop dnsmasq[1008]: FAILED to start up
Jan 5 08:56:17 simon-laptop NetworkManager[937]: <info> NetworkManager (version 1.0.4) is starting...
...
Jan 5 08:56:18 simon-laptop NetworkManager[937]: <info> interface-parser: parsing file /etc/network/interfaces
...
Jan 5 08:56:18 simon-laptop NetworkManager[937]: <info> found bridge ports none for br-vz0
Jan 5 08:56:18 simon-laptop NetworkManager[937]: <info> adding bridge port none to eni_ifaces
Jan 5 08:56:18 simon-laptop NetworkManager[937]: <info> management mode: unmanaged

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: dnsmasq 2.75-1
ProcVersionSignature: Ubuntu 4.3.0-5.16-generic 4.3.3
Uname: Linux 4.3.0-5-generic x86_64
ApportVersion: 2.19.3-0ubuntu2
Architecture: amd64
CurrentDesktop: Unity
Date: Tue Jan 5 09:53:30 2016
PackageArchitecture: all
SourcePackage: dnsmasq
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
Simon Déziel (sdeziel) wrote :
Revision history for this message
Simon Déziel (sdeziel) wrote :

Adding the following to the [Unit] section of dnsmasq.service fixes the problem:

 After=network-online.target
 Wants=network-online.target

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in dnsmasq (Ubuntu):
status: New → Confirmed
Revision history for this message
Immatix (immatix) wrote :

I believe I was having the same problem, I use the --bind-interfaces option as well as a few --interface options for different interfaces, and dnsmasq would regularly fail to start during boot because one of the interfaces wasn't present yet. Logs show dnsmasq was being started before network.target was reached.

Apr 30 22:23:19 sirius systemd[1]: Starting dnsmasq - A lightweight DHCP and caching DNS server...
Apr 30 22:23:20 sirius dnsmasq[1152]: dnsmasq: syntax check OK.
Apr 30 22:23:20 sirius dnsmasq[1296]: dnsmasq: unknown interface trunk.3
Apr 30 22:23:20 sirius systemd[1]: dnsmasq.service: Control process exited, code=exited status=2
Apr 30 22:23:20 sirius systemd[1]: Failed to start dnsmasq - A lightweight DHCP and caching DNS server.
Apr 30 22:23:20 sirius systemd[1]: dnsmasq.service: Unit entered failed state.
Apr 30 22:23:20 sirius systemd[1]: dnsmasq.service: Failed with result 'exit-code'.
---
Apr 30 22:23:27 sirius systemd[1]: Reached target Network.

I've since added

 After=network-online.target
 Wants=network-online.target

to the [Unit] section of /lib/systemd/system/dnsmasq.service and restarted once and the problem didn't occur, but this hasn't been extensively tested.

Changed in dnsmasq (Ubuntu):
importance: Undecided → High
Changed in hundredpapercuts:
status: New → Confirmed
importance: Undecided → High
Revision history for this message
Max Chen (maxchen) wrote :

yeah, Immatix! Thanks so much
added

 After=network-online.target
 Wants=network-online.target

to the [Unit] section of /lib/systemd/system/dnsmasq.service

waiting for the fixed to be release

Revision history for this message
Robie Basak (racb) wrote :

04:47 <rbasak> stgraber: do you have an opinion on the proposed fix for bug 1531184 please?

04:49 <stgraber> rbasak: it's probably fine. Note that anyone who actually has the dnsmasq package installed on their system is likely to run into other problems, so not a big priority for us.

04:49 <stgraber> rbasak: (dnsmasq-base is what we care about in the distro and that doesn't ship the init script)

Changed in dnsmasq (Ubuntu):
importance: High → Low
tags: added: bitesize
Revision history for this message
Sarraceno (jfhtomas) wrote :

I have openvswitch-switch, which did required from me to apply some changes to systemd service file that seems not to be aware of virtual devices up.

So, bottom line, /etc/network/interface files are taking action in a way that smashes the rush to boot up, so dnsmasq also become a victim, and for me the solution above was not compatible.

So I had to add to dnsmasq.service the follow lines:

     Requires=sys-subsystem-net-devices-<needed device>.device
     After=sys-subsystem-net-devices-<needed device>.device
     Wants=sys-subsystem-net-devices-<needed device>.device

Got the above info from

     systemctl --full | grep subsystem

Best regards!

Revision history for this message
John Radley (jradxl) wrote :

I have found I needed to add
    After=network-online.target
to dnsmasq.service
to ensure my ethernet port, ens18, is available, on boot.
Using Ubuntu 16.04.2
In my case, I am using dnsmasq as per the PiHole installation where the IP is static and dhcpclient is turned off. But this is clearly not a PiHole problem as it just installs Dnsmasq.
Rather surprised I need to add this report, and that this Bug report isn't a higher Importance and not resolved in 3 months.
Surely loads of users will have this issue?
Kind Regards

Revision history for this message
Robie Basak (racb) wrote :

> Rather surprised I need to add this report, and that this Bug report isn't a higher Importance and not resolved in 3 months.
Surely loads of users will have this issue?

dnsmasq (as opposed to dnsmasq-base) is not in main in Ubuntu, so relies entirely on community support. It is not installed by default on Ubuntu, and is an "unusual end-user configuration" as far as I can tell. And a workaround appears available and is trivial.

If somebody wants to drive getting a fix landed, please do.

David Britton (dpb)
Changed in dnsmasq (Ubuntu):
assignee: nobody → David Britton (davidpbritton)
assignee: David Britton (davidpbritton) → nobody
Revision history for this message
David Britton (dpb) wrote :
Changed in dnsmasq (Ubuntu):
assignee: nobody → David Britton (davidpbritton)
David Britton (dpb)
Changed in dnsmasq (Ubuntu):
status: Confirmed → In Progress
Changed in hundredpapercuts:
status: Confirmed → In Progress
assignee: nobody → David Britton (davidpbritton)
Revision history for this message
David Britton (dpb) wrote :

Submitted patch upstream, will revisit in a couple weeks before taking on an extra deviation in Ubuntu (since currently we do not have any delta).

Changed in hundredpapercuts:
assignee: David Britton (davidpbritton) → nobody
Changed in dnsmasq (Ubuntu):
assignee: David Britton (davidpbritton) → nobody
Changed in dnsmasq (Debian):
status: Unknown → New
David Britton (dpb)
Changed in dnsmasq (Ubuntu):
status: In Progress → Confirmed
Changed in hundredpapercuts:
status: In Progress → Confirmed
Revision history for this message
Alecz20 (alexguzu) wrote :

I also see this issue after installing pihole which uses dnsmasq.

As a solution until we have a fix available, would it not be better to run `sudo systemctl edit dnsmasq` and add the three lines:
[Unit]
After=network-online.target
Wants=network-online.target

rather than edit /lib/systemd/system/dnsmasq.service directly?

Revision history for this message
Bean Kylla (beankylla) wrote :

The solution proposed by Alecz20 (alexguzu) works for me :)

Bryce Harrington (bryce)
summary: - dnsmasq doesn't start on boot because its interface isn't up yet
+ [SRU] dnsmasq doesn't start on boot because its interface isn't up yet
Bryce Harrington (bryce)
description: updated
Bryce Harrington (bryce)
description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Added bug 1777094 as a dup which is about the same issue (and essentially the same suggested solution).

Also related on askubuntu:
https://askubuntu.com/questions/536787/dnsmasq-starting-before-eth0-is-ready-and-therefore-not-binding

Revision history for this message
Terrence Houlahan (f1linux) wrote :

The reference paelzer references is related to Ubuntu 14.04. In Ubuntu 18.04, "/etc/netplan/01-network-manager-all.yaml" defaults all network device management to NetworkManager. Therefore any solution will have to be to NetworkManager's systemd service file.

My solution which I've tested and consistently raises dnsmasq up only after the interfaces are up in 18.04 is as follows:

sed -i 's/After=network.target/After=NetworkManager-wait-online.service/' /lib/systemd/system/dnsmasq.service

The 18.04 askubuntu related link is:
https://askubuntu.com/questions/1124530/systemd-starts-dnsmasq-before-starting-network-manager/1154727#1154727

Bryce Harrington (bryce)
description: updated
Revision history for this message
Benjamin Bach (benjaoming) wrote :

The fix by Terrence Houlahan works for me on 18.04. Thanks!

However, I'm wondering if adding this "After" clause is a problem since the package doesn't currently depend on Network Manager? Does "NetworkManager-wait-online.service" satisfy a more generic SystemD target?

From the dnsmasq package:

Depends: netbase, dnsmasq-base,
         init-system-helpers (>= 1.18~), lsb-base (>= 3.0-6)

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@Robie - I subscribed you as I wondered if this is another one for your list of bugs related to "binding to IPs available late".

Revision history for this message
Robie Basak (racb) wrote :

Thanks Christian. This bug was already on my list.

More details here: https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

It is not obvious to me that adding After/Wants=network-online.target to the packaging for _all_ users, or anything similar, is the correct fix here. Will this delay boot for laptop users who are offline, for example?

Revision history for this message
Benjamin Bach (benjaoming) wrote :

Adding to my previous message: After=NetworkManager-wait-online.service will not work, as it doesn't respect for instance setups where networkd handles connections.

@Robie Basak

> Will this delay boot for laptop users who are offline, for example?

If someone has installed the dnsmasq package with the dnsmasq.service file, they already intend to run this as a network-dependent service? It's not part of a normal user's setup - I think they have dnsmasq-base, which doesn't include the service.

dnsmasq-base:

> This package contains the dnsmasq executable and documentation, but not the infrastructure required to run it as a system daemon. For that, install the dnsmasq package.

Revision history for this message
Robie Basak (racb) wrote : Re: [Bug 1531184] Re: [SRU] dnsmasq doesn't start on boot because its interface isn't up yet

On Tue, Sep 24, 2019 at 04:57:59PM -0000, Benjamin Bach wrote:
> @Robie Basak
>
> > Will this delay boot for laptop users who are offline, for example?
>
> If someone has installed the dnsmasq package with the dnsmasq.service
> file, they already intend to run this as a network-dependent service?
> It's not part of a normal user's setup - I think they have dnsmasq-base,
> which doesn't include the service.

They already intend to run this as a service, yes. But why do you
qualify that as "network-dependent"? dnsmasq is still useful on a laptop
only occasionally connected to the Internet, is it not? What if a user
wants it inward-facing, for example for VMs?

Revision history for this message
Zhang Youfu (zhangyoufu) wrote :

For my installation, I found that dnsmasq is running with -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new arguments.

/etc/dnsmasq.d/libvirt-daemon specified bind-interfaces, causing dnsmasq trying to bind to each interface on startup, which was unfortunately not available yet.

Comment out the CONFIG_DIR= directive in /etc/default/dnsmasq works for me.

Revision history for this message
Dave Musicant (musicant) wrote :

Terrence Houlahan's fix also works for me on 18.04 Many thanks!

Revision history for this message
Lei Zhao (leeijaw) wrote :

For my case, I simply cannot update the systemd to solve this problem since the address I want to bind belongs to virtual tunnel device which only gets added when there is a new tunnel connection. Oops!

Revision history for this message
Lucas Kanashiro (lucaskanashiro) wrote :

The Debian maintainer suggested to also add a WantedBy= entry in the Install section:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774970#32

tags: added: server-next
Changed in dnsmasq (Ubuntu Xenial):
importance: Undecided → Low
Changed in dnsmasq (Ubuntu Bionic):
importance: Undecided → Low
Changed in dnsmasq (Ubuntu Xenial):
status: New → Confirmed
Changed in dnsmasq (Ubuntu Bionic):
status: New → Confirmed
tags: removed: server-next
Robie Basak (racb)
tags: added: network-online-ordering
Revision history for this message
Sergio Durigan Junior (sergiodj) wrote :

Xenial has reached end of standard support.

Changed in dnsmasq (Ubuntu Xenial):
status: Confirmed → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.