dnsmasq started too early, not getting all interfaces

In your netplan example config, keyword 'optional' means that systemd-networkd won't wait until this interface is up before declaring to systemd that the network is online. THis means dnsmasq is starting before the vlan0 interface is created.

If you remove the optional value that should ensure it is up and configured before the dnsmasq service starts.

So it is not possible to have an interface as optional (for other reasons beyond that example) and still have a dnsmasq running for it?

In former versions of ubuntu I had my bridges (some containing usb ethernet interfaces to bridge virtual with physical machines) configured in /etc/network/if-up.d, and started separate dnsmasq instances for every single one of these bridges.

Now there is no regular way to start a daemon like dnsmasq through ifup-scripts, since netplan does not support them. Some people try to workaround that problem with udev clauses, starting systemd services, but then it's overcomplicated and error prone.

So what is the offical way to have services like dnsmasq run for optional interfaces under 18.04?

"Don't use optional" is a workaround, not a solution to the problem. If there is no ifup-script anymore, then how are services to be started, once the interface is up (late or just sometimes)?

https://netplan.io/faq#example-for-an-ifupdown-legacy-hook-for-post-uppost-down-states

You can use a networkd-dispatcherd to hook on vlan0 becoming routeable
and then start/restart the dnsmasq service;

On Fri, Jun 15, 2018 at 12:36 PM, Hadmut Danisch <email address hidden> wrote:
> So it is not possible to have an interface as optional (for other
> reasons beyond that example) and still have a dnsmasq running for it?
>
> In former versions of ubuntu I had my bridges (some containing usb
> ethernet interfaces to bridge virtual with physical machines) configured
> in /etc/network/if-up.d, and started separate dnsmasq instances for
> every single one of these bridges.
>
> Now there is no regular way to start a daemon like dnsmasq through ifup-
> scripts, since netplan does not support them. Some people try to
> workaround that problem with udev clauses, starting systemd services,
> but then it's overcomplicated and error prone.
>
> So what is the offical way to have services like dnsmasq run for
> optional interfaces under 18.04?
>
>
> "Don't use optional" is a workaround, not a solution to the problem. If there is no ifup-script anymore, then how are services to be started, once the interface is up (late or just sometimes)?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1777094
>
> Title:
> dnsmasq started too early, not getting all interfaces
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/dnsmasq/+bug/1777094/+subscriptions

setting optional: false does not solve the problem, a systemctl restart dnsmasq is still required.

I see, interesting. The dnsmasq.service does not wait for
network-online.target;

This appears to be a duplicate of:

https://bugs.launchpad.net/ubuntu/+source/dnsmasq/+bug/1531184

In addition to the netplan config to ensure it waits for your bridge
to come up, you need to delay dnsmasq.service until after the network
is online. You can do that with something like this:

% mkdir -p /etc/systemd/system/dnsmasq.service.d
% echo -e "[Unit]\nAfter=network-online.target\nWants=network-online.target"
| sudo tee /etc/systemd/system/dnsmasq.service.d/network-online.conf
% reboot

After rebooting, you should see that networkd runs, brings up the
interfaces, then network-online.target is reached, and lastly dnsmasq
is then started.

% journalctl -b -o short-monotonic | egrep "(Reboot|networkd|Network
is Online|Starting dnsmasq)"
[1835835.825873] b1 systemd-networkd[173]: vlan0: IPv6 successfully enabled
[1835835.827054] b1 systemd-networkd[173]: vlan0: netdev ready
[1835835.837274] b1 systemd-networkd[173]: Enumeration completed
[1835835.864747] b1 systemd-networkd[173]: eth1: Lost carrier
[1835835.871748] b1 systemd-networkd[173]: eth0: DHCPv4 address
10.8.107.77/24 via 10.8.107.1
[1835835.879184] b1 systemd-networkd[173]: eth1: IPv6 successfully disabled
[1835835.908530] b1 systemd-networkd[173]: eth1: Gained carrier
[1835835.908903] b1 systemd-networkd[173]: eth1: Configured
[1835835.909878] b1 systemd-networkd[173]: vlan0: Gained carrier
[1835835.913807] b1 systemd-networkd-wait-online[175]: managing: eth1
[1835836.288648] b1 systemd-networkd[173]: eth0: Gained IPv6LL
[1835836.289022] b1 systemd-networkd[173]: eth0: Configured
[1835836.290405] b1 systemd-networkd-wait-online[175]: managing: eth1
[1835836.290834] b1 systemd-networkd-wait-online[175]: managing: eth0
[1835836.291077] b1 systemd-networkd-wait-online[175]: ignoring: lo
[1835837.760738] b1 systemd-networkd[173]: vlan0: Gained IPv6LL
[1835837.761713] b1 systemd-networkd[173]: vlan0: Configured
[1835837.762522] b1 systemd-networkd-wait-online[175]: managing: eth1
[1835837.763413] b1 systemd-networkd-wait-online[175]: managing: eth0
[1835837.763705] b1 systemd-networkd-wait-online[175]: ignoring: lo
[1835837.764057] b1 systemd-networkd-wait-online[175]: managing: vlan0
[1835838.445945] b1 systemd[1]: Reached target Network is Online.
[1835838.496394] b1 dbus-daemon[195]: [system] Activating via systemd:
service name='org.freedesktop.hostname1'
unit='dbus-org.freedesktop.hostname1.service' requested by ':1.0'
(uid=100 pid=173 comm="/lib/systemd/systemd-networkd "
label="unconfined")
[1835838.516683] b1 systemd[1]: Starting dnsmasq - A lightweight DHCP
and caching DNS server...
On Tue, Jun 19, 2018 at 3:51 AM Hadmut Danisch <email address hidden> wrote:
>
> setting optional: false does not solve the problem, a systemctl
> restart dnsmasq is still required.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1777094
>
> Title:
> dnsmasq started too early, not getting all interfaces
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/dnsmasq/+bug/1777094/+subscriptions

dnsmasq _after_ networking?

Isn't it part of networking and should be running and ready for other services depending on networking?

(my guess is that Ubuntu has never defined the targets very precisely. it should distinguish between low level /interface level configuration and networking services.)

On Tue, Jun 19, 2018 at 9:41 AM Hadmut Danisch <email address hidden> wrote:
>
> dnsmasq _after_ networking?
>
> Isn't it part of networking and should be running and ready for other
> services depending on networking?

Yes, but it's complicated; see below.

>
>
> (my guess is that Ubuntu has never defined the targets very precisely. it should distinguish between low level /interface level configuration and networking services.)
>

This is rather precisely defined; not by Ubuntu but by the init
system. In systemd there are many targets that indicate various state
of the system during boot[1][2]
For networking, the ones that matter are:

network-pre.target (before systemd has attempted to bring the network up)
network.target (systemd has started to bring network up; for
Artful/Bionic, this means systemd-networkd.service has started)
network-online.target ("required" networking interfaces are up,
configured, routable)

Dnsmasq itself is a networking service, so arguably, it is fine to run
at After=network.target; but network.target does not guarantee that
all of networking has been configured. Dnsmasq has two choices. 1)
it can run later, at network-online.target which means systemd is
done with creating and configuring interfaces. 2) start at
After=network.target but listen to dbus or netlink layer to react to
when a
configured interface is present and ready for it to use.

Since dnsmasq does not watch dbus or netlink for new interfaces and
when they're configured, then the only solution is to run after such
a time that the configured interfaces are ready for use by dnsmasq.

1. https://www.freedesktop.org/software/systemd/man/systemd.special.html
2. https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1777094
>
> Title:
> dnsmasq started too early, not getting all interfaces
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/dnsmasq/+bug/1777094/+subscriptions

[Expired for netplan.io (Ubuntu) because there has been no activity for 60 days.]

[Expired for dnsmasq (Ubuntu) because there has been no activity for 60 days.]

The problem lives in the [Unit] section in the After= directive which when modified as follows raises dnsmasq.service unbroken on boot:

sed -i 's/After=network.target/After=NetworkManager-wait-online.service/' /etc/systemd/system/multi-user.target.wants/dnsmasq.service

That's a fairly tidy, and *reproducible* fix which works on successive reboots.

In respect to my above post, the correct target of the sed expression should be as follows:

sed -i 's/After=network.target/After=NetworkManager-wait-online.service/' /lib/systemd/system/dnsmasq.service

Please note the fix remains nonetheless valid and ensures that dnsmasq rises up correctly on boot-

Hi Terrence, yeah and while yours is still NetworkManager specific it is still essentially the same as bug 1531184 - making this a dup to have all on one.