Restarting juju machine unit process causes fan-network lxcs to lose IPs

Bug #1903520 reported by Pedro Guimarães
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Triaged
High
Unassigned

Bug Description

Hi,

Using Juju 2.8.5. After upgrading to 2.8.6 (both controller and models), I've realized all my LXCs lost their IPs. Those are fan-networking LXCs.

Looking into this issue, I've realized that fan-networking addresses are set using a dhcp server.
I've realized that, after upgrading; that dhcp server will disappear and containers will fail with lost leases:

Nov 6 15:43:45 juju-cf278f-0-lxd-2 systemd-networkd[190]: eth0: DHCP lease lost

I've noticed that also happens after restarting machine's service.
The reason is that, although dnsmasq-dhcp process PPID is 1, on systemd, it shows as subordinate to jujud-machine:

             ├─jujud-machine-0.service
             │ ├─10808 bash /etc/systemd/system/jujud-machine-0-exec-start.sh
             │ ├─10819 /var/lib/juju/tools/machine-0/jujud machine --data-dir /var/lib/juju --machine-id 0 --debug
             │ └─11573 dnsmasq -u dnsmasq --strict-order --bind-interfaces --pid-file=/run/ubuntu-fan/dnsmasq-fan-252.pid --conf-file= --listen-address 252.0.53.65 --dhcp-range 252.0.53.66,252.0.53.78 -- ......

I can see on juju logs that it does run "fanctl up -a" after service restart:
/var/log/juju/machine-0.log:2020-11-09 10:05:42 DEBUG juju.worker.fanconfigurer fanconfigurer.go:69 Launched fanctl up -a - result /usr/sbin/fanctl: /etc/network/fan:40 10.192.0.0/12 252.0.0.0/8 -- failed err=1

The only way I can see now to bring the dhcp up is to manually run the same dhcp command.

Revision history for this message
Adam Dyess (addyess) wrote :

Experienced the same when upgrading from juju 2.6.10 -> 2.7.8 -> 2.8.7

after running `fanctl up -a` we saw the dnsmasq service start

to restore lxd containers we had to do this:

lxc stop <all-containers>
fanctl down -a
fanctl up -a
lxc start <all-containers>

Revision history for this message
James Troup (elmo) wrote :

I've unsubscribed ~field-critical and subscribed ~field-high as there is a workaround.

Changed in juju:
status: New → Triaged
importance: Undecided → High
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.