lxd container gets no ipv4 address

Bug #1831892 reported by Erik Lönroth
30
This bug affects 6 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Triaged
High
Unassigned

Bug Description

Context:
* I have a MAAS with a number of kvm instances.
* Nodes get ipv6 addresses and ipv4 addresses.
* Nodes (charms) can be deployed with juju. So there is no general problem with deployment.

Bug scenario:
* I'm deploying openstack with juju: https://jujucharms.com/openstack-lxd/bundle/0
* All instances with LXD are created, but they get ONLY ipv6 (where they should get both ipv4 + ipv6)
* Deployment of charms comes to a stop (because it cant communicate with lxc instances over ipv6?)

Fix/worksround:
* Restarting the dhclient on all the lxc instances makes the deployment of the charms continue, they get ipv4 addresses and deployment continues. (See image attached for description)

Expectation:
I would expect that the lxc instance would get an ipv4 address, OR, that juju would somehow use the ipv6 address and not block when deploying charms.

======
$ juju --version
2.6.3-disco-amd64

$ juju show-controller
    agent-version: 2.6.2
    mongo-version: 3.6.3

$ juju lxd container series = bionic

Revision history for this message
Erik Lönroth (sssler-scania) wrote :
description: updated
description: updated
Revision history for this message
Richard Harding (rharding) wrote :

Unfortunately there's some holes in Juju and ipv6 support. It's cases like this we might be able to find a small fix for in the progress of other work but really should be a solid look from top to bottom making sure ipv6 works as intended throughout.

Changed in juju:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
George Kraft (cynerva) wrote :

I'm encountering this as well. In my case, it seems to be caused by a malformed ipv6 nameserver address that ends up in /etc/netplan/99-juju.yaml in the container. The address appears as '<address>%5'.

Output from /var/log/cloud-init-output.log with address changed to protect the innocent:

No /sbin/ifup, applying netplan configuration.
/etc/netplan/99-juju.yaml:12:20: Error in network definition: malformed address 'fe80::abcd:abcd:abcd:abcd%5', must be X.X.X.X or X:X:X:X:X:X:X:X
        addresses: [10.123.123.123, 'fe80::abcd:abcd:abcd:abcd%5']

Pen Gale (pengale)
Changed in juju:
importance: Medium → High
Revision history for this message
SysXpert (sysxpert) wrote :

Seeing the same issue in cloud-init-output.log
/etc/netplan/99-juju.yaml:12:20: Error in network definition: malformed address 'fe80::36:88ff:febd:a773%3', must be X.X.X.X or X:X:X:X:X:X:X:X
        addresses: ['fe80::36:88ff:febd:a773%3', 192.168.1.23, 192.168.1.20]

Is there a workardound ? Do we know where it gets the ipv6 address and how we can disable it ?
=====
juju --version
2.7.6-focal-amd64

juju show-controller
xxxxxx:
  details:
    uuid: xxxxxxxx
    controller-uuid: xxxxxxxxxx
    api-endpoints: ['192.168.1.34:17070', '192.168.1.32:17070', '192.168.1.33:17070']
    cloud: maas
    region: default
    agent-version: 2.8.7
    agent-git-commit: ee2cfeb2c8c716af763a011e184ddea879c0985d
    controller-model-version: 2.8.7
    mongo-version: 3.6.8
=======

Revision history for this message
Pen Gale (pengale) wrote :

Hi, @sysxpert. Thank you for the follow-up.

If you create an lxd node on one of the kvm machines directly, rather than by using Juju, does it come up with ipv4 addresses?

It's unclear whether the underlying issue here is in Juju or in LXD.

Revision history for this message
John A Meinel (jameinel) wrote :

I don't think the dhclient should be used, as the IP addresess for LXD containers in a MAAS environment should be coming from MAAS and not from the local system.

Note that "fe80::abcd:abcd:abcd:abcd%5" is not a valid address to use, as fe80 is a link-local only address. (Eg you can't route to an fe80 address from outside of a direct connection.)

It is unclear from the description what the MAAS configuration is, what spaces are configured and what bindings you're using.

Revision history for this message
Arif Ali (arif-ali) wrote :

juju version: 2.8.8
MAAS version: 2.9.2
OS: bionic

I've got a similar issue in my lab environment, and it seems to be related to link-local address on the hypervisor.

When I disable [2] link-local on the corresponding machine where the LXD containers are running, and then do a "juju add-unit lxd:0", the lxd unit comes up.

If I don't disble link-local, then I have to run these [1] commands from my machine to fix all the LXD containers to get them working again.

When I create the lxd container using "lxc launch ubuntu:bionic", it will start start up without any problems, as it will be using DHCP by default via cloud-init.

I'll probably continue investigating, and last resort may be re-bootstrapping my MAAS/juju environment, to see if I can fix this somehow.

[1] https://paste.ubuntu.com/p/NG754kBg38/
[2] https://paste.ubuntu.com/p/tB35m3BwYp/

Revision history for this message
Arif Ali (arif-ali) wrote :

So my workaround to this particular issue is shown below

* Create a file called juju-model-default.yaml, with something similar to [1], this just sets "link-local: []" on all the physical interfaces on the machine that will host the LXD containers. You may need to adapt this for your environment

* Create a model, that you need to deploy your machines and hence the lxd containers

* apply the command below

    juju model-config juju-model-default.yam

* Now deploy your units as normal, and everything should work

This was all possible with fixes from LP#1701429 and LP#1919144

[1] https://paste.ubuntu.com/p/XdgcZzqB75/

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.