default DNS IP for node from unexpected interface (not always first interface or same network)

Trent,

IIRC, the DNS entry defaults to the PXE interface. If the PXE interface is part of a bond, then the default DNS entry is created against the bond, and same for the bridge. I have tested this with:

a. ens1 - non-PXE iface
b. ens2 - non-PXE iface
c. ens3 - PXE iface

1. I deployed the machine, I confirm that default DNS entry was given to (c) above.
2. I created a bond with both ens1/ens2 (bond0), and the default DNS entry was given to ens3.
3. I then create a bond between ens1/ens3 (bond0), and the default DNS entry was created to the bond0.
4. I then created a bridge against ens3 (br0), and the default DNS entry was created against br0 (which is the PXE interface).
5. I then created a bridge against ens1 (br0), and the default DNS entry was created against ens3 (which is the PXE interface).

So, from my testing it is working as expected, where the PXE interface is the interface where the default DNS entry is generated.

That said, could you provide extra information and determine whether the DNS entry, in your case, is also being created against the PXE interface and any other information you think would be relevant?

Thanks!

FWIW, a couple things worth noting:

1. I did tests both ways, in which ens3 had the largest DB id, as well as the smalls DB id, and in both situations, it would always get the default DNS entry.
2. All interfaces were set to Auto Assign to ensure they get an IP from MAAS and a DNS entry.

On the other hand, I did a couple experiments to highlight what was does, so I did this:

a. ens0 (non-PXE interface) was set to 'Auto-Assign'
b. ens3 (PXE interface) was set to 'DHCP'

The results yield as expected, which were:

1. Even though ens3 is the PXE interface, it was set to DHCP, which meant we don't really know the IP this interface has.
2. On deployment, ens0 would get the default DNS entry, because we know the IP, as it was assigned to MAAS.
3. Once the machine PXE boots of MAAS over ens3, it did so because it got an IP from MAAS' DHCP, as such, the default DNS entry was updated to point at ens3's IP address, and ens0 was rolled back to ens0.<hostname>.<domain>, which is expected behavior.

One thing you didn't try that I did try (as this was the actual customer scenario) was a bridge of a bond. Maybe that changes the behavior?

I'll try re-test again and post the results, though I just upgraded my MAAS to 2.4 - not sure if there were changes for 2.4. So I'll try reproduce my original test first and then provide more precise instructions on exactly what I did and the results.

I did a quick test, and this does seem that this changes it. It seems when the bridge on a bond is created MAAS picks another interface for the default DNS domain, even tough the underlying physical interface for the bond/bridge is indeed the PXE one....

I think I'll need to run a few more tests to be sure, as the last test was with fake ifaces, so the deployment never finished. Either way, I'm going to target this to 2.5 & 2.4.

I have a customer using 2.3 who needs to update the DNS record for their machine to be the correct (primary) interface in this scenario. Unfortunately in 2.3 it seems you cannot update the existing DNS record like you can in 2.4 (maas dnsresources read does not return any of the machine hostname records, thus you can't update them - and if you create a record alongside it - DNS returns both IPs instead of replacing the auto generated record)

Is there some other way we can work around this issue on 2.3? As the customer is on Xenial, it seems 2.4 is not available for Xenial so it is not feasible to upgrade to 2.4 for them. Thus we need a fix or workaround for 2.3.

Hi Trent,

MAAS 2.3 -> 2.4 has not changed anything with regards to DNS. So updating
the record should be completely possible. Could you please provide a list
of commands with examples outputs?

On Mon, Jul 9, 2018 at 11:59 PM, Trent Lloyd <email address hidden>
wrote:

> I have a customer using 2.3 who needs to update the DNS record for their
> machine to be the correct (primary) interface in this scenario.
> Unfortunately in 2.3 it seems you cannot update the existing DNS record
> like you can in 2.4 (maas dnsresources read does not return any of the
> machine hostname records, thus you can't update them - and if you create
> a record alongside it - DNS returns both IPs instead of replacing the
> auto generated record)
>
> Is there some other way we can work around this issue on 2.3? As the
> customer is on Xenial, it seems 2.4 is not available for Xenial so it is
> not feasible to upgrade to 2.4 for them. Thus we need a fix or
> workaround for 2.3.
>
> --
> You received this bug notification because you are subscribed to MAAS.
> https://bugs.launchpad.net/bugs/1776604
>
> Title:
> default DNS IP for node from unexpected interface (not always first
> interface or same network)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1776604/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: product=maas; milestone=2.5.0; status=Incomplete;
> importance=High; assignee=None;
> Launchpad-Bug: product=maas; productseries=2.4; milestone=2.4.1;
> status=New; importance=High; assignee=None;
> Launchpad-Bug-Tags: internal sts
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: andreserl lathiat
> Launchpad-Bug-Reporter: Trent Lloyd (lathiat)
> Launchpad-Bug-Modifier: Trent Lloyd (lathiat)
> Launchpad-Message-Rationale: Subscriber (MAAS)
> Launchpad-Message-For: andreserl
>

--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

I think the difference in behavior between a bonded interface and a bridged bond can be explained by the following section of SQL:

    ORDER BY
        node.hostname,
        is_boot DESC,
        family(staticip.ip),
        CASE
            WHEN interface.type = 'bond' AND
                parent.id = node.boot_interface_id THEN 1
            WHEN interface.type = 'physical' AND
                interface.id = node.boot_interface_id THEN 2
            WHEN interface.type = 'bond' THEN 3
            WHEN interface.type = 'physical' THEN 4
            WHEN interface.type = 'vlan' THEN 5
            WHEN interface.type = 'alias' THEN 6
            WHEN interface.type = 'unknown' THEN 7
            ELSE 8

This query was originally written before bridges were introduced in MAAS, and does not appear to consider them at all.

It looks like we might require another join here to check for the parent-of-a-parent, since we won't know if the bridge's parent-of-a-parent (the bond) is the boot interface. The query currently just checks direct parents.

@lathiat, are you able to test the patch in this merge proposal to see if it fixes the issue for you?

https://code.launchpad.net/~mpontillo/maas/+git/maas/+merge/349384

I think this is the fix, and it's passing our tests, but I feel that it's risky to change this code, so I'd prefer someone else to verify the patch, too.

@Trent, @Andres,

I did not reproduce the problem by using maas=2.3.2-6485-ge93e044-0ubuntu1~16.04.1, my test result [1] shows PXE IP 192.168.100.205 is still in the bond/bridge br-bond0, can you share your detail steps to let me know what am I missing? thanks.

[1] https://imgur.com/a/Mp4ilC1

Joshua: did you look up the DNS name? I don't see the results of that in your screenshot.
The problem is purely the DNS name, e.g. run "ping compute.maas"

@Trent,

My test result shows DNS for bond/bridge is also normal, pls refer the following output, thanks.

ubuntu@maas:~$ ping -c 1 compute.maas
PING compute.maas (192.168.100.205) 56(84) bytes of data.
64 bytes from 192-168-100-205.maas (192.168.100.205): icmp_seq=1 ttl=64 time=0.368 ms

ubuntu@maas:~$ nslookup compute.maas
Server: 192.168.100.3
Address: 192.168.100.3#53
Name: compute.maas
Address: 192.168.100.205

This is the other info for this test - https://pastebin.canonical.com/p/SRrVw84rnn/

I have successfully reproduced the problem after restarting maas server, the DNS IP changed from 192.168.100.205 to 10.12.2.3

ubuntu@maas:~$ nslookup compute
Server: 192.168.100.3
Address: 192.168.100.3#53
Name: compute.maas
Address: 10.12.2.3

I will test the fixed patch tomorrow, will update the test result soon.

Test the fix patch, it works well - https://imgur.com/a/gRumssB

Awesome! Thank you for all the testing Hua!

On Tue, Jul 31, 2018 at 8:01 PM Hua Zhang <email address hidden>
wrote:

> Test the fix patch, it works well - https://imgur.com/a/gRumssB
>
> --
> You received this bug notification because you are subscribed to MAAS.
> https://bugs.launchpad.net/bugs/1776604
>
> Title:
> default DNS IP for node from unexpected interface (not always first
> interface or same network)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1776604/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: product=maas; milestone=2.5.0alpha1; status=Fix Released;
> importance=High; <email address hidden>;
> Launchpad-Bug: product=maas; productseries=2.3; milestone=2.3.x;
> status=Fix Committed; importance=High; assignee=
> <email address hidden>;
> Launchpad-Bug: product=maas; productseries=2.4; milestone=2.4.1;
> status=Fix Committed; importance=High; assignee=
> <email address hidden>;
> Launchpad-Bug-Tags: internal sts
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: andreserl lathiat mpontillo zhhuabj
> Launchpad-Bug-Reporter: Trent Lloyd (lathiat)
> Launchpad-Bug-Modifier: Hua Zhang (zhhuabj)
> Launchpad-Message-Rationale: Subscriber (MAAS)
> Launchpad-Message-For: andreserl
>
--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

Looks like this has been released to 2.3 (i see the patch in 2.3.5-6511-gf466fdb-0ubuntu1)