Wrong metadata url in enlist cloud-config

Bug #2022926 reported by Michal Kielkowski
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
MAAS
Fix Committed
High
Anton Troyanov
3.2
Fix Released
High
Anton Troyanov
3.3
Fix Committed
High
Anton Troyanov
3.4
Fix Released
High
Anton Troyanov

Bug Description

Hello,

this is single host, rack+region scenario. Version 3.3.3. Metadata url is presented without a specific port and it's unreachable on default 80.

Steps to reproduce:

When visiting url passed on to kernel/initrd by bootloader:

http://192.168.0.234:5248/MAAS/metadata/latest/enlist-preseed/?op=get_enlist_preseed

I receive following cloud-config script

#cloud-config
apt:
  preserve_sources_list: false
  primary:
  - arches:
    - amd64
    - i386
    uri: http://archive.ubuntu.com/ubuntu
  - arches:
    - default
    uri: http://ports.ubuntu.com/ubuntu-ports
  proxy: http://192.168.0.234:8000/
  security:
  - arches:
    - amd64
    - i386
    uri: http://archive.ubuntu.com/ubuntu
  - arches:
    - default
    uri: http://ports.ubuntu.com/ubuntu-ports
  sources_list: 'deb $PRIMARY $RELEASE universe restricted main multiverse

    # deb-src $PRIMARY $RELEASE universe restricted main multiverse

    deb $PRIMARY $RELEASE-updates universe restricted main multiverse

    # deb-src $PRIMARY $RELEASE-updates universe restricted main multiverse

    deb $PRIMARY $RELEASE-backports universe restricted main multiverse

    # deb-src $PRIMARY $RELEASE-backports universe restricted main multiverse

    deb $SECURITY $RELEASE-security universe restricted main multiverse

    # deb-src $SECURITY $RELEASE-security universe restricted main multiverse

    '
datasource:
  MAAS:
    metadata_url: http://192.168.0.234/MAAS/metadata/
manage_etc_hosts: true
packages:
- python3-yaml
- python3-oauthlib
power_state:
  condition: test ! -e /tmp/block-poweroff
  delay: now
  mode: poweroff
  timeout: 1800
rsyslog:
  remotes:
    maas: 192.168.0.234:5247

Please note metadat_url not having a port specified. The enlistment process hangs during ephemeral image startup after

[ OK ] Reached target Host and Netowrka Name Lookups

and then typically times out but with no datasource configured and resulting in failed enlistment.

You can work it around by replacing default /etc/maas/preseed/enlist template:

{{preseed_data}}

with

#cloud-config
apt:
  preserve_sources_list: false
  primary:
  - arches:
    - amd64
    - i386
    uri: http://archive.ubuntu.com/ubuntu
  - arches:
    - default
    uri: http://ports.ubuntu.com/ubuntu-ports
  security:
  - arches:
    - amd64
    - i386
    uri: http://archive.ubuntu.com/ubuntu
  - arches:
    - default
    uri: http://ports.ubuntu.com/ubuntu-ports
  sources_list: 'deb $PRIMARY $RELEASE main restricted multiverse universe

    # deb-src $PRIMARY $RELEASE main restricted multiverse universe

    deb $PRIMARY $RELEASE-updates main restricted multiverse universe

    # deb-src $PRIMARY $RELEASE-updates main restricted multiverse universe

    deb $PRIMARY $RELEASE-backports main restricted multiverse universe

    # deb-src $PRIMARY $RELEASE-backports main restricted multiverse universe

    deb $SECURITY $RELEASE-security main restricted multiverse universe

    # deb-src $SECURITY $RELEASE-security main restricted multiverse universe

    '
datasource:
  MAAS:
    metadata_url: http://10.141.200.9:5248/MAAS/metadata/
manage_etc_hosts: true
packages:
- python3-yaml
- python3-oauthlib
power_state:
  condition: test ! -e /tmp/block-poweroff
  delay: now
  mode: poweroff
  timeout: 1800
rsyslog:
  remotes:
    maas: 10.141.200.9:5247

Commisioning and deployment are not affected.
With regards
Michal K.

Related branches

Revision history for this message
Anton Troyanov (troyanov) wrote :

Hello Michal!

Do you have MAAS running behind a loadbalancer or reverse proxy?

Or maybe your machine has multiple interfaces?

---
FTR
The function that templates this config starts here: src/maasserver/compose_preseed.py:661

def build_metadata_url(request, route, rack_controller, node=None, extra=""):
    host = _get_rackcontroller_host(request, node=node)
    if host is None and rack_controller is not None:
        host = rack_controller.fqdn
    return (
        request.build_absolute_uri(route) + extra
        if not host
        else f"{request.scheme}://{host}:{RACK_CONTROLLER_PORT}{route}{extra}"
    )

Revision history for this message
Michal Kielkowski (mikiel) wrote :

No loadbalancer, no reverse proxy (except for MaaS own nginx). I use external dhcp scenario.
I find this error consistently appearing accross my 2 setups. One in AWS vpc, one in my private virtual lab. Let me know if you need addtional info or traces/logs.

Net config from the lab env:

Network netplans (ubuntu server 22.04) config:
network:
  ethernets:
    eth0:
      addresses:
        - 192.168.0.234/24
      nameservers:
        addresses: [192.168.0.1]
        search: [lan]
      routes:
        - to: default
          via: 192.168.0.1
  version: 2

root@maas:/var/log/maas# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:15:5d:38:01:29 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.234/24 brd 192.168.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fd77:eeee:7bdb::64f/128 scope global dynamic noprefixroute
       valid_lft 42713sec preferred_lft 42713sec
    inet6 fd77:eeee:7bdb:0:215:5dff:fe38:129/64 scope global mngtmpaddr noprefixroute
       valid_lft forever preferred_lft forever
    inet6 fe80::215:5dff:fe38:129/64 scope link
       valid_lft forever preferred_lft forever
root@maas:/var/log/maas#

Revision history for this message
Alexsander de Souza (alexsander-souza) wrote :

We need to update the nginx configuration from

proxy_set_header Host $host;

to

proxy_set_header Host $http_host;

otherwise we lose the port information

Changed in maas:
status: New → Triaged
importance: Undecided → High
milestone: none → 3.5.0
Changed in maas:
assignee: nobody → Anton Troyanov (troyanov)
Changed in maas:
status: Triaged → Fix Committed
Revision history for this message
Yuriy Tabolin (olddanmer) wrote :

Have the same issue on 3.3.4

Workaround works for me: add 'listen 80;' to /usr/lib/python3/dist-packages/provisioningserver/templates/http/rackd.nginx.conf.template and restart maas-rackd

Revision history for this message
Anton Troyanov (troyanov) wrote :

Hi Yuri,

The fix was backported to 3.3 branch after 3.3.4 was released (thats why it is affected).

The version that should have the fix is 3.3.5 (however it is not released yet)

Revision history for this message
Alan Baghumian (alanbach) wrote :

I encountered this issue with MAAS 3.3.4 today, simply upgrading to 3.3.5 from the edge channel resolved the issues.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.