sunbeam launch failed with port binding errors

The snap explicitly sets the internal host in nova, but not in neutron.

So nova gets socket.fqdn whereas neutron defaults to socket.get_hostname

Actually that's foobar - OVN is set to socket.getfqdn as is the host configuration in nova.conf.

So they should match

Can you check what the host key is set to in:

/var/snap/openstack-hypervisor/common/etc/nova/nova.conf

please

nova.conf has the following configuration for host key
host = dev.internal.cloudapp.net

From the below log, <host>:<nodename> --> host name is populated from host key in conf (via sunbeam) and nodename (hypervisor_hostname) is populated from libvirt hostinfo (libvirt uses gethostname i guess). And there comes the discrepancy.

Jun 13 21:47:47 dev nova-compute[538724]: 2023-06-13 21:47:47.330 538724 INFO nova.compute.resource_tracker [None req-5eff8636-991c-401f-9d96-e2c8b29144f0 - - - - - -] Compute node record created for dev.internal.cloudapp.net:dev.3pmbhi1rcrau3nnvk2nd1bwztb.ax.internal.cloudapp.net with uuid: 2c41448c-ab3d-454c-a4b8-a822ad522ab5

PR for review: https://github.com/openstack-snaps/snap-openstack/pull/155

I'm running into a very similar issue.

ubuntu@crustle:~$ source demo-openrc
ubuntu@crustle:~$ python3 -c "import socket; print(socket.getfqdn())"
crustle.segmaas.1ss
ubuntu@crustle:~$ openstack hypervisor list
HttpException: 403: Client Error for url: http://10.20.21.11/openstack-nova/v2.1/os-hypervisors/detail, Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed.
ubuntu@crustle:~$ hostname -f
crustle.segmaas.1ss
ubuntu@crustle:~$ sunbeam cluster list
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┓
┃ Node ┃ Status ┃ Control ┃ Compute ┃ Storage ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━┩
│ crustle.segmaas.1ss │ up │ x │ x │ │
└─────────────────────┴────────┴─────────┴─────────┴─────────┘
ubuntu@crustle:~$ sudo snap get openstack-hypervisor node
Key Value
node.fqdn crustle.segmaas.1ss
node.ip-address 10.230.57.128
ubuntu@crustle:~$

Attaching full nova compute logs from journalctl. nova logs at https://pastebin.canonical.com/p/3WvYC3dX9T/ , some logs from the charm container at https://pastebin.canonical.com/p/wBtCTxFJ5k/ and neutron-server logs at https://pastebin.canonical.com/p/6PnfTKPG5P/

https://pastebin.canonical.com/p/xdk4knBGj2/ has the relevant logs in journalctl when the vm creation was tried using a command like,

openstack server create --flavor m1.small --image "ubuntu" testvm-nikhil

Or even this approach.

ubuntu@crustle:~$ sunbeam cluster list
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┓
┃ Node ┃ Status ┃ Control ┃ Compute ┃ Storage ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━┩
│ crustle.segmaas.1ss │ up │ x │ x │ │
└─────────────────────┴────────┴─────────┴─────────┴─────────┘
ubuntu@crustle:~$ sunbeam launch ubuntu --name test
Launching an OpenStack instance ...
⠦ Creating the OpenStack instance ... Instance creation request failed: Server:4dee2c40-13a6-4b21-8ffc-867d4ed02a77 transitioned to failure state ERROR
Error: Unable to request new instance. Please run `sunbeam configure` first.
ubuntu@crustle:~$ sunbeam openrc > admin-openrc
ubuntu@crustle:~$ sunbeam configure --accept-defaults --openrc admin-openrc
Writing openrc to admin-openrc ... done
ubuntu@crustle:~$ sunbeam launch ubuntu --name test
Launching an OpenStack instance ...
Found sunbeam key in OpenStack!
⠸ Creating the OpenStack instance ... Instance creation request failed: Server:c0f5a495-1af3-40b4-b583-dbdc27fc5393 transitioned to failure state ERROR
Error: Unable to request new instance. Please run `sunbeam configure` first.
ubuntu@crustle:~$ sunbeam configure
Local or remote access to VMs [local/remote] (local):
CIDR of OpenStack external network - arbitrary but must not be in use (10.20.20.0/24):
Populate OpenStack cloud with demo user, default images, flavors etc [y/n] (y):
Username to use for access to OpenStack (demo):
Password to use for access to OpenStack (T3********):
Network range to use for project network (192.168.122.0/24):
List of nameservers guests should use for DNS resolution (10.230.56.2):
Enable ping and SSH access to instances? [y/n]...