Following upgrade to 6.5.0-7 kernel in mantic cloud images we are seeing a regression in our cloud image tests. The test runs the following:
```
lxd init --auto --storage-backend dir
lxc launch ubuntu-daily:mantic mantic
lxc info mantic
lxc exec mantic -- cloud-init status --wait
```
The `lxc exec mantic -- cloud-init status --wait` times out after 240s and will fail our test as a result.
I have been able to replicate in a local VM
```
wget http://cloud-images.ubuntu.com/mantic/20231005/mantic-server-cloudimg-amd64.img
wget --output-document=launch-qcow2-image-qemu.sh https://gist.githubusercontent.com/philroche/14c241c086a5730481e24178b654268f/raw/7af95cd4dfc8e1d0600e6118803d2c866765714e/gistfile1.txt
chmod +x launch-qcow2-image-qemu.sh
./launch-qcow2-image-qemu.sh --password passw0rd --image ./mantic-server-cloudimg-amd64.img
cat <<EOF > "./reproducer.sh"
#!/bin/bash -eux
lxd init --auto --storage-backend dir
lxc launch ubuntu-daily:mantic mantic
lxc info mantic
lxc exec mantic -- cloud-init status --wait
EOF
chmod +x ./reproducer.sh
sshpass -p passw0rd scp -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -P 2222 ./reproducer.sh ubuntu@127.0.0.1:~/
sshpass -p passw0rd ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -p 2222 ubuntu@127.0.0.1 sudo apt-get update
sshpass -p passw0rd ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -p 2222 ubuntu@127.0.0.1 sudo apt-get upgrade --assume-yes
sshpass -p passw0rd ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -p 2222 ubuntu@127.0.0.1 ./reproducer.sh
```
The issue is not present with the 6.5.0-5 kernel and the issue is present regardless of the container launched. I tried the jammy container to test this.
From my test VM
```
ubuntu@cloudimg:~$ uname --all
Linux cloudimg 6.5.0-7-generic #7-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 29 09:14:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@cloudimg:~$ uname --kernel-release
6.5.0-7-generic
```
This is a regression in our test that will block 23.10 cloud image release next week.
This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:
apport-collect 2038567
and then change the status of the bug to 'Confirmed'.
If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.
This change has been made by an automated script, maintained by the Ubuntu Kernel Team.