autopkgtest ignores --architecture flag

A login into the image is possible via U-Boot:

qemu-system-arm -machine virt -cpu cortex-a15 -m 1G -smp cores=2 -bios /usr/lib/u-boot/qemu_arm/u-boot.bin -nographic -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0,id=rng-device0 -device virtio-net-pci,netdev=eth0 -netdev user,id=eth0,hostfwd=tcp::8022-:22 -drive file=autopkgtest-mantic-armhf.img,if=virtio,format=qcow2

In the U-Boot console use

virtio scan
load virtio 0:1 $kernel_addr_r boot/vmlinuz
load virtio 0:1 $ramdisk_addr_r boot/initrd.img
setenv bootargs root=/dev/vda1
bootz $kernel_addr_r $ramdisk_addr_r:$filesize $fdtcontroladdr

grub.cfg has this line:

linux /boot/vmlinuz-6.5.0-5-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0

But the last command indicates that the serial console is ttyAMA0:

ubuntu@autopkgtest:~$ last
ubuntu ttyAMA0 Wed Sep 20 13:50 still logged in
reboot system boot 6.5.0-5-generic Wed Sep 20 13:43 still running

File /etc/default/grub.d/50-cloudimg-settings.cfg introduces the console=ttyS0 commandline argument.

On arm64 the command 'autopkgtest -U --apt-pocket=proposed -a armhf -s *.dsc -- qemu autopkgtest-mantic-armhf.img' invokes QEMU with:

qemu-system-aarch64 -machine virt -cpu host -m 1024 -smp 1 -nographic -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:10022-:22 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0,id=rng-device0 -monitor unix:/tmp/autopkgtest-qemu.vpg40tc_/monitor,server=on,wait=off -virtfs local,id=autopkgtest,path=/tmp/autopkgtest-qemu.vpg40tc_/shared,security_model=none,mount_tag=autopkgtest -device virtio-serial -chardev socket,path=/tmp/autopkgtest-qemu.vpg40tc_/hvc0,server=on,wait=off,id=hvc0 -device virtconsole,chardev=hvc0 -chardev socket,path=/tmp/autopkgtest-qemu.vpg40tc_/hvc1,server=on,wait=off,id=hvc1 -device virtconsole,chardev=hvc1 -serial unix:/tmp/autopkgtest-qemu.vpg40tc_/ttyS0,server=on,wait=off -drive index=0,file=/tmp/autopkgtest-qemu.vpg40tc_/overlay.img,format=qcow2,cache=unsafe,if=virtio,discard=unmap -drive if=pflash,format=raw,unit=0,read-only=on,file=/usr/share/AAVMF/AAVMF_CODE.fd -drive if=pflash,format=raw,unit=1,file=/tmp/autopkgtest-qemu.vpg40tc_/efivars.fd -enable-kvm

The CPU argument is wrong. It needs to be: -cpu host,aarch64=off.

On amd64 'autopkgtest -U --apt-pocket=proposed -a armhf -s *.dsc -- qemu autopkgtest-mantic-armhf.img' invokes QEMU with:

qemu-system-x86_64 -m 1024 -smp 1 -nographic -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:10022-:22 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0,id=rng-device0 -monitor unix:/tmp/autopkgtest-qemu.h0flvsby/monitor,server=on,wait=off -virtfs local,id=autopkgtest,path=/tmp/autopkgtest-qemu.h0flvsby/shared,security_model=none,mount_tag=autopkgtest -device virtio-serial -chardev socket,path=/tmp/autopkgtest-qemu.h0flvsby/hvc0,server=on,wait=off,id=hvc0 -device virtconsole,chardev=hvc0 -chardev socket,path=/tmp/autopkgtest-qemu.h0flvsby/hvc1,server=on,wait=off,id=hvc1 -device virtconsole,chardev=hvc1 -serial unix:/tmp/autopkgtest-qemu.h0flvsby/ttyS0,server=on,wait=off -serial unix:/tmp/autopkgtest-qemu.h0flvsby/ttyS1,server=on,wait=off -drive index=0,file=/tmp/autopkgtest-qemu.h0flvsby/overlay.img,format=qcow2,cache=unsafe,if=virtio,discard=unmap -enable-kvm -cpu host

The architecture flag is completely ignored.

On arm64 the following seems to work:

autopkgtest -U --apt-pocket=proposed -a armhf -s *.dsc -- qemu --qemu-options="-cpu host,aarch64=off" autopkgtest-mantic-armhf.img

But the '-cpu host,aarch64=off' QEMU option should be derived automatically from the '-a armhf' argument.

I ran into this. Found out that I need to specify --qemu-architecture [1] to autopkgtest-virt-qemu, so I believe your command would then be:

$ autopkgtest -U --apt-pocket=proposed -a armhf -s *.dsc -- qemu --qemu-architecture=$arch autopkgtest-mantic-armhf.img

Not sure if $arch should be arm or armhf.

But yes, I think this is redundant and should somehow be derived from autopkgtest's own -a flag.

[1] https://manpages.ubuntu.com/manpages/jammy/en/man1/autopkgtest-virt-qemu.1.html

Thanks Mateus,

autopkgtest -U --apt-pocket=proposed -a armhf -s *.dsc -- qemu --qemu-architecture=arm autopkgtest-mantic-armhf.img

leads to usage of -cpu host,aarch64=off.

autopkgtest should add --dpkg-architecture=armhf when invoking autopkgtest-virt-qemu.

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
https://iso.qa.ubuntu.com/qatracker/reports/bugs/2036730

myself, Paride Legovini and Brian Murray have encountered a bug where we can't even run autopkgtest-buildvm-ubuntu-cloud with armhf. We similarly get to this:

EFI stub: Booting Linux Kernel...
EFI stub: Entering in SVC mode with MMU enabled
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services...

But can never get to the step of actually running autopkgtest.

For clarity we used the exact command line mentioned at the beginning of this bug:
autopkgtest-buildvm-ubuntu-cloud -v --arch armhf --release mantic

"autopkgtest-buildvm-ubuntu-cloud -v --arch armhf --release mantic" on amd64 launches QEMU as

qemu-system-arm -machine virt -m 512 -smp 1 -nographic -net nic,model=virtio -net user,hostfwd=tcp:127.0.0.1:10022-:22 -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0,id=rng-device0 -monitor unix:/tmp/autopkgtest-qemu.n4bb59zs/monitor,server=on,wait=off -virtfs local,id=autopkgtest,path=/tmp/autopkgtest-qemu.n4bb59zs/shared,security_model=none,mount_tag=autopkgtest -device virtio-serial -chardev socket,path=/tmp/autopkgtest-qemu.n4bb59zs/hvc0,server=on,wait=off,id=hvc0 -device virtconsole,chardev=hvc0 -chardev socket,path=/tmp/autopkgtest-qemu.n4bb59zs/hvc1,server=on,wait=off,id=hvc1 -device virtconsole,chardev=hvc1 -serial unix:/tmp/autopkgtest-qemu.n4bb59zs/ttyS0,server=on,wait=off -drive index=0,file=/tmp/autopkgtest-buildvm-ubuntu-cloud1bu3xpgi/mantic-server-cloudimg-armhf.img,format=qcow2,if=virtio,discard=unmap -drive index=1,file=/tmp/autopkgtest-buildvm-ubuntu-cloud1bu3xpgi/autopkgtest.seed,format=raw,if=virtio,discard=unmap,readonly -drive if=pflash,format=raw,unit=0,read-only=on,file=/usr/share/AAVMF/AAVMF32_CODE.fd -drive if=pflash,format=raw,unit=1,file=/tmp/autopkgtest-qemu.n4bb59zs/efivars.fd

So here the QEMU architecture looks right.

When I run this command I see no issue except for missing feedback to the user for many minutes:

BdsDxe: loading Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x5,0x0)
BdsDxe: starting Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x5,0x0)
EFI stub: Booting Linux Kernel...
EFI stub: Entering in SVC mode with MMU enabled
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services...
Moving image into final destination ./autopkgtest-mantic-armhf.img

The problem here seems to be using console=/dev/ttyS0 instead of console=/dev/ttyAMA0 on the kernel command line.

So we have two separate issues to resolve:

- serial console on the command line for armhf images
- setting --qemu-architecture=arm for the 'autopkgtest -a armhf' command.

https://cloud-images.ubuntu.com/mantic/current/mantic-server-cloudimg-armhf.img contains file
/etc/default/grub.d/50-cloudimg-settings.cfg with line
GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0"

This line comes from livecd-rootfs:

live-build/ubuntu-cpc/hooks.d/chroot/999-cpc-fixes.chroot:189:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0"

So this needs to become architecture specific or maybe the line could be dropped.

For fixing the serial console on armhf and arm64 a merge request was created:
https://code.launchpad.net/~xypron/livecd-rootfs/+git/livecd-rootfs/+merge/453082