Make linux-kvm bootable in LXD VMs
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
cloud-images |
Invalid
|
Undecided
|
Unassigned | ||
linux-kvm (Ubuntu) |
Fix Released
|
High
|
Unassigned | ||
Focal |
Fix Released
|
High
|
Stefan Bader |
Bug Description
The `disk-kvm.img` images which are to be preferred when run under virtualization, currently completely fail to boot under UEFI.
A workaround was put in place such that LXD instead will pull generic-based images until this is resolved, this however does come with a much longer boot time (as the kernel panics, reboots and then boots) and also reduced functionality from cloud-init, so we'd still like this fixed in the near future.
To get things behaving, it looks like we need the following config options to be enable in linux-kvm:
- CONFIG_EFI_STUB
- CONFIG_VSOCKETS
- CONFIG_
- CONFIG_
== Rationale ==
We'd like to be able to use the linux-kvm based images for LXD, those will directly boot without needing the panic+reboot behavior of generic images and will be much lighter in general.
We also need the LXD agent to work, which requires functional virtio vsock.
== Test case ==
- wget http://
- wget http://
- lxc image import focal-server-
- lxc launch bug1873809 v1
- lxc console v1
- <check that it boots to login prompt>
- <disconnect with ctrl+a-q>
- lxc exec v1 bash
To validate a new kernel, you'll need to manually repack the .img file and install the new kernel in there.
== Regression potential ==
I don't know who else is using those kvm images right now, but those changes will cause a change to the kernel binary such that it contains the EFI stub bits + a signature. This could cause some (horribly broken) systems to no longer be able to boot that kernel. Though considering that such a setup is common to our other kernels, this seems unlikely.
Also, this will be introducing virtio vsock support which again, could maybe confused some horribly broken systems?
In either case, the kernel conveniently is the only package which ships multiple versions concurently, so rebooting on the previous kernel is always an option, mitigating some of the risks.
-- Details from original report --
User report on the LXD side: https:/
I've reproduced this issue with:
- wget http://
- qemu-system-x86_64 -bios /usr/share/
On the graphical console, you'll see EDK2 load (TianoCore) followed by basic boot messages and then a message from grub (error: can't find command `hwmatch`).
Those also appear on successful boots of other images so I don't think there's anything concerning that. However it'll hang indefinitely and eat up all your CPU.
Switching to the text console view (serial0), you'll see the same issue as that LXD report:
BdsDxe: failed to load Boot0001 "UEFI QEMU DVD-ROM QM00003 " from PciRoot(
BdsDxe: loading Boot0002 "UEFI QEMU HARDDISK QM00001 " from PciRoot(
BdsDxe: starting Boot0002 "UEFI QEMU HARDDISK QM00001 " from PciRoot(
error: can't find command `hwmatch'.
e!!!! X64 Exception Type - 0D(#GP - General Protection) CPU Apic ID - 00000000 !!!!
ExceptionData - 0000000000000000
RIP - 000000003FF2DA12, CS - 0000000000000038, RFLAGS - 0000000000200202
RAX - AFAFAFAFAFAFAFAF, RCX - 000000003E80F108, RDX - AFAFAFAFAFAFAFAF
RBX - 0000000000000398, RSP - 000000003FF1C638, RBP - 000000003FF34360
RSI - 000000003FF343B8, RDI - 0000000000001000
R8 - 000000003E80F108, R9 - 000000003E815B98, R10 - 0000000000000065
R11 - 0000000000002501, R12 - 0000000000000004, R13 - 000000003E80F100
R14 - 0000000000000000, R15 - 0000000000000000
DS - 0000000000000030, ES - 0000000000000030, FS - 0000000000000030
GS - 0000000000000030, SS - 0000000000000030
CR0 - 0000000080010033, CR2 - 0000000000000000, CR3 - 000000003FC01000
CR4 - 0000000000000668, CR8 - 0000000000000000
DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000
DR3 - 0000000000000000, DR6 - 00000000FFFF0FF0, DR7 - 0000000000000400
GDTR - 000000003FBEEA98 0000000000000047, LDTR - 0000000000000000
IDTR - 000000003F2D8018 0000000000000FFF, TR - 0000000000000000
FXSAVE_STATE - 000000003FF1C290
!!!! Find image based on IP(0x3FF2DA12) /build/
If booting in a SecureBoot enabled environment, you instead get a `Access Denied` at kernel loading time, indicating that the kernel binary isn't a normal signed kernel. That has the same result (boot hangs) but without the crash message.
CVE References
description: | updated |
Changed in linux-kvm (Ubuntu): | |
assignee: | nobody → Colin Ian King (colin-king) |
importance: | Undecided → High |
Changed in cloud-images: | |
status: | Invalid → Confirmed |
assignee: | nobody → Roufique Hossain (roufique) |
Changed in linux-kvm (Ubuntu): | |
assignee: | Colin Ian King (colin-king) → Roufique Hossain (roufique) |
status: | New → Incomplete |
status: | Incomplete → Confirmed |
Changed in cloud-bl-tutorials: | |
status: | New → Confirmed |
Changed in linux-kvm (Ubuntu Focal): | |
assignee: | nobody → Stefan Bader (smb) |
importance: | Undecided → High |
status: | New → In Progress |
Changed in linux-kvm (Ubuntu Focal): | |
status: | In Progress → Fix Committed |
Ff we can't easily track down & fix the source of this issue, an alternative is to modify the streams to strip the kvm-img hash from the LXD metadata record, forcing all our users onto the non-kvm image. Do note that this comes with its own issues though.
Those non-kvm images attempt to boot without an initrd, which fails under LXD due to missing virtio drivers. The kernel then panics and reboots back into grub, this time booting with an initrd.
On top of the added boot time and CPU load this causes, it also means that the VMs never boot properly on first boot. This breaks LXD's cloud-init support, so those VMs will never get their cloud-init data.