Activity log for bug #1874147

Date Who What changed Old value New value Message
2020-04-21 21:33:41 David Baird bug added bug
2020-04-21 21:56:26 David Baird bug task added grub2 (Ubuntu)
2020-04-21 23:41:56 Dexuan Cui bug added subscriber Dexuan Cui
2020-05-12 16:17:06 David Baird summary focal-server-cloudimg-amd64.img panics on first boot, cannot open root device virt-install of focal-server-cloudimg-amd64.img panics on first boot, breaking cloud-init
2020-05-12 16:19:04 David Baird description Various Focal cloudimg builds fail on first time ever booting (e.g. https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img). Here is the failed output: [ 0.000000] Linux version 5.4.0-24-generic (buildd@lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1 ... [ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6 [ 1.045238] Please append a correct "root=" boot option; here are the available partitions: [ 1.046921] 0b00 1048575 sr0 [ 1.046922] driver: sr [ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu [ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014 [ 1.052904] Call Trace: [ 1.053459] dump_stack+0x6d/0x9a [ 1.054158] panic+0x101/0x2e3 [ 1.054812] mount_block_root+0x23f/0x2e8 [ 1.055645] mount_root+0x38/0x3a [ 1.056319] prepare_namespace+0x13f/0x194 [ 1.057236] kernel_init_freeable+0x231/0x255 [ 1.058131] ? rest_init+0xb0/0xb0 [ 1.058834] kernel_init+0xe/0x100 [ 1.059535] ret_from_fork+0x35/0x40 [ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) When booted, the second time, it works! The issue appears to be a bug in /boot/grub/grub.cfg, pertaining to "initrdfail." On first ever boot, grub will try to boot *without* an initrd, which (apparently) makes the kernel unable to mount root. Relevant section of /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {         recordfail         load_video         gfxmode $linux_gfx_mode         insmod gzio         if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi         insmod part_gpt         insmod ext2         if [ x$feature_platform_search_hint = xy ]; then           search --no-floppy --fs-uuid --set=root  aca11948-bdc4-4726-9aca-ead9dcce6a5e         else           search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e         fi         if [ "${initrdfail}" = 1 ]; then           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0           initrd        /boot/initrd.img-5.4.0-24-generic         else           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0 panic=-1         fi         initrdfail } Various Focal cloudimg builds panic on first time ever booting with "Cannot open root device" when booting with virt-install (libvirt). E.g. https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img Here is the failed output: [ 0.000000] Linux version 5.4.0-24-generic (buildd@lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1 ... [ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6 [ 1.045238] Please append a correct "root=" boot option; here are the available partitions: [ 1.046921] 0b00 1048575 sr0 [ 1.046922] driver: sr [ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu [ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014 [ 1.052904] Call Trace: [ 1.053459] dump_stack+0x6d/0x9a [ 1.054158] panic+0x101/0x2e3 [ 1.054812] mount_block_root+0x23f/0x2e8 [ 1.055645] mount_root+0x38/0x3a [ 1.056319] prepare_namespace+0x13f/0x194 [ 1.057236] kernel_init_freeable+0x231/0x255 [ 1.058131] ? rest_init+0xb0/0xb0 [ 1.058834] kernel_init+0xe/0x100 [ 1.059535] ret_from_fork+0x35/0x40 [ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) When booted, the second time, it works! The issue appears to be a bug in /boot/grub/grub.cfg, pertaining to "initrdfail." On first ever boot, grub will try to boot *without* an initrd, which (apparently) makes the kernel unable to mount root. Relevant section of /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {         recordfail         load_video         gfxmode $linux_gfx_mode         insmod gzio         if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi         insmod part_gpt         insmod ext2         if [ x$feature_platform_search_hint = xy ]; then           search --no-floppy --fs-uuid --set=root  aca11948-bdc4-4726-9aca-ead9dcce6a5e         else           search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e         fi         if [ "${initrdfail}" = 1 ]; then           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0           initrd        /boot/initrd.img-5.4.0-24-generic         else           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0 panic=-1         fi         initrdfail }
2020-05-12 16:21:49 David Baird description Various Focal cloudimg builds panic on first time ever booting with "Cannot open root device" when booting with virt-install (libvirt). E.g. https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img Here is the failed output: [ 0.000000] Linux version 5.4.0-24-generic (buildd@lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1 ... [ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6 [ 1.045238] Please append a correct "root=" boot option; here are the available partitions: [ 1.046921] 0b00 1048575 sr0 [ 1.046922] driver: sr [ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu [ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014 [ 1.052904] Call Trace: [ 1.053459] dump_stack+0x6d/0x9a [ 1.054158] panic+0x101/0x2e3 [ 1.054812] mount_block_root+0x23f/0x2e8 [ 1.055645] mount_root+0x38/0x3a [ 1.056319] prepare_namespace+0x13f/0x194 [ 1.057236] kernel_init_freeable+0x231/0x255 [ 1.058131] ? rest_init+0xb0/0xb0 [ 1.058834] kernel_init+0xe/0x100 [ 1.059535] ret_from_fork+0x35/0x40 [ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) When booted, the second time, it works! The issue appears to be a bug in /boot/grub/grub.cfg, pertaining to "initrdfail." On first ever boot, grub will try to boot *without* an initrd, which (apparently) makes the kernel unable to mount root. Relevant section of /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {         recordfail         load_video         gfxmode $linux_gfx_mode         insmod gzio         if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi         insmod part_gpt         insmod ext2         if [ x$feature_platform_search_hint = xy ]; then           search --no-floppy --fs-uuid --set=root  aca11948-bdc4-4726-9aca-ead9dcce6a5e         else           search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e         fi         if [ "${initrdfail}" = 1 ]; then           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0           initrd        /boot/initrd.img-5.4.0-24-generic         else           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0 panic=-1         fi         initrdfail } Various Focal cloudimg builds panic on first time ever booting with "Cannot open root device" when booting with virt-install (libvirt). After panicing, it reboots, and the second boot succeeds (using initrd on the second try). However, it needs to succeed on the first try without a panic. This breaks the cloud-init NoCloud process which expects the first time booting to succeed. E.g. https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img Here is the failed output: [ 0.000000] Linux version 5.4.0-24-generic (buildd@lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1 ... [ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6 [ 1.045238] Please append a correct "root=" boot option; here are the available partitions: [ 1.046921] 0b00 1048575 sr0 [ 1.046922] driver: sr [ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu [ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014 [ 1.052904] Call Trace: [ 1.053459] dump_stack+0x6d/0x9a [ 1.054158] panic+0x101/0x2e3 [ 1.054812] mount_block_root+0x23f/0x2e8 [ 1.055645] mount_root+0x38/0x3a [ 1.056319] prepare_namespace+0x13f/0x194 [ 1.057236] kernel_init_freeable+0x231/0x255 [ 1.058131] ? rest_init+0xb0/0xb0 [ 1.058834] kernel_init+0xe/0x100 [ 1.059535] ret_from_fork+0x35/0x40 [ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) When booted, the second time, it works! The issue appears to be a bug in /boot/grub/grub.cfg, pertaining to "initrdfail." On first ever boot, grub will try to boot *without* an initrd, which (apparently) makes the kernel unable to mount root. Relevant section of /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {         recordfail         load_video         gfxmode $linux_gfx_mode         insmod gzio         if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi         insmod part_gpt         insmod ext2         if [ x$feature_platform_search_hint = xy ]; then           search --no-floppy --fs-uuid --set=root  aca11948-bdc4-4726-9aca-ead9dcce6a5e         else           search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e         fi         if [ "${initrdfail}" = 1 ]; then           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0           initrd        /boot/initrd.img-5.4.0-24-generic         else           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0 panic=-1         fi         initrdfail }
2020-05-12 16:26:29 David Baird bug task added virt-manager (Ubuntu)
2020-05-12 16:30:05 David Baird description Various Focal cloudimg builds panic on first time ever booting with "Cannot open root device" when booting with virt-install (libvirt). After panicing, it reboots, and the second boot succeeds (using initrd on the second try). However, it needs to succeed on the first try without a panic. This breaks the cloud-init NoCloud process which expects the first time booting to succeed. E.g. https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img Here is the failed output: [ 0.000000] Linux version 5.4.0-24-generic (buildd@lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1 ... [ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6 [ 1.045238] Please append a correct "root=" boot option; here are the available partitions: [ 1.046921] 0b00 1048575 sr0 [ 1.046922] driver: sr [ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu [ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014 [ 1.052904] Call Trace: [ 1.053459] dump_stack+0x6d/0x9a [ 1.054158] panic+0x101/0x2e3 [ 1.054812] mount_block_root+0x23f/0x2e8 [ 1.055645] mount_root+0x38/0x3a [ 1.056319] prepare_namespace+0x13f/0x194 [ 1.057236] kernel_init_freeable+0x231/0x255 [ 1.058131] ? rest_init+0xb0/0xb0 [ 1.058834] kernel_init+0xe/0x100 [ 1.059535] ret_from_fork+0x35/0x40 [ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) When booted, the second time, it works! The issue appears to be a bug in /boot/grub/grub.cfg, pertaining to "initrdfail." On first ever boot, grub will try to boot *without* an initrd, which (apparently) makes the kernel unable to mount root. Relevant section of /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {         recordfail         load_video         gfxmode $linux_gfx_mode         insmod gzio         if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi         insmod part_gpt         insmod ext2         if [ x$feature_platform_search_hint = xy ]; then           search --no-floppy --fs-uuid --set=root  aca11948-bdc4-4726-9aca-ead9dcce6a5e         else           search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e         fi         if [ "${initrdfail}" = 1 ]; then           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0           initrd        /boot/initrd.img-5.4.0-24-generic         else           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0 panic=-1         fi         initrdfail } Various Focal cloudimg builds panic on first time ever booting with "Cannot open root device" when booting with virt-install (libvirt). After panicing, it reboots, and the second boot succeeds (using initrd on the second try). However, it needs to succeed on the first try without a panic. This breaks the cloud-init NoCloud process which expects the first time booting to succeed. E.g. https://cloud-images.ubuntu.com/focal/20200418/focal-server-cloudimg-amd64.img Here is the failed output: [ 0.000000] Linux version 5.4.0-24-generic (buildd@lgw01-amd64-017) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 (Ubuntu 5.4.0-24.28-generic 5.4.30) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-24-generic root=PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3 ro console=tty1 console=ttyS0 panic=-1 ... [ 1.043162] VFS: Cannot open root device "PARTUUID=8a7c6798-57bf-4624-80a0-b896a73939f3" or unknown-block(0,0): error -6 [ 1.045238] Please append a correct "root=" boot option; here are the available partitions: [ 1.046921] 0b00 1048575 sr0 [ 1.046922] driver: sr [ 1.048215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [ 1.049795] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 5.4.0-24-generic #28-Ubuntu [ 1.051189] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014 [ 1.052904] Call Trace: [ 1.053459] dump_stack+0x6d/0x9a [ 1.054158] panic+0x101/0x2e3 [ 1.054812] mount_block_root+0x23f/0x2e8 [ 1.055645] mount_root+0x38/0x3a [ 1.056319] prepare_namespace+0x13f/0x194 [ 1.057236] kernel_init_freeable+0x231/0x255 [ 1.058131] ? rest_init+0xb0/0xb0 [ 1.058834] kernel_init+0xe/0x100 [ 1.059535] ret_from_fork+0x35/0x40 [ 1.060327] Kernel Offset: 0x1f800000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) When booted, the second time, it works! The issue appears to be that cloudimg's kernel doesn't come with the necessary drivers pre-compiled into the kernel on a KVM/QEMU system. And, combined with a recent (2019) change to /boot/grub/grub.cfg which doesn't load initrd on the first boot, this ensures that the first time booting will fail. Relevant section of /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-aca11948-bdc4-4726-9aca-ead9dcce6a5e' {         recordfail         load_video         gfxmode $linux_gfx_mode         insmod gzio         if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi         insmod part_gpt         insmod ext2         if [ x$feature_platform_search_hint = xy ]; then           search --no-floppy --fs-uuid --set=root  aca11948-bdc4-4726-9aca-ead9dcce6a5e         else           search --no-floppy --fs-uuid --set=root aca11948-bdc4-4726-9aca-ead9dcce6a5e         fi         if [ "${initrdfail}" = 1 ]; then           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0           initrd        /boot/initrd.img-5.4.0-24-generic         else           linux /boot/vmlinuz-5.4.0-24-generic root=LABEL=cloudimg-rootfs ro  console=tty1 console=ttyS0 panic=-1         fi         initrdfail }
2020-05-28 07:25:39 Christian Ehrhardt  bug added subscriber Christian Ehrhardt 