Can't boot: "error: out of memory." immediately after the grub menu
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OEM Priority Project |
Fix Released
|
Critical
|
jeremyszu | ||
grub |
Unknown
|
Unknown
|
|||
grub2-signed (Ubuntu) |
Fix Released
|
Critical
|
Julian Andres Klode | ||
Bionic |
Fix Released
|
Undecided
|
Unassigned | ||
Focal |
Fix Released
|
Undecided
|
Unassigned | ||
Jammy |
Fix Released
|
Undecided
|
Unassigned | ||
Kinetic |
Fix Released
|
Undecided
|
Unassigned | ||
grub2-unsigned (Ubuntu) |
Fix Released
|
Critical
|
Julian Andres Klode | ||
Bionic |
Fix Released
|
Undecided
|
Unassigned | ||
Focal |
Fix Released
|
Undecided
|
Unassigned | ||
Jammy |
Fix Released
|
Undecided
|
Unassigned | ||
Kinetic |
Fix Released
|
Undecided
|
Unassigned |
Bug Description
[Workaround]
Today, 2023-02-14, the following link works to find the workarounds:
https:/
[Impact]
* In some cases, if the users’ initramfs grow bigger, then it’ll likely not be able to be loaded by grub2.
* Some real cases from OEM projects:
In many built-in 4k monitor laptops with nvidia drivers, the u-d-c puts the nvidia*.ko to initramfs which grows the initramfs to ~120M. Also the gfxpayload=auto will remain to use 4K resolution since it’s what EFI POST passed.
In this case, the grub isn't able to load initramfs because the grub_memalign() won't be able to get suitable memory for the larger file:
```
#0 grub_memalign (align=1, size=592214020) at ../../.
#1 0x000000007dd7b074 in grub_malloc (size=592214020) at ../../.
#2 0x000000007dd7a2c8 in grub_verifiers_open (io=0x7bc02d80, type=131076)
at ../../.
#3 0x000000007dd801d4 in grub_file_open (name=0x7bc02f00 "/boot/
type=131076) at ../../.
#4 0x000000007bcd5a30 in ?? ()
#5 0x000000007fe21247 in ?? ()
#6 0x000000007bc030c8 in ?? ()
#7 0x000000017fe21238 in ?? ()
#8 0x000000007bcd5320 in ?? ()
#9 0x000000007fe21250 in ?? ()
#10 0x0000000000000000 in ?? ()
```
Based on grub_mm_dump, we can see the memory region starvation in <1G addresses:
Type Start End # Pages Attributes
Available 000000000000000
BS_Data 000000000008700
Available 000000000008800
Reserved 000000000009F00
Available 000000000010000
LoaderCode 000000000100000
Available 000000000102200
BS_Data 00000000238A800
Available 000000002392800
BS_Data 000000002886100
LoaderCode 000000002AB0A00
BS_Data 000000002ACF900
Available 000000002B2FB00
BS_Data 000000002B61200
Available 000000002B63100
BS_Data 000000002B63300
Available 000000002B63D00
BS_Data 000000002B64A00
Available 000000002B64F00
BS_Data 000000002B66700
LoaderCode 000000002D8D600
BS_Data 000000002D8EA00
LoaderCode 000000002D92600
BS_Data 000000002D93300
BS_Code 000000002D96A00
BS_Data 000000002D97400
Available 000000002E37800
...
Reserved 000000003C08B00
ACPI_NVS 000000004193000
ACPI_Recl 0000000041B3000
BS_Data 0000000041BFF00
Available 000000010000000
Reserved 00000000000A000
Reserved 0000000041C0000
Reserved 000000004400000
Reserved 000000004940000
Reserved 000000004C00000
Reserved 000000005000000
MMIO 00000000C000000
Reserved 00000000FED2000
MMIO 00000000FF80000
...
LoaderCode: 562 Pages (2,301,952 Bytes)
LoaderData: 0 Pages (0 Bytes)
...
Available : 1,917,598 Pages (7,854,481,408 Bytes)
Based on UEFI Specification Section 7.2[1] and UEFI driver writers’ guide 4.2.3[2], we can ask 32bits+ on AllocatePages().
As most X86_64 platforms should support 64 bits addressing, we should extend GRUB_EFI_
* When users grown the initramfs, then probably will get initramfs not found which really annoyed and impact the user experience (system not able to boot).
[Test Plan]
tl;dr: Create an initrd that fails to boot with the old grub and boot it successfully with the new grub.
* detailed instructions how to reproduce the bug:
1. Any method to grow the initramfs, such as install nvidia-driver.
2. If developers would like to reproduce, then could dd if=/dev/random of=... bs=1M count=500, something like:
```
$ cat /usr/share/
#!/bin/sh
PREREQ=""
prereqs()
{
echo "$PREREQ"
}
case $1 in
# get pre-requisites
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/
dd if=/dev/random of=${DESTDIR}
```
And then update-initramfs
* After cloning grub2 (2.06-2ubuntu16) from lunar-proposed and built, the results as following:
421M /boot/initrd.
453M /boot/initrd.
471M /boot/initrd.
Only 453M is because verifier will consume the same memory size of initrd in <1G memories.
The loadable initrd will locate at >4G if machine/kernel support it.
[Where problems could occur]
* The changes almost in i386/efi, thus the impact will be in the i386 / x86_64 EFI system.
The other change is to modify the “grub-core/
Thus it should not impact them.
* There are some “#if defined(
If everything works as expected, then i386 should working good.
If not lucky, based on “UEFI writers’ guide”[2], the i386 will get > 4GB memory region and never be able to access.
[Other Info]
* The package grub2 (2.06-2ubuntu16) is now in lunar-proposed.
---
Upgraded from 19.04 to current 19.10 using "do-release-upgrade -d". Can still boot using the previous 5.0.0-25-generic kernel, but the 5.2.0-15-generic fails to start.
On selecting Ubuntu from Grub, the message "error: out of memory." is immediately shown. Pressing a key attempts to start boot-up but fails to mount root fs.
Machine is HP Spectre X360 with 8GB RAM. Under kernel 5.0.0, free shows the following (run from Gnome terminal):
total used free shared buff/cache available
Mem: 7906564 1761196 3833240 1020216 2312128 4849224
Swap: 1003516 0 1003516
Kernel packages installed:
linux-generic 5.2.0.15.16 amd64
linux-headers-
linux-headers-
linux-headers-
linux-image-
linux-image-
linux-image-generic 5.2.0.15.16 amd64
linux-modules-
linux-modules-
linux-modules-
linux-modules-
Photo of kernel panic attached.
NVMe drive partition layout (GPT):
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p2 1050624 2549759 1499136 732M Linux filesystem
/dev/nvme0n1p3 2549760 1000214527 997664768 475.7G Linux filesystem
$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
root ubuntu-vg -wi-ao---- 474.75g
swap_1 ubuntu-vg -wi-ao---- 980.00m
Partition 3 is LUKS encrypted. Root LV is ext4.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/
CurrentDesktop: ubuntu:
InstallationDate: Installed on 2019-08-15 (18 days ago)
InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Release amd64 (20190416)
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 002: ID 04f2:b593 Chicony Electronics Co., Ltd HP Wide Vision FHD Camera
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: HP HP Spectre x360 Convertible 13-ae0xx
Package: linux (not installed)
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=
ProcVersionSign
RelatedPackageV
linux-
linux-
linux-firmware 1.181
Tags: eoan
Uname: Linux 5.0.0-25-generic x86_64
UpgradeStatus: Upgraded to eoan on 2019-09-02 (0 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True
dmi.bios.date: 05/17/2019
dmi.bios.vendor: AMI
dmi.bios.version: F.25
dmi.board.
dmi.board.name: 83B9
dmi.board.vendor: HP
dmi.board.version: 56.43
dmi.chassis.type: 31
dmi.chassis.vendor: HP
dmi.chassis.
dmi.modalias: dmi:bvnAMI:
dmi.product.family: 103C_5335KV HP Spectre
dmi.product.name: HP Spectre x360 Convertible 13-ae0xx
dmi.product.sku: 2QH38EA#ABU
dmi.sys.vendor: HP
affects: | ubuntu → linux (Ubuntu) |
tags: |
added: jammy removed: disco eoan |
tags: | added: jiayi oem-priority originate-from-1972964 |
Changed in oem-priority: | |
assignee: | nobody → jeremyszu (os369510) |
importance: | Undecided → Critical |
status: | New → In Progress |
tags: | added: jellyfish-edge-staging |
Changed in grub2-signed (Ubuntu): | |
status: | New → Confirmed |
Changed in initramfs-tools (Ubuntu): | |
status: | New → Confirmed |
description: | updated |
description: | updated |
Changed in oem-priority: | |
status: | In Progress → Triaged |
tags: | added: patch |
Changed in oem-priority: | |
status: | Triaged → In Progress |
Changed in oem-priority: | |
status: | In Progress → Triaged |
summary: |
- Out of Memory on boot with 5.2.0 kernel + Out of Memory on boot |
tags: | added: kinetic |
Changed in grub2-signed (Ubuntu): | |
importance: | Undecided → Critical |
Changed in initramfs-tools (Ubuntu): | |
importance: | Undecided → Critical |
Changed in linux (Ubuntu): | |
importance: | Undecided → Critical |
summary: |
- Out of Memory on boot + Can't boot: "error: out of memory." immediately after the grub menu |
tags: | added: foundations-todo |
Changed in grub2-signed (Ubuntu): | |
status: | Confirmed → Triaged |
Changed in initramfs-tools (Ubuntu): | |
status: | Confirmed → Won't Fix |
Changed in grub2-unsigned (Ubuntu): | |
status: | New → Triaged |
importance: | Undecided → Critical |
tags: | added: originate-from-1994098 stella |
tags: | added: fr-2934 |
Changed in grub2-unsigned (Ubuntu): | |
assignee: | nobody → Julian Andres Klode (juliank) |
Changed in grub2-signed (Ubuntu): | |
assignee: | nobody → Julian Andres Klode (juliank) |
description: | updated |
tags: | added: originate-from-1998320 somerville |
Changed in grub2-unsigned (Ubuntu): | |
status: | Triaged → Fix Committed |
Changed in grub2-signed (Ubuntu): | |
status: | Triaged → Fix Committed |
description: | updated |
description: | updated |
Changed in grub2-unsigned (Ubuntu): | |
status: | Fix Committed → Fix Released |
tags: | added: originate-from-1998995 |
tags: | added: originate-from-2000298 |
Changed in grub2-signed (Ubuntu): | |
milestone: | none → ubuntu-22.04.2 |
Changed in grub2-unsigned (Ubuntu): | |
milestone: | none → ubuntu-22.04.2 |
no longer affects: | initramfs-tools (Ubuntu Kinetic) |
no longer affects: | initramfs-tools (Ubuntu Jammy) |
no longer affects: | initramfs-tools (Ubuntu Focal) |
no longer affects: | initramfs-tools (Ubuntu Bionic) |
no longer affects: | linux (Ubuntu) |
no longer affects: | initramfs-tools (Ubuntu) |
no longer affects: | linux (Ubuntu Bionic) |
no longer affects: | linux (Ubuntu Focal) |
no longer affects: | linux (Ubuntu Jammy) |
no longer affects: | linux (Ubuntu Kinetic) |
description: | updated |
Changed in grub2-signed (Ubuntu Bionic): | |
status: | New → In Progress |
Changed in grub2-signed (Ubuntu Focal): | |
status: | New → In Progress |
Changed in grub2-signed (Ubuntu Jammy): | |
status: | New → In Progress |
Changed in grub2-signed (Ubuntu Kinetic): | |
status: | New → In Progress |
Changed in grub2-unsigned (Ubuntu Bionic): | |
status: | New → In Progress |
Changed in grub2-unsigned (Ubuntu Kinetic): | |
status: | New → In Progress |
Changed in grub2-unsigned (Ubuntu Jammy): | |
status: | New → In Progress |
Changed in grub2-unsigned (Ubuntu Focal): | |
status: | New → In Progress |
Changed in grub2-unsigned (Ubuntu Kinetic): | |
status: | In Progress → Fix Committed |
tags: | added: verification-needed verification-needed-kinetic |
Changed in grub2-signed (Ubuntu Kinetic): | |
status: | In Progress → Fix Committed |
Changed in grub2-unsigned (Ubuntu Jammy): | |
status: | In Progress → Fix Committed |
tags: | added: verification-needed-jammy |
Changed in grub2-signed (Ubuntu Jammy): | |
status: | In Progress → Fix Committed |
tags: |
added: verification-done-kinetic removed: verification-needed-kinetic |
description: | updated |
tags: |
added: verification-done-bionic removed: block-proposed-bionic verification-needed-bionic |
Changed in oem-priority: | |
status: | Triaged → Fix Released |
description: | updated |
summary: |
- Can't boot: "error: out of memory." immediately after the grub menu + Can't boot: "error: out of memory." immediately after having selected + ubuntu on the grub menu if it is populated with a lof of entries. |
summary: |
Can't boot: "error: out of memory." immediately after having selected - ubuntu on the grub menu if it is populated with a lof of entries. + ubuntu on the grub menu if it is populated with a lot of entries. |
Changed in grub2-signed (Ubuntu): | |
status: | Fix Committed → Fix Released |
tags: | removed: foundations-todo |
This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:
apport-collect 1842320
and then change the status of the bug to 'Confirmed'.
If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.
This change has been made by an automated script, maintained by the Ubuntu Kernel Team.