| 2023-06-29 21:47:22 |
William Tu |
description |
* Explain the bug(s)
We've found that for Jammy 5.15 and also 5.4.0-1049 kernel running on Bluefield, the kdump doesn't work when enabling secure boot[1].
* How to test
# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0xcfe00000
/boot/vmlinuz-5.4.0-1049-bluefield
kdump initrd:
/boot/initrd.img-5.4.0-1049-bluefield
current state: Not ready to kdump
kdump-tools.service - Kernel crash dump capture service
Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor preset: enabled)
Active: active (exited) since Thu 2023-04-13 19:21:01 UTC; 4 days ago
Process: 1975 ExecStart=/etc/init.d/kdump-tools start (code=exited, status=0/SUCCESS)
Main PID: 1975 (code=exited, status=0/SUCCESS)
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net systemd[1]: Starting Kernel crash dump capture serv>
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[1975]: Starting kdump-tools:
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu>
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu>
Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2525]: syscall kexec_file_load not avai>
Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * failed to load kdump kernel
* Possible reason
Currently, a problem faced by arm64 is if a kernel image is signed by a MOK key, loading it via the kexec_file_load() system call would be rejected with the error in dmesg
"Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7".
I backported the two below [2]:
0d519cadf751 arm64: kexec_file: use more system keyrings to verify kernel image signature
c903dae8941d kexec, KEYS: make the code in bzImage64_verify_sig generic
However still kdump/kexec fails due to lockdown
[ 353.298348] Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7
[ 364.833004] audit: type=1400 audit(1686619435.331:16): apparmor="STATUS" operation="profile_load" profile="unconfined" name="cri-containerd.apparmor.d" pid=15407 comm="apparmor_parser"
If I disable kernel lockdown (CONFIG_SECURITY_LOCKDOWN_LSM=n), then kexec works ok. But this is not an acceptable workaround.
* How to fix
I got it working (secure boot + lockdown config enabled + kdump/kexec) using kernel v6.4 on bluefield, which means if we backport properly, mostly missing some arch/arm64/ patches, to BlueField 5.15 kernel, we can get it working.
Reference
1. https://docs.nvidia.com/networking/display/BlueFieldDPUOSv392/UEFI%20Secure%20Boot
2. https://www.spinics.net/lists/arm-kernel/msg979554.html
3. https://mjg59.dreamwidth.org/50577.html |
* Explain the bug(s)
We've found that for Jammy 5.15 and also 5.4.0-1049 kernel running on Bluefield, the kdump doesn't work when enabling secure boot[1].
* How to test
Make sure kernel config:
CONFIG_SECURITY_LOCKDOWN_LSM=y
CONFIG_SECURITY_LOCKDOWN_LSM_EARLY=y
CONFIG_LOCK_DOWN_IN_SECURE_BOOT=y
The kdump kernel we use is actually signed correctly
# file /boot/vmlinuz-5.15.0-1015-bluefield
/boot/vmlinuz-5.15.0-1015-bluefield: gzip compressed data, was "vmlinuz-5.15.0 1015-bluefield.efi.signed",
# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0xcfe00000
/boot/vmlinuz-5.4.0-1049-bluefield
kdump initrd:
/boot/initrd.img-5.4.0-1049-bluefield
current state: Not ready to kdump
kdump-tools.service - Kernel crash dump capture service
Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor preset: enabled)
Active: active (exited) since Thu 2023-04-13 19:21:01 UTC; 4 days ago
Process: 1975 ExecStart=/etc/init.d/kdump-tools start (code=exited, status=0/SUCCESS)
Main PID: 1975 (code=exited, status=0/SUCCESS)
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net systemd[1]: Starting Kernel crash dump capture serv>
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[1975]: Starting kdump-tools:
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu>
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu>
Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2525]: syscall kexec_file_load not avai>
Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * failed to load kdump kernel
* Possible reason
Currently, a problem faced by arm64 is if a kernel image is signed by a MOK key, loading it via the kexec_file_load() system call would be rejected with the error in dmesg
"Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7".
I backported the two below [2]:
0d519cadf751 arm64: kexec_file: use more system keyrings to verify kernel image signature
c903dae8941d kexec, KEYS: make the code in bzImage64_verify_sig generic
However still kdump/kexec fails due to lockdown
[ 353.298348] Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7
[ 364.833004] audit: type=1400 audit(1686619435.331:16): apparmor="STATUS" operation="profile_load" profile="unconfined" name="cri-containerd.apparmor.d" pid=15407 comm="apparmor_parser"
If I disable kernel lockdown (CONFIG_SECURITY_LOCKDOWN_LSM=n), then kexec works ok. But this is not an acceptable workaround.
* How to fix
I got it working (secure boot + lockdown config enabled + kdump/kexec) using kernel v6.4 on bluefield, which means if we backport properly, mostly missing some arch/arm64/ patches, to BlueField 5.15 kernel, we can get it working.
Reference
1. https://docs.nvidia.com/networking/display/BlueFieldDPUOSv392/UEFI%20Secure%20Boot
2. https://www.spinics.net/lists/arm-kernel/msg979554.html
3. https://mjg59.dreamwidth.org/50577.html |
|
