Comment 0 for bug 2025396

* Explain the bug(s)

We've found that for Jammy 5.15 and also 5.4.0-1049 kernel running on Bluefield, the kdump doesn't work when enabling secure boot[1].

* How to test

# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0xcfe00000
   /boot/vmlinuz-5.4.0-1049-bluefield
kdump initrd:
   /boot/initrd.img-5.4.0-1049-bluefield
current state: Not ready to kdump

kdump-tools.service - Kernel crash dump capture service
     Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor preset: enabled)
     Active: active (exited) since Thu 2023-04-13 19:21:01 UTC; 4 days ago
    Process: 1975 ExecStart=/etc/init.d/kdump-tools start (code=exited, status=0/SUCCESS)
   Main PID: 1975 (code=exited, status=0/SUCCESS)
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net systemd[1]: Starting Kernel crash dump capture serv>
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[1975]: Starting kdump-tools:
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu>
Apr 13 19:20:59 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * Creating symlink /var/lib/kdu>
Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2525]: syscall kexec_file_load not avai>
Apr 13 19:21:01 br2-004-a-dpu.nd-sjc6w.nvmetal.net kdump-tools[2184]: * failed to load kdump kernel

* Possible reason
Currently, a problem faced by arm64 is if a kernel image is signed by a MOK key, loading it via the kexec_file_load() system call would be rejected with the error in dmesg
"Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7".

I backported the two below [2]:
0d519cadf751 arm64: kexec_file: use more system keyrings to verify kernel image signature
c903dae8941d kexec, KEYS: make the code in bzImage64_verify_sig generic

However still kdump/kexec fails due to lockdown
[ 353.298348] Lockdown: kexec: kexec of unsigned images is restricted; see man kernel_lockdown.7
[ 364.833004] audit: type=1400 audit(1686619435.331:16): apparmor="STATUS" operation="profile_load" profile="unconfined" name="cri-containerd.apparmor.d" pid=15407 comm="apparmor_parser"

If I disable kernel lockdown (CONFIG_SECURITY_LOCKDOWN_LSM=n), then kexec works ok. But this is not an acceptable workaround.

* How to fix
I got it working (secure boot + lockdown config enabled + kdump/kexec) using kernel v6.4 on bluefield, which means if we backport properly, mostly missing some arch/arm64/ patches, to BlueField 5.15 kernel, we can get it working.

Reference
1. https://docs.nvidia.com/networking/display/BlueFieldDPUOSv392/UEFI%20Secure%20Boot
2. https://www.spinics.net/lists/arm-kernel/msg979554.html
3. https://mjg59.dreamwidth.org/50577.html