Encountering an issue with memcpy_fromio causing failed boot of SEV-enabled guest
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
In Progress
|
Undecided
|
Chengen Du | ||
Bionic |
New
|
Undecided
|
Unassigned | ||
Focal |
Fix Released
|
Medium
|
Chengen Du | ||
Jammy |
Fix Released
|
Medium
|
Chengen Du | ||
linux-gcp (Ubuntu) |
Invalid
|
Undecided
|
Unassigned | ||
Bionic |
New
|
Undecided
|
Unassigned | ||
Focal |
Fix Released
|
Undecided
|
Unassigned | ||
Jammy |
Fix Released
|
Medium
|
Thadeu Lima de Souza Cascardo |
Bug Description
[Impact]
When launching a SEV-enabled guest, the guest kernel panics with the following call trace,
indicating a critical error in the system.
==========
[ 1.090638] software IO TLB: Memory encryption is active and system is using DMA bounce buffers
[ 1.092105] Linux agpgart interface v0.103
[ 1.092716] BUG: unable to handle page fault for address: ffff9b820003d068
[ 1.093445] #PF: supervisor read access in kernel mode
[ 1.093966] #PF: error_code(0x0000) - not-present page
[ 1.094481] PGD 800100000067 P4D 800100000067 PUD 8001001d7067 PMD 8001001da067 PTE 80000000fed40173
[ 1.094629] Oops: 0000 [#1] SMP NOPTI
[ 1.094629] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.15.0-46-generic #49-Ubuntu
[ 1.094629] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
[ 1.094629] RIP: 0010:memcpy_
[ 1.094629] Code: cc cc cc 0f 1f 44 00 00 55 48 89 e5 48 85 d2 74 28 40 f6 c6 01 75 30 48 83 fa 01 76 06 40 f6 c6 02 75 1c 48 89 d1 48 c1 e9 02 <f3> a5 f6 c2 02 74 02 66 a5 f6 c2 01 74 01 a4 5d e9 14 b3 97 00 66
[ 1.094629] RSP: 0018:ffff9b8200
[ 1.094629] RAX: ffff9b820003d040 RBX: ffff9b820001bac0 RCX: 0000000000000002
[ 1.094629] RDX: 0000000000000008 RSI: ffff9b820003d068 RDI: ffff9b820001ba90
[ 1.094629] RBP: ffff9b820001ba50 R08: 0000000000000f80 R09: 0000000000000f80
[ 1.094629] R10: 00000000fed40080 R11: ffff9b820001bac0 R12: ffff8cc7068eca48
[ 1.094629] R13: ffff8cc700a64288 R14: 0000000000000000 R15: 00000000fed40080
[ 1.094629] FS: 000000000000000
[ 1.094629] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1.094629] CR2: ffff9b820003d068 CR3: 0000800174a10000 CR4: 0000000000350ee0
[ 1.094629] Call Trace:
[ 1.094629] <TASK>
[ 1.094629] crb_map_
[ 1.094629] ? radix_tree_
[ 1.094629] ? _raw_spin_
[ 1.094629] crb_acpi_
[ 1.094629] acpi_device_
[ 1.094629] really_
[ 1.094629] __driver_
[ 1.094629] driver_
[ 1.094629] __driver_
[ 1.094629] ? __device_
[ 1.094629] bus_for_
[ 1.094629] driver_
[ 1.094629] bus_add_
[ 1.094629] driver_
[ 1.094629] ? init_tis+0xfd/0xfd
[ 1.094629] acpi_bus_
[ 1.094629] crb_acpi_
[ 1.094629] do_one_
[ 1.094629] do_initcalls+
[ 1.094629] kernel_
[ 1.094629] ? rest_init+
[ 1.094629] kernel_
[ 1.094629] ? rest_init+
[ 1.094629] ret_from_
[ 1.094629] </TASK>
[ 1.094629] Modules linked in:
[ 1.094629] CR2: ffff9b820003d068
[ 1.094629] ---[ end trace 3d6d81c42a3c2030 ]---
[ 1.094629] RIP: 0010:memcpy_
[ 1.094629] Code: cc cc cc 0f 1f 44 00 00 55 48 89 e5 48 85 d2 74 28 40 f6 c6 01 75 30 48 83 fa 01 76 06 40 f6 c6 02 75 1c 48 89 d1 48 c1 e9 02 <f3> a5 f6 c2 02 74 02 66 a5 f6 c2 01 74 01 a4 5d e9 14 b3 97 00 66
[ 1.094629] RSP: 0018:ffff9b8200
[ 1.094629] RAX: ffff9b820003d040 RBX: ffff9b820001bac0 RCX: 0000000000000002
[ 1.094629] RDX: 0000000000000008 RSI: ffff9b820003d068 RDI: ffff9b820001ba90
[ 1.094629] RBP: ffff9b820001ba50 R08: 0000000000000f80 R09: 0000000000000f80
[ 1.094629] R10: 00000000fed40080 R11: ffff9b820001bac0 R12: ffff8cc7068eca48
[ 1.094629] R13: ffff8cc700a64288 R14: 0000000000000000 R15: 00000000fed40080
[ 1.094629] FS: 000000000000000
[ 1.094629] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1.094629] CR2: ffff9b820003d068 CR3: 0000800174a10000 CR4: 0000000000350ee0
[ 1.094629] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
[ 1.094629] Kernel Offset: 0x200000 from 0xffffffff81000000 (relocation range: 0xffffffff80000
[ 1.094629] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009 ]---
==========
[Fix]
The issue arises when launching kernels in libvirt-managed SEV virtual machines due to the addition of a tpm-crb device by virt-install.
Upstream commit 4009a4ac82dd has fixed this issue.
==========
Author: Joerg Roedel <email address hidden>
Date: Mon Mar 21 10:33:51 2022 +0100
x86/sev: Unroll string mmio with CC_ATTR_
The io-specific memcpy/memset functions use string mmio accesses to do
their work. Under SEV, the hypervisor can't emulate these instructions
because they read/write directly from/to encrypted memory.
==========
[Test Plan]
1. Use virt-install to create a SEV-enabled guest
virt-install --name sev_guest --memory 16384 --vcpus 16 --boot uefi --disk /root/focal-
2. Poweron the guest and kernel panic occurred.
[Where problems could occur]
SEV doesn't support string I/O, so the patch unrolls the string I/O operation into a loop operating on one element at a time.
The affected range is limited to virtual machines and specific platforms that support SEV (i.e., AMD Epyc) and have SEV-ES not enabled.
Changed in linux (Ubuntu): | |
assignee: | nobody → ChengEn, Du (chengendu) |
Changed in linux (Ubuntu Focal): | |
status: | New → In Progress |
Changed in linux (Ubuntu Jammy): | |
status: | New → In Progress |
Changed in linux (Ubuntu Focal): | |
assignee: | nobody → ChengEn, Du (chengendu) |
Changed in linux (Ubuntu Jammy): | |
assignee: | nobody → ChengEn, Du (chengendu) |
Changed in linux (Ubuntu): | |
status: | Incomplete → In Progress |
Changed in linux (Ubuntu Jammy): | |
importance: | Undecided → Medium |
Changed in linux (Ubuntu Focal): | |
importance: | Undecided → Medium |
Changed in linux (Ubuntu Jammy): | |
status: | In Progress → Fix Committed |
Changed in linux (Ubuntu Focal): | |
status: | In Progress → Fix Committed |
Changed in linux-gcp (Ubuntu): | |
status: | New → Invalid |
Changed in linux-gcp (Ubuntu Jammy): | |
status: | New → In Progress |
importance: | Undecided → Medium |
assignee: | nobody → Thadeu Lima de Souza Cascardo (cascardo) |
Changed in linux-gcp (Ubuntu Jammy): | |
status: | In Progress → Fix Committed |
tags: |
added: verification-done-focal verification-done-jammy removed: verification-needed-focal verification-needed-jammy |
This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:
apport-collect 2020319
and then change the status of the bug to 'Confirmed'.
If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.
This change has been made by an automated script, maintained by the Ubuntu Kernel Team.