ubuntu-kernel-tests

vmx_apicv_test in ubuntu_kvm_unit_tests failed on B-KVM (FAIL: x2apic - reading 0x090: x2APIC op triggered GP.)

Bug #1827866 reported by Po-Hsu Lin on 2019-05-06

This bug affects 2 people

	Status	Importance	Assigned to
ubuntu-kernel-tests	Triaged	Undecided	Unassigned
linux-kvm (Ubuntu)	Confirmed	Undecided	Unassigned
Bionic	Confirmed	Undecided	Unassigned
linux-oracle (Ubuntu)	Confirmed	Undecided	Unassigned
Xenial	Confirmed	Undecided	Unassigned
Bionic	Confirmed	Undecided	Unassigned

Bug Description

vmx_apicv_test from ubuntu_kvm_unit_tests failed on B-KVM:

  FAIL: x2apic - reading 0x090: x2APIC op triggered GP.
  FAIL: x2apic - reading 0x090: x2APIC op triggered GP.
  FAIL: x2apic - reading 0x090: x2APIC op triggered GP.
  FAIL: x2apic - reading 0x090: x2APIC op triggered GP.
  FAIL vmx_apicv_test (9239 tests, 4 unexpected failures)

Complete test log: https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/1827866/+attachment/5521805/+files/vmx_apicv_test-b-kvm.log

See original description

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2019-05-06:

Dependencies.txt Edit (2.8 KiB, text/plain; charset="utf-8")
ProcCpuinfoMinimal.txt Edit (725 bytes, text/plain; charset="utf-8")
ProcEnviron.txt Edit (288 bytes, text/plain; charset="utf-8")

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2019-05-06:

vmx_apicv_test timeouted on B-KVM:
FAIL vmx_apicv_test (timeout; duration=10)

description:

updated

Po-Hsu Lin (cypressyew) on 2019-09-16

tags:

added: ubuntu-kvm-unit-tests

Po-Hsu Lin (cypressyew) on 2019-10-17

tags:	added: 4.15 4.4
tags:	added: sru-20190930
summary:	- vmx_apicv_test in ubuntu_kvm_unit_tests failed on X-KVM + vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2019-10-18: Re: vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM

Can be found on X-Azure 4.15 as well:
Running '/home/azure/autotest/client/tmp/ubuntu_kvm_unit_tests/src/kvm-unit-tests/tests/vmx_apicv_test'
BUILD_HEAD=e2c275c4
timeout -k 1s --foreground 10 /usr/bin/qemu-system-x86_64 -nodefaults -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -machine accel=kvm -kernel /tmp/tmp.O2MSEezMNL -smp 1 -cpu host,+vmx -append apic_reg_virt_test virt_x2apic_mode_test # -initrd /tmp/tmp.nFaa5lDEut
enabling apic
paging enabled
cr0 = 80010011
cr3 = 477000
cr4 = 20

Test suite: apic_reg_virt_test
qemu-system-x86_64: terminating on signal 15 from pid 21120
FAIL vmx_apicv_test (timeout; duration=10)
qemu:qemu_cpu_kick_thread: No such process

tags:	added: azure
summary:	- vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM + vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM X-azure

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2019-11-01: Re: vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM X-azure

Found on B-aws amd64 4.15

tags:

added: sru-20191021

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2019-11-07:

Failed on Oracle B-4.15
Passed on Oracle B-5.0

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2019-11-08:

Failed on B-AWS-5.3 bare-metal instance i3.metal

PASS: x2apic - writing 0x78 to 0x080: x2APIC write; got 0x78, want 0x78
Unhandled exception 13 #GP at ip 00000000004071a3
error_code=0000 rflags=00010006 cs=00000008
rax=0000000000000078 rcx=0000000000000809 rdx=0000000000000000 rbx=0000000000000009
rbp=000000000047efdf rsi=000000000042568d rdi=000000000042568d
r8=000000000000000a r9=00000000000003f8 r10=000000000000000d r11=0000000000000000
r12=0000000000000000 r13=0000000000000000 r14=0000000000403dcc r15=0000000000000000
cr0=0000000080010031 cr2=0000000000000000 cr3=0000000000477000 cr4=0000000000002020
cr8=0000000000000007
STACK: @4071a3 40170b 4004dd
FAIL vmx_apicv_test

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2019-12-02:

Passed with B-KVM 4.15.0-1051.51 this cycle.

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2019-12-11:

This failure can be found on Eoan bare metal node fili (5.3.0-25.27)

Unhandled exception 13 #GP at ip 00000000004071bf
error_code=0000 rflags=00010006 cs=00000008
rax=0000000000000078 rcx=0000000000000809 rdx=0000000000000000 rbx=0000000000000009
rbp=000000000047efdf rsi=000000000000000a rdi=00000000000003fd
r8=000000000042568d r9=00000000000003f8 r10=000000000000000d r11=0000000000000000
r12=0000000000000000 r13=0000000000000000 r14=0000000000000000 r15=0000000000403d36
cr0=0000000080010031 cr2=0000000000000000 cr3=0000000000477000 cr4=0000000000002020
cr8=0000000000000007
STACK: @4071bf 40173d 4004dd
FAIL vmx_apicv_test

tags:

added: eoan sru-20191202

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2020-01-22:

This issue can be found on AWS Trusty 4.4 as well (4.4.0-1061.65-aws):

timeout -k 1s --foreground 10 /usr/bin/qemu-system-x86_64 -nodefaults -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio -device pci-testdev -machine accel=kvm -kernel /tmp/tmp.QWPJWlQ7uF -smp 1 -cpu host,+vmx -append apic_reg_virt_test virt_x2apic_mode_test # -initrd /tmp/tmp.OCXCEUzFNz
enabling apic
paging enabled
cr0 = 80010011
cr3 = 477000
cr4 = 20

Test suite: apic_reg_virt_test
--- Virtualize APIC accesses test ---
FAIL: xapic - reading 0x000: x86/vmx_tests.c:2184: Assertion failed: (expected) == (actual)
LHS: 0x000000000000002c - 0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0010'1100 - 44
RHS: 000000000000000000 - 0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000 - 0
Expected VMX_APIC_ACCESS, got VMX_EXC_NMI.
STACK: 405e1c 409a1e 401556 4039f1 400312

Test suite: virt_x2apic_mode_test
--- Baseline test ---
FAIL: xapic - reading 0x000: x2apic - reading 0x000: x86/vmx_tests.c:2184: Assertion failed: (expected) == (actual)
LHS: 0x0000000000000012 - 0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0001'0010 - 18
RHS: 000000000000000000 - 0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000'0000 - 0
Expected VMX_VMCALL, got VMX_EXC_NMI.
STACK: 405e1c 405e46 40c979 401556 4039f1 400312
SUMMARY: 8 tests, 2 unexpected failures
FAIL vmx_apicv_test (8 tests, 2 unexpected failures)

tags:

added: aws trusty

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2020-01-22:

#10

BTW for AWS T-4.4, this is only failing on:
* r5.metal
* c5.metal

But passed with i3.metal

Kelsey Steele (kelsey-steele) on 2020-10-09

tags:

added: 5.4 bionic oracle sru-20200921

Revision history for this message

Kelsey Steele (kelsey-steele) wrote on 2020-10-09:

#11

Failed on Bionic/oracle : 5.4.0-1026.26~18.04.1 : amd64

1225. 09/24 08:14:26 DEBUG| utils:0153| [stdout] Unhandled exception 13 #GP at ip 00000000004071a3
1226. 09/24 08:14:26 DEBUG| utils:0153| [stdout] error_code=0000 rflags=00010006 cs=00000008
1227. 09/24 08:14:26 DEBUG| utils:0153| [stdout] rax=0000000000000078 rcx=0000000000000809 rdx=0000000000000000 rbx=0000000000000009
1228. 09/24 08:14:26 DEBUG| utils:0153| [stdout] rbp=000000000047efdf rsi=000000000042568d rdi=000000000042568d
1229. 09/24 08:14:26 DEBUG| utils:0153| [stdout] r8=000000000000000a r9=00000000000003f8 r10=000000000000000d r11=0000000000000000
1230. 09/24 08:14:26 DEBUG| utils:0153| [stdout] r12=0000000000000000 r13=0000000000000000 r14=0000000000403dcc r15=0000000000000000
1231. 09/24 08:14:26 DEBUG| utils:0153| [stdout] cr0=0000000080010031 cr2=0000000000000000 cr3=0000000000477000 cr4=0000000000002020
1232. 09/24 08:14:26 DEBUG| utils:0153| [stdout] cr8=0000000000000007
1233. 09/24 08:14:26 DEBUG| utils:0153| [stdout] STACK: @4071a3 40170b 4004dd
1234. 09/24 08:14:26 DEBUG| utils:0153| [stdout] FAIL vmx_apicv_test

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2020-11-17: Re: vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM X-azure / X-aws

#12

Still visible in Xenial 4.4.0-1118.132 AWS c5.metal

tags:	added: sru-20201109
summary:	- vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM X-azure + vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM X-azure / + X-aws

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2021-01-22:

#13

Found on B-aws-5.4
Unhandled exception 13 #GP at ip 00000000004071a3
error_code=0000 rflags=00010006 cs=00000008
rax=0000000000000078 rcx=0000000000000809 rdx=0000000000000000 rbx=0000000000000009
rbp=000000000047efdf rsi=000000000042568d rdi=000000000042568d
r8=000000000000000a r9=00000000000003f8 r10=000000000000000d r11=0000000000000000
r12=0000000000000000 r13=0000000000000000 r14=0000000000403dcc r15=0000000000000000
cr0=0000000080010031 cr2=0000000000000000 cr3=0000000000477000 cr4=0000000000002020
cr8=0000000000000007
STACK: @4071a3 40170b 4004dd
FAIL vmx_apicv_test

tags:

added: sru-20210104

Revision history for this message

Kelsey Steele (kelsey-steele) wrote on 2021-03-11:

#14

Also found on Bionic/hwe-5.4 5.4.0-67.75~18.04.1 host rumford, passed on Grumman

tags:

added: hwe-5.4 sru-20210222

Kelsey Steele (kelsey-steele) on 2021-04-10

tags:

added: hwe sru-20210315

Kleber Sacilotto de Souza (kleber-souza) on 2021-05-04

tags:

added: sru-20210412

Revision history for this message

Ian May (ian-may) wrote on 2021-05-07:

#15

bionic/linux-oracle: 4.15.0-1071.79

Revision history for this message

Ian May (ian-may) wrote on 2021-05-07:

#16

bionic/linux-oracle-5.4: 5.4.0-1044.47~18.04.1

Revision history for this message

Ian May (ian-may) wrote on 2021-05-27:

#17

Found on bionic/linux-aws-5.4: 5.4.0-1049.51~18.04.1

tags:

added: sru-20210510

Revision history for this message

Guilherme G. Piccoli (gpiccoli) wrote on 2021-06-18:

#18

Observed in B/KVM, cycle sru-20210531.

tags:

added: kvm linux-kvm sru-20210531

Revision history for this message

Guilherme G. Piccoli (gpiccoli) wrote on 2021-06-21:

#19

Observed in B-5.4/aws, cycle sru-20210531 .

Krzysztof Kozlowski (krzk) on 2021-07-01

tags:

added: sru-20210621

Sean Feole (sfeole) on 2021-07-12

Changed in ubuntu-kernel-tests:
status:	New → Triaged
tags:	added: hinted

Revision history for this message

Guilherme G. Piccoli (gpiccoli) wrote on 2021-07-15:

#20

Observed in B-5.4/aws, cycle sru-20210621 .

Revision history for this message

Guilherme G. Piccoli (gpiccoli) wrote on 2021-07-15:

#21

Observed this on B/oracle-5.4, cycle sru-20210621 .

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2021-08-27: Re: vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM / X-aws

#22

Didn't see this anymore on X-azure with updated "hirsute" branch in kvm-unit-tests, remove tags and modify bug title.

summary:	- vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM X-azure / - X-aws + vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM / X-aws
tags:	removed: azure
summary:	- vmx_apicv_test in ubuntu_kvm_unit_tests failed on X/B-KVM / X-aws + vmx_apicv_test in ubuntu_kvm_unit_tests failed on B-KVM / X-aws

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2021-08-27: Re: vmx_apicv_test in ubuntu_kvm_unit_tests failed on B-KVM / X-aws

#23

We don't have X-kvm anymore, modify bug title.

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2021-08-27:

#24

This is still visible on B-kvm 4.15.0-1099.101, with different failure. Will update the bug description.

tags:	removed: 4.4
tags:	added: sru
tags:	added: sru-20210816 removed: sru
no longer affects:	linux-azure (Ubuntu)
no longer affects:	linux-azure (Ubuntu Xenial)
no longer affects:	linux-azure (Ubuntu Bionic)
no longer affects:	linux-kvm (Ubuntu Xenial)

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2021-08-31:

#25

vmx_apicv_test-b-kvm.log Edit (139.2 KiB, text/plain)

description:	updated
summary:	- vmx_apicv_test in ubuntu_kvm_unit_tests failed on B-KVM / X-aws + vmx_apicv_test in ubuntu_kvm_unit_tests failed on B-KVM (FAIL: x2apic - + reading 0x090: x2APIC op triggered GP.)
description:	updated

Revision history for this message

Po-Hsu Lin (cypressyew) wrote on 2022-10-06:

#26

With kvm-unit-test repo updated to d8a4f9e5 (ci: Update the list of tests that we run in the Fedora Cirrus-CI), this test is still failing with the same error message.

tags:

added: sru-20220919

Revision history for this message

Launchpad Janitor (janitor) wrote on 2022-10-06:

#27

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in linux-kvm (Ubuntu Bionic):
status:	New → Confirmed
Changed in linux-kvm (Ubuntu):
status:	New → Confirmed
Changed in linux-oracle (Ubuntu Bionic):
status:	New → Confirmed
Changed in linux-oracle (Ubuntu Xenial):
status:	New → Confirmed
Changed in linux-oracle (Ubuntu):
status:	New → Confirmed