nested KVM fails - KVM: entry failed, hardware error 0x0

arges wrote a script in bug #1329434 which might be enough to repro this, but I don't have access to the hw myself to test

Hmm, that's a non current kernel - I wonder if someone can test with the current lts-xenial?

Status changed to 'Confirmed' because the bug affects multiple users.

We hit this late last week too, I'm having difficulty reproducing it though. smb thinks there could be some issue with checks failing for feature bits not passed into the guest because they were new and thus perhaps a problem when using pretty recent hardware.

For me it's enough to boot an instance on an affected lgw01 compute node, install qemu-system-x86, download a cloud image and run "/usr/bin/qemu-system-x86_64 -smp 1 -net none -m 512M -nographic -drive format=raw,cache=unsafe,file=zesty-server-cloudimg-amd64.img -machine accel=kvm -enable-kvm -cpu host"

Adding some miscellaneous data points.

I'm still unable to reproduce this outside of a lgw01 compute node that Laney set up for me, running xenial with qemu 1:2.5+dfsg-5ubuntu10.10 and linux 4.4.0-72. It doesn't happen on every host. I'm still experimenting to see if I can reproduce this on some hardware I control.

The failures happen before the L2 kernel starts, so the problem must be in L0 or L1. I've experimented with using older versions of qemu (1:2.5+dfsg-5ubuntu10.6) and the kernel (4.4.0-62.83) in L1 but still see the issues with those.

Stefan Bader and I have continued trying to reproduce this. Here's what we've done:

- Tried to reproduce on various pieces of hardware available to us.
- Requested the hardware information about the host machine from IS and tested on the most similar machine available to us (same generation Xeon CPU, but a different model).
- Requested information about relevant software versions on the host machine (trusty with a 4.4.0-59 hwe kernel) and tried to reproduce using the same software versions.
- Requested the libvirt configuration for the instance which exhibits the failures and tried using a similar configuration.

So far we are still unable to reproduce the problem.

Maybe IS happens to have a box that's not provisioned yet and you can borrow? Or you can borrow that one for a bit if lgw01 has some spare capacity? Or they'll upgrade to the current lts-xenial kernel so we maybe get lucky. :)

Facing the same issue with KVM enabled L2 VM on following setup

CPU Model - Skylake-Client

L0:
Ubuntu 16.04.3 LTS
Kernel - 4.4.0-98-generic & 4.4.0-101-generic

libvirtd (libvirt) 2.5.0-3ubuntu5.5~cloud0
QEMU emulator version 2.8.0(Debian 1:2.8+dfsg-3ubuntu2.5~cloud0)

L1:
Ubuntu 16.04.3 LTS
Kernel - 4.4.0-98-generic

libvirtd (libvirt) 1.3.1-1ubuntu10.15
QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.16)

Able to launch L2 VM in qemu mode.

With same L0,L1 configuration able to launch L2 VM even with KVM on SandyBridge Machine

The last time something like this was fixed was https://bugs.launchpad.net/bugs/1329434. The fix was to disable APIC virtualization in nested guests. Did it suddenly get turned on again? Any patches we could test? Thanks.

Changing the script to be used with xenial instead of trusty: https://gist.github.com/kmindi/4fb1332509a39b6214b0e8b39611f5ad

I can also confirm the bug on the following hardware configuration:

CPU Model - Skylkake (Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz)

L0 and L1:
Ubuntu 16.04.3 LTS
Kernel - 4.4.0-104-generic
libvirtd (libvirt) 1.3.1 (libvirt-bin 1.3.1-1ubuntu10)
QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.16)

Running into the same as well.
Is there any workaround on this?

I am sorry if this bug is to old, but i have this problem and cannot find the solution until now.

I can reproduce this bug with following HW:

Ubuntu 16.04.5 LTS
Kernel 4.4.0-131-generic
libvirtd (libvirt) 1.3.1

QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.32)

Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz

But i have different machine which use Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz with same package version but have no problem.

Does this bug already fixed?

Reproduced in Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz

Ubuntu 16.04.5 LTS
4.4.0-134-generic

libvirt 4.0.0
QEMU 4.0.0
Running hypervisor: QEMU 2.11.1

Nested KVM not working, instances log "KVM: entry failed, hardware error 0x0"

This happens in about half of the xenial semaphoreci.com instances as well:

$ uname -a
Linux semaphore-light-1809b 4.4.0-131-generic #157-Ubuntu SMP Thu Jul 12 15:51:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux