Fails to boot cirros QEMU image with tuned running

Bug #1774000 reported by Martin Pitt
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tuned (Debian)
New
Unknown
tuned (Ubuntu)
Incomplete
Undecided
Unassigned

Bug Description

A recent security update broke booting of some images, particularly CirrOS [3]:

$ wget https://download.cirros-cloud.net/0.3.5/cirros-0.3.5-i386-disk.img
$ qemu-system-x86_64 -enable-kvm -nographic cirros-0.3.5-i386-disk.img -snapshot
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2]

And then nothing happens at all any more, other than QEMU using 100% CPU. This also affects version 0.4.0 and x86_64, so it's not particularly sensitive to guest changes.

I'm testing this with (nested) KVM inside an Ubuntu 18.04 LTS VM.

When going back to an older image that was built 5 days ago [4], it works:

# qemu-system-x86_64 -enable-kvm -nographic cirros-0.3.5-i386-disk.img -snapshot
qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2]
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 3.2.0-80-virtual (buildd@komainu) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #116-Ubuntu SMP Mon Mar 23 17:48:17 UTC 2015 (Ubuntu 3.2.0-80.116-virtual 3.2.68)
[...]

This shows that the "ECX.svm" warning already happened before and seems to be unrelated.

The most recent security update of Qemu 2.11+dfsg-1ubuntu7.2 [1] and Linux 4.15.0-22.24 [2] are already on that previous image, so it seems this is some dependency update in between these stacks. I. e. this is *not* a regression in QEMU itself or Linux. I file it against qemu for now, but I will bisect which particular update was responsible.

[1] https://launchpad.net/ubuntu/+source/qemu/1:2.11+dfsg-1ubuntu7.2
[2] https://launchpad.net/ubuntu/+source/linux/4.15.0-22.24
[3] https://download.cirros-cloud.net/
[4] https://github.com/cockpit-project/cockpit/pull/9221

Revision history for this message
Martin Pitt (pitti) wrote :

Found it. This happens as soon as you install tuned, which auto-starts tuned.service. After "systemctl stop tuned", QEMU works again. Retitled accordingly, so this is not a regression.

summary: - Regression: Fails to boot cirros image
+ Fails to boot cirros QEMU image with tuned running
affects: qemu (Ubuntu) → tuned (Ubuntu)
Revision history for this message
Scott Moser (smoser) wrote :

@Martin,
Can you file this upstream with 'tuned' ? or is it not a problem with upstream?

Revision history for this message
Martin Pitt (pitti) wrote :

I tested this on Fedora 28, which also has tuned 2.9.0, but a slightly newer QEMU (2.11.1, as opposed to 2.11 on Ubuntu 18.04), and a newer kernel (4.16.10 instead of 4.15.0 in Ubuntu 18.04).

The invocation in the unit (/usr/bin/python -Es /usr/sbin/tuned -l -P) is exactly the same, so is the active profile ("virtual-guest"), profile_mode ("auto"), and the config file tuned-main.conf, which is bit by bit the same between the two.

However, this also affects Debian testing, which has Linux 4.16.5 and a newer QEMU (2.12). I filed a bug there as well now.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Hi Martin,
tuned is usually not installed, at least not on the new guest you are booting, so to be sure are we talking about this setup:

Host
  - KVM-Host
  - not so important, but what OS is this?
Guest1
  - KVM-Guest
  - runs an Ubuntu Cosmic Image
  - also KVM-Nested-Host
  - tuned runs here in virt-guest profile
Guest2
  - nested KVM-Guest
  - runs CirrOS image
  - Booting this guest hangs with the newer tuned

I assume tuned runs in the default config here.
So it spawns its service, has a dbus listener but none of the specials (like kernel commandline changes).
But if I not miss something the virt-guest profile should then by default only change vm.dirty_ratio and vm.swappiness - none of these should be super important.

I'm trying to recreate now with above assumptions, let me know if anything differs.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Setup as assumed above:
1. Host Bionic
2. Guest1 Cosmic
3. Guest2 Cosmic or CirrOS

Guest1 is at:
linux-image-virtual 4.15.0.20.23
libvirt-daemon-system 4.0.0-1ubuntu11
qemu-system-x86 1:2.11+dfsg-1ubuntu10

Test 1 - no tuned on Guest1, to know we are good in the general setup
 ok - libvirt/uvtool guest
 ok - CirrOS guest directly in qemu-system

Test 2 - install tuned in Guest1 - log at /var/log/tuned/tuned.log
 - I see base and cpu pluging initializing
 - re-applying systemctl
 - apply tuning of virtual-guest config
   I compared sysctl changes and found kernel.sched_min_granularity_ns and kernel.sched_wakeup_granularity_ns bumped up by ~4-5, but that should not break anything.
   Also I found the expected dirty/swappiness ratio - also should not break anything

Test 3 - run guests with tuned installed
 ok - libvirt/uvtool guest
 ok - CirrOS guest directly in qemu-system

Ok, I'm not able to reproduce yet, so I think my assumptions on your setup were not correct.
Let me know what is different please.

Changed in tuned (Ubuntu):
status: New → Incomplete
Changed in tuned (Debian):
status: Unknown → New
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.