Cpu utilization showing system time for kvm guests (performance) (sysstat)

Bug #1755979 reported by bugproxy
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Fix Released
High
Canonical Kernel Team
linux (Ubuntu)
Fix Released
High
Joseph Salisbury
Artful
Fix Released
High
Joseph Salisbury
Bionic
Fix Released
High
Joseph Salisbury

Bug Description

== SRU Justification ==
All KVM guest cpu utilization is being reported as sys% time in the host.
It turns out Guest time accounting (CONFIG_VIRT_CPU_ACCOUNTING_GEN) has
been broken since v4.13 by commit 8b24e69fc47e436.

Commit 61bd0f66ff92 fixes this bug and is included in mainline as of v4.16-rc5.
The commit was cc'd to upstream stable, but has not yet landed in upstream 4.15.

== Fix ==
61bd0f66ff92 ("KVM: PPC: Book3S HV: Fix guest time accounting with VIRT_CPU_ACCOUNTING_GEN")

== Regression Potential ==
Low. Also cc'd to stable, so it has had additionl upstream review. Limited to powerpc.

== Test Case ==
A test kernel was built with this patch and tested by the original bug reporter.
The bug reporter states the test kernel resolved the bug.

== Comment: #0 - JENIFER HOPPER <email address hidden> - 2018-03-14 16:20:47 ==
+++ This bug was initially created as a clone of Bug #163375 +++

---Problem Description---
All KVM guest cpu utilization is being reported as sys% time in the host.
It turns out Guest time accounting (CONFIG_VIRT_CPU_ACCOUNTING_GEN) has been broken since v4.13 by commit:

commit 8b24e69fc47e43679bb29ddb481aa0e8dce2a3c5
Author: Paul Mackerras <email address hidden>
Date: Mon Jun 26 15:45:51 2017 +1000

KVM: PPC: Book3S HV: Close race with testing for signals on guest entry

---

That is now fixed and upstream accepted in Paulus' tree as git commit:

https://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc.git/commit/?h=kvm-ppc-fixes&id=61bd0f66ff92d5ce765ff9850fd3cbfec773c560
("KVM: PPC: Book3S HV: Fix guest time accounting with VIRT_CPU_ACCOUNTING_GEN")

Please include this patch in Ubuntu so that guest cpu accounting is correct.

Contact Information = <email address hidden>

---uname output---
Linux ltc-b114 4.15.0-10-generic #11-Ubuntu SMP Tue Feb 13 18:21:52 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = 5104-22C

---Debugger---
A debugger is not configured

---Steps to Reproduce---
 Start a KVM guest, run something that generates cpu utilization in the guest, observe mpstat on the host.

CVE References

bugproxy (bugproxy)
tags: added: architecture-ppc64le bugnameltc-165731 severity-high targetmilestone-inin---
Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → kernel-package (Ubuntu)
Frank Heimes (fheimes)
Changed in ubuntu-power-systems:
importance: Undecided → High
affects: kernel-package (Ubuntu) → linux (Ubuntu)
Changed in ubuntu-power-systems:
assignee: nobody → Canonical Kernel Team (canonical-kernel-team)
status: New → Triaged
tags: added: triage-g
Changed in linux (Ubuntu):
status: New → Triaged
importance: Undecided → High
Changed in linux (Ubuntu Artful):
status: New → In Progress
Changed in linux (Ubuntu Bionic):
status: Triaged → In Progress
Changed in linux (Ubuntu Artful):
importance: Undecided → High
assignee: nobody → Joseph Salisbury (jsalisbury)
Changed in linux (Ubuntu Bionic):
assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) → Joseph Salisbury (jsalisbury)
Revision history for this message
Joseph Salisbury (jsalisbury) wrote :

I built a test kernel with commit 61bd0f66ff92. The test kernel can be downloaded from:
http://kernel.ubuntu.com/~jsalisbury/lp1755979

Can you test this kernel and see if it resolves this bug?

Note, to test this kernel, you need to install both the linux-image and linux-image-extra .deb packages.

Thanks in advance!

Frank Heimes (fheimes)
Changed in ubuntu-power-systems:
status: Triaged → In Progress
Revision history for this message
bugproxy (bugproxy) wrote : Comment bridged from LTC Bugzilla

------- Comment From <email address hidden> 2018-03-21 16:39 EDT-------
With this test kernel, I now see guest time reported correctly in the host:

03:32:17 PM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
03:32:19 PM all 0.00 0.00 0.91 0.00 0.00 0.02 0.00 64.58 0.00 34.48

4.15.0-12-generic #13~lp175597

Revision history for this message
Joseph Salisbury (jsalisbury) wrote :
description: updated
Changed in linux (Ubuntu Bionic):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Artful):
status: In Progress → Fix Committed
Frank Heimes (fheimes)
Changed in ubuntu-power-systems:
status: In Progress → Fix Committed
Revision history for this message
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-artful' to 'verification-done-artful'. If the problem still exists, change the tag 'verification-needed-artful' to 'verification-failed-artful'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-artful
Revision history for this message
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2018-04-11 12:01 EDT-------
I verified the -proposed kernel fixes the issue:

10:59:35 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
10:59:37 AM all 0.01 0.00 0.08 0.00 0.00 0.01 0.00 20.68 0.00 79.21
10:59:37 AM 0 0.53 0.00 0.00 0.00 0.00 0.00 0.00 51.34 0.00 48.13
10:59:37 AM 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 76.00 0.00 24.00

Selected version '4.13.0.39.42' (Ubuntu:17.10/artful-proposed [ppc64el]) for 'linux-image-generic'

tags: added: verification-done-artful
removed: verification-needed-artful
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (40.4 KiB)

This bug was fixed in the package linux - 4.15.0-15.16

---------------
linux (4.15.0-15.16) bionic; urgency=medium

  * linux: 4.15.0-15.16 -proposed tracker (LP: #1761177)

  * FFe: Enable configuring resume offset via sysfs (LP: #1760106)
    - PM / hibernate: Make passing hibernate offsets more friendly

  * /dev/bcache/by-uuid links not created after reboot (LP: #1729145)
    - SAUCE: (no-up) bcache: decouple emitting a cached_dev CHANGE uevent

  * Ubuntu18.04:POWER9:DD2.2 - Unable to start a KVM guest with default machine
    type(pseries-bionic) complaining "KVM implementation does not support
    Transactional Memory, try cap-htm=off" (kvm) (LP: #1752026)
    - powerpc: Use feature bit for RTC presence rather than timebase presence
    - powerpc: Book E: Remove unused CPU_FTR_L2CSR bit
    - powerpc: Free up CPU feature bits on 64-bit machines
    - powerpc: Add CPU feature bits for TM bug workarounds on POWER9 v2.2
    - powerpc/powernv: Provide a way to force a core into SMT4 mode
    - KVM: PPC: Book3S HV: Work around transactional memory bugs in POWER9
    - KVM: PPC: Book3S HV: Work around XER[SO] bug in fake suspend mode
    - KVM: PPC: Book3S HV: Work around TEXASR bug in fake suspend state

  * Important Kernel fixes to be backported for Power9 (kvm) (LP: #1758910)
    - powerpc/mm: Fixup tlbie vs store ordering issue on POWER9

  * Ubuntu 18.04 - IO Hang on some namespaces when running HTX with 16
    namespaces (Bolt / NVMe) (LP: #1757497)
    - powerpc/64s: Fix lost pending interrupt due to race causing lost update to
      irq_happened

  * fwts-efi-runtime-dkms 18.03.00-0ubuntu1: fwts-efi-runtime-dkms kernel module
    failed to build (LP: #1760876)
    - [Packaging] include the retpoline extractor in the headers

linux (4.15.0-14.15) bionic; urgency=medium

  * linux: 4.15.0-14.15 -proposed tracker (LP: #1760678)

  * [Bionic] mlx4 ETH - mlnx_qos failed when set some TC to vendor
    (LP: #1758662)
    - net/mlx4_en: Change default QoS settings

  * AT_BASE_PLATFORM in AUXV is absent on kernels available on Ubuntu 17.10
    (LP: #1759312)
    - powerpc/64s: Fix NULL AT_BASE_PLATFORM when using DT CPU features

  * Bionic update to 4.15.15 stable release (LP: #1760585)
    - net: dsa: Fix dsa_is_user_port() test inversion
    - openvswitch: meter: fix the incorrect calculation of max delta_t
    - qed: Fix MPA unalign flow in case header is split across two packets.
    - tcp: purge write queue upon aborting the connection
    - qed: Fix non TCP packets should be dropped on iWARP ll2 connection
    - sysfs: symlink: export sysfs_create_link_nowarn()
    - net: phy: relax error checking when creating sysfs link netdev->phydev
    - devlink: Remove redundant free on error path
    - macvlan: filter out unsupported feature flags
    - net: ipv6: keep sk status consistent after datagram connect failure
    - ipv6: old_dport should be a __be16 in __ip6_datagram_connect()
    - ipv6: sr: fix NULL pointer dereference when setting encap source address
    - ipv6: sr: fix scheduling in RCU when creating seg6 lwtunnel state
    - mlxsw: spectrum_buffers: Set a minimum quota for CPU port traffic
    - net: phy: Tell caller result ...

Changed in linux (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (5.6 KiB)

This bug was fixed in the package linux - 4.13.0-39.44

---------------
linux (4.13.0-39.44) artful; urgency=medium

  * linux: 4.13.0-39.44 -proposed tracker (LP: #1761456)

  * intel-microcode 3.20180312.0 causes lockup at login screen(w/ linux-
    image-4.13.0-37-generic) (LP: #1759920) // CVE-2017-5715 (Spectre v2
    Intel) // CVE-2017-5754
    - x86/mm: Reinitialize TLB state on hotplug and resume

  * intel-microcode 3.20180312.0 causes lockup at login screen(w/ linux-
    image-4.13.0-37-generic) (LP: #1759920) // CVE-2017-5715 (Spectre v2 Intel)
    - Revert "x86/mm: Only set IBPB when the new thread cannot ptrace current
      thread"
    - x86/speculation: Use Indirect Branch Prediction Barrier in context switch

  * DKMS driver builds fail with: Cannot use CONFIG_STACK_VALIDATION=y, please
    install libelf-dev, libelf-devel or elfutils-libelf-devel (LP: #1760876)
    - [Packaging] include the retpoline extractor in the headers

  * retpoline hints: primary infrastructure and initial hints (LP: #1758856)
    - [Packaging] retpoline-extract: flag *0xNNN(%reg) branches
    - x86/speculation, objtool: Annotate indirect calls/jumps for objtool
    - x86/speculation, objtool: Annotate indirect calls/jumps for objtool on 32bit
    - x86/paravirt, objtool: Annotate indirect calls
    - [Packaging] retpoline -- add safe usage hint support
    - [Packaging] retpoline-check -- only report additions
    - [Packaging] retpoline -- widen indirect call/jmp detection
    - [Packaging] retpoline -- elide %rip relative indirections
    - [Packaging] retpoline -- clear hint information from packages
    - KVM: x86: Make indirect calls in emulator speculation safe
    - KVM: VMX: Make indirect call speculation safe
    - x86/boot, objtool: Annotate indirect jump in secondary_startup_64()
    - SAUCE: early/late -- annotate indirect calls in early/late initialisation
      code
    - SAUCE: vga_set_mode -- avoid jump tables
    - [Config] retpoline -- switch to new format
    - [Packaging] retpoline hints -- handle missing files when RETPOLINE not
      enabled
    - [Packaging] final-checks -- remove check for empty retpoline files

  * retpoline: ignore %cs:0xNNN constant indirections (LP: #1752655)
    - [Packaging] retpoline -- elide %cs:0xNNNN constants on i386

  * zfs system process hung on container stop/delete (LP: #1754584)
    - SAUCE: Fix non-prefaulted page deadlock (LP: #1754584)

  * zfs-linux 0.6.5.11-1ubuntu5 ADT test failure with linux 4.15.0-1.2
    (LP: #1737761)
    - SAUCE: (noup) Update zfs to 0.6.5.11-1ubuntu3.2

  * AT_BASE_PLATFORM in AUXV is absent on kernels available on Ubuntu 17.10
    (LP: #1759312)
    - powerpc/64s: Fix NULL AT_BASE_PLATFORM when using DT CPU features

  * btrfs and tar sparse truncate archives (LP: #1757565)
    - Btrfs: move definition of the function btrfs_find_new_delalloc_bytes
    - Btrfs: fix reported number of inode blocks after buffered append writes

  * efifb broken on ThunderX-based Gigabyte nodes (LP: #1758375)
    - drivers/fbdev/efifb: Allow BAR to be moved instead of claiming it

  * Intel i40e PF reset due to incorrect MDD detection (continues...)
    (LP: #1723127)
    - i40e/i40ev...

Read more...

Changed in linux (Ubuntu Artful):
status: Fix Committed → Fix Released
Manoj Iyer (manjo)
Changed in ubuntu-power-systems:
status: Fix Committed → Fix Released
bugproxy (bugproxy)
tags: added: targetmilestone-inin1804
removed: targetmilestone-inin---
Brad Figg (brad-figg)
tags: added: cscc
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.