Comment 4 for bug 1806012

(as context to this information, apparently this particularly bad performance experienced with 'powersave' happens when the BIOS power control is set to the default, and goes away when in the BIOS you set power management to 'os control' - so there is some additional information needed to determine why this particular case offers bad performance, when as shown below, powersave/performance governors should not normally present more than a few percent performance difference)

I would have not expected the governor choice (powersave or otherwise) to limit performance so severely as to prevent a VM from booting/working usefully. I would expect the frequency governor settings to see make a difference in benchmarks and power usage, not general interactive performance. The phoronix data referred to later supports that view (the performance difference is minimal generally). The behavior you experienced is really a bug in my view.

On modern Intel CPUs (Sandy Bridge and newer, many 2011/2012+ models but varies depending on the exact CPU) the Intel "Pstate" driver is used which is significantly different to the older "cpufreq" driver. This is important to note as you have the two different drivers in use based on which CPU you have - rather than OS (Xenial/Bionic use the same settings).

Although both drivers have governor modes with the name "powersave" and "performance" they are similar in name only and their behavior is quite different and they do not share any code. To that end you may find different behavior between some kind of test/lab environment which is not unlikely to have much older hardware and current new hardware FCBs. It would also be good to know for this specific badly broken system which scaling_driver was in use and what the precise processor model information from /proc/cpuinfo.

This article from Phoronix was released recently which compares the performance with various different benchmarks as well as power-usage of the various driver and governor mode combinations (it's a good read separately)
https://www.phoronix.com/scan.php?page=article&item=linux50-pstate-cpufreq

It has a few interesting observations. In the majority of benchmarks the performance between the two is very similar, and in fact the p-state powersave governor is slightly faster (!) than the pstate performance governor in many of the tests by a small margin. Another major observation from the phoronix data is that the CPUfreq-powersave governor is VERY significantly slower, by a factor of 4-5 times in most cases.

While the *cpufreq*-powersave (which remember, is different to the intel_pstate-powersave governor, which should be used) governor is very slow, it should also not be used by default on any Xenial or Bionic system from what I can see unless I am missing another script/tool that is changing the governor somewhere (I couldn't see any scripts in the charm or qemu packages that do so). If we read the code of the systemd service on bionic to set the CPU scheduler, we find that the script /lib/systemd/set-cpufreq (which is an Ubuntu/Debian addition, not systemd upstream, xenial uses more or less the same script at /etc/init.d/ondemand) it is quite simple and will basically prefer "interactive", "ondemand" and "powersave" in that order if they exist. This should result in non-pstate systems using ondemand (correct) and pstate systems using powersave (also correct). So it seems the bug is most likely not in the script, but some strange interaction with the BIOS that needs to be investigated further.

To that end if anyone with an affected system with noticeably worse performance in powersave/ondemand, I'd love to either get access or see the following data:
 - Output of "sudo dmidecode"
 - A copy of /sys/devices/system/cpu/cpufreq (tar is fine) with particular attention to the values of scaling_driver and scaling_governor
 - A basic CPU benchmark, run under 'powertop' to collect information on the frequency and idle states. You can run that like so "sudo powertop -C test.csv -r test.html --workload=/usr/bin/updatedb" - it will output a CSV and HTML file with the data. But we probably need a better benchmark that updatedb and I'm not 100% sure off the top of my head what we could use - I suggest we could try a few things on a "broken" machine to figure out benchmark we can use that reflects the poor performance - updatedb may well do the job but it's not what I'd actually suggest we use.

Ideally we would collect this information in a matrix of all 4 combinations of: CPU Governor (Default Ubuntu, User Optimised) and BIOS setting (OS Control, BIOS Default) so we can understand why we get the pathologically bad performance in some cases of BIOS Default + powersave.