Pods Cannot allocate memory, even with memory_over_commit_ratio set 10.0

The error message below comes from QEMU

2019-05-30T17:08:05.186659Z qemu-system-x86_64: cannot set up guest memory 'pc.ram': Cannot allocate memory

AS such, what I see here is that MAAS is allowing you to create the VM, but qemu is not... so I think you should investigate why qemu doesn;t allow you to.

Hi John,
your system has (at the time of the logs) ~2.5G free and ~1.5G that it could free up.
There is no swap set up.
That is very likely to fail to be spawned and qemu is just telling you so.
There is nothing that qemu does "wrong" if it just can't get as much memory as it needs.

I have read quickly into [1] where this maas tunable is defined.
And setting 10 only allows you to try to define guests up to such size. But you need sooner or later some space those can live in. Overcommit is only good until used and sizing of a machine should always have at least the required amount as swap or - even if you are lucky and can start things - you might later on crash hard on an OOM which is even worse.

@Maas team - do you ensure that (memory*ratio)+baseConsumption (e.g. 15%) is <= memory+swap of the system (as an upper limit)? If not such issues as seen here will happen and I don't yet see any component doing wrong.

[1]: https://docs.maas.io/2.4/en/nodes-comp-hw#configuration

No we do not ensure that. We leave that up to the user to define a proper over-commit ratio for their systems.

Good to know Blake, thanks.

IMHO you should consider this a low prio, but very nice to have item.
Without it seems like calling for trouble later on.
Cases are:
a) Like John here unable to start guests they expected to be able to
b) (more critical) people that also changed vm.overcommit_memory to allow the initial alloc breaking hard later on.

In a perfect world I'd envision a slider like this:

Overcommit ratio:
1.0 * 10
|----------|----------------|

* The recommended upper limit at (PhysMem-15%)+Swap
That way it would still be under the full control of the user, but this bug here would not grow into a common issue.

Until such code that adds something like an "upper recommended bound" is written you could just make things more clear with a comment at the slider maybe?

Therefore I'd be happy if you would consider this as a feature request.

@John - from your POV are you fine for now with the explanations or is there something else that was missed but is needed for you?

It appears your system is over committed to much. qemu needs a minimal amount of resources to be able to start the VM. Leaving this open as our docs and the UI should explain this better.

Tracking a doc task and a UI task here, nothing to do for MAAS core itself

This could be done similar to the helptext in commissioning -> minimum kernel. To implement the suggested solution the frontend would need to have info about the physical mem and the swap of the VM host.

To recap:

The suggested solution of this is to display a warning if somebody sets the overcommit ratio so that is would exceed (PhysMem-15%)+Swap .
This makes sense because you cannot deploy more machines than you have memory and swap. It should still be under the control of the user as they might be sure what they are doing and are only starting some of those machines at the same time.