Need to not display hibernation in UI if do not have enough swap space

Also to add we have found that at times the installer (partman) will not create enough space even on larger harddisks.

This bug is somewhat related (not exactly). Though this is what we are doing to work around the issue when seen:
https://bugs.launchpad.net/ubuntu/+source/upower/+bug/432598

Surely the size needed is not an exact science. You need the amount of memory /currently in use/ (excluding caching) plus 4Mb, not the amount of RAM present. If the machine is using swap then total memory + 4Mb may not be enough. IIRC, the algorithm is that cache it thrown out, and everything remaining in memory is copied to swap.

Therefore no swap size is guaranteed to allow hibernation. Even if you have 4GB RAM and 32GB swap, if you happen to have 33GB virtual memory in use, you cannot hibernate.

The best solution would be a real-time check that the system at present is using less memory than the swap size (+4Mb) - even better it should have a message in it's place, like "Hibernation not available - too much memory in use. Shut down applications and retry".

There are two things to consider: physical memory to build an image before writing to disk and also available swap to do the write.

The kernel generates an memory image of the frozen system before writing it to disk.

Any pages in the zone regions that can be freed are freed. I believe this includes pages that are cached but not dirty (i.e. don't need writing back to disk).

Next, each zone is scanned an the number of present pages are totalled up, excluding the "nosave memory" (see later).
You can see the zone info in /proc/zoneinfo, the "present" field has the number of total pages present. The total excludes the "nosave memory" regions - this info can be seen by:

dmesg |grep "PM: Registered nosave memory:"

.. this tells you the nosave regions. These are pages that are marked in specific regions that do not need saving. These need to be turned into page sizes and taken from the "present" field from the zone info to give you an idea of how many pages need dumping in the final hibernate image.

However, the kernel calculates all of this in two sweeps, first the low-mem pages then the high-mem pages. It's rather complex to say the least.

Also, one needs to add in page meta information. This is calculated as: number of image pages * 8 and rounded up to the nearest page. In other words, for every 1M, we have 256 pages, and hence 2K of meta page info. So for a 1GB image, we have 2M of meta data too.

That's the image size. However, one needs to be able to allocate a hunk of memory for this image in physical memory before dumping to disk, and also have enough page space on swap for this image too.

Cheat Mode:

Instead of getting the calculator out and figuring out how big the swap needs to be, one can get a feel for the size by just doing a successful syspend and grepping through the output from dmesg and look for: "PM: Need to copy". This will tell you how many pages needed to be copied. However, this is only sensible for the desktop scenario you just hibernated. Some users may overload their system with many apps, in which case the hibernation image will be huge, in fact, there may not be enough free physical memory to generate an internal image before dumping it to swap.

The kernel will report the size of the hibernation image too, which can be grepped for. Look for "PM: Hibernation image created".

There may be mileage in shrinking the working set down by forcibly dropping pages, e.g.:

sync
sleep 1
sync
# Freeing the page cache:
echo 1 > /proc/sys/vm/drop_caches
# Free dentries and inodes:
echo 2 > /proc/sys/vm/drop_caches
# Free the page cache, dentries and the inodes:
echo 3 > /proc/sys/vm/drop_caches

But this maybe redundant. I've not experimented. And dropping caches is painful and if you don't sync it may cause data loss.

Marking the report as Triaged and assigning the tasks to the teams that are taking care of those package, thanks Jerone.

The indicator just asks upower whether it's able to hibernate. Why was that task added, doesn't it show the error message that upower returns? I recently tested it in XFCE, and there that works, so upower is sending an appropriate error.

upower itself already does have logic to check whether we have enough swap space. Currently it says "yes" if there is at least (active memory) * 0.8 available swap space. For the corner cases where this isn't enough, the UI should return an appropriate error message. Also, I regularly hibernate with less swap than I have memory, just because most memory is used by caches. This seems to do fine for most cases. Is there some practical scenario where this fails? We could certainly bump the 0.8 to 1, in which case it should almost always succeed.

It seems to me that indicator-session is doing the right thing here. We're not currently displaying a dialog if it fails as we couldn't figure out what it would say. "Failed to Hibernate" -- well, the fact that the display is on probably told you that already :)

@Martin
             Yes we need to move it from .8 to 1 . This has to work 100% of the time and not expose the ability to hibernate to the users if the system knows it can't do it.

@Ted
        Having the failure message is part of the issue. If you know hibernate is going to fail do not expose it to the user.

Jerone Young [2010-07-04 22:09 -0000]:
> Having the failure message is part of the issue. If you know
> hibernate is going to fail do not expose it to the user.

We don't know in advance that hibernate is going to fail. We only know
that after the fact.

@Martin
         What I was saying is if we can calculate to know that we will or will not have enough space then that would prevent from failing, at least from not having enough swap space.

On Sun, 2010-07-04 at 22:09 +0000, Jerone Young wrote:
> Having the failure message is part of the issue. If you know hibernate is
> going to fail do not expose it to the user.

What should the error message say? Obviously a message saying
"Hibernate failed" wouldn't provide new information. We don't really
know why it failed. I don't think we provide a way to dynamically
adjust swap space, so there is no way to fix it.

The only thing I can think of is that we'd run apport so that a bug
could be filled. But, I'm not sure how we'd distinguish between a
hibernate that failed an a hibernation that was really short -- seems
possible though.

  affects indicator-session
  status incomplete

  affects ubuntu/indicator-session
  status incomplete

As for the reason, it should be mentioned in /var/log/pm-hibernate.log, or at least in /var/log/kern.log. But those are mostly suitable for attaching to bug reports. As for an error message for display, upower should provide that through DBusErrors.

Colin, thanks for your detailled explanation in comment 4. You didn't mention compression, though. Isn't the image written to disk compressed? If so, it's rather hard to predict, of course, but it might explain why upstream chose that "0.8" factor. Google isn't very helpful in rsolving this question, I'm afraid.

Once we can ascertain whether or not native Linux hibernation uses compression, I can take this discussion to the upstream ML.

To clarify, the kernel does not compress the image during hibernation.

Discussed with upstream, I pushed the fix to upstream trunk:
http://cgit.freedesktop.org/upower/commit/?id=fc7c6e003a626367558300ce8566e346758ad8e7

Jerone, do we need this backported to lucid in OEM projects (the common PPA)?

I just checked the code again, and it was actually the other way around. The previous 80% was a bug in the wrong direction, it meant that you can only hibernate if your active memory is up to 80% of available swap space (not the inverse, as I originally thought). So this severely erred on the side of caution, and I'll fix it upstream to be 99% instead.

I also tested a hibernate attempt with not having enough swap, and confirm that upower fails with a D-Bus error message right away:

$ dbus-send --system --print-reply --dest=org.freedesktop.UPower --reply-timeout=60000 /org/freedesktop/UPower org.freedesktop.UPower.Hibernate
Error org.freedesktop.UPower.GeneralError: Not enough swap space

However, this isn't shown anywhere in the UI.

Now, the thing which is actually wrong in upower is that it checks available swap/used memory at the time when it starts up, instead of when you attempt to hibernate. I'll fix that.

Fixed testing for swap space upstream now:
http://cgit.freedesktop.org/upower/commit/?id=79c7b27332721a8ef6f24db585939de200927f07

It's planned to do a new release tomorrow, which I'll upload to sid and maverick.

@Martin
          Not really for 10.04. Only moving forward for 10.10 and up. Having it in Lucid is a "nice to have". Thanks for looking into this Martin!

@Martin
          Was upower the only thing that needed this change? I plan to test out with build of 10.10 this week.

Jerone Young [2010-07-12 0:18 -0000]:
> Was upower the only thing that needed this change?

The upower change should make hibernation robust enough so that it
hardly ever fails after trying. If you don't have swap (or encrypted
swap etc.), the indicator now does not offer hibernation at all. It
will fail immediately now if you have very little swap/too much memory
usage. That error message is not displayed anywhere, though (the
indicator task).

> I plan to test out with build of 10.10 this week.

It's not yet in Maverick, just committed upstream so far. I asked
upstream for a new release, and it's planned for today.

This bug was fixed in the package upower - 0.9.5-1

---------------
upower (0.9.5-1) unstable; urgency=low

  * New upstream release:
    - Dynamic testing for enough hibernate swap. (LP: #599352)
    - Port UPower to libusb1 to avoid unfixable crashes. (LP: #522827)
  * debian/rules: Build a PO template during package build, for translation
    systems. (LP: #538321)
  * debian/control: Switch libusb-dev build dependency to libusb-1.0-0-dev,
    and add libimobiledevice-dev to support the new backend.
  * debian/control: Bump Standards-Version to 3.9.0; no changes necessary.
  * Drop transitional devicekit-power-doc package. It's been in testing long
    enough, and we do not want to carry it into the squeeze release.
 -- Martin Pitt <email address hidden> Mon, 12 Jul 2010 14:43:12 +0200

I did the upower part now. Not sure how much you need the error displaying in the indicator.

@Maritn
         Thanks for the work again. No need to display the error. Just need to make sure it doesn't show up in the indicator & session menu.

It's not shown if there is too little swap for hibernation when the session is started. However, since we cannot constantly monitor the active RAM, it can happen that hibernate fails if right after login the active memory is less than available swap, but over time available swap decreases, active memory increases, and memory becomes greater than swap.

According to comment 24 it seems that Jerone is happy with this now?

@Ted

Just curious why you rated this as Low?

If a user initiates hibernate from the UI, and it fails, there's no error displayed to a user. Couldn't this indirectly lead to loss of a user's data?

On Tue, 2010-09-28 at 22:46 +0000, Tony Espy wrote:
> Just curious why you rated this as Low?
>
> If a user initiates hibernate from the UI, and it fails, there's no
> error displayed to a user. Couldn't this indirectly lead to loss of a
> user's data?

I guess that I feel the cases that would lead to the loss of data the
better indicator is the lack of the screen being on. That's a huge
notification that the hibernate failed, a notification would just give
more information on why it failed not any better indication that it
failed.

I don't understand why this is filed against indicator-session. If Hibernate fails, the error message you get (if any) should be the same regardless of whether you triggered it from indicator-session, from the power button dialog, from the battery getting critically low, or any other way. So it doesn't belong in the indicator-session code.

In any case, this bug is fixed in that the menu no longer shows Hibernate when there isn't enough memory. Then again, it doesn't show Hibernate in any other case, either. If we reintroduce the Hibernate feature later, we may need to revisit this.