`systemctl hibernate` incorrectly reports "Not enough swap space for hibernation"

One thing I have noticed, is that on boot:

$ cat /sys/power/resume
0:0

I can't test right now, but I _think_ that before the holiday break setting that to 8:2[0] and restarting systemd-logind meant that hibernate did then work.

[0] $ ls -l /sys/dev/block/8:2
lrwxrwxrwx 1 root root 0 Jan 5 09:08 /sys/dev/block/8:2 -> ../../devices/pci0000:00/0000:00:01.3/0000:02:00.1/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda2

and how much RAM does the system have? and how much is used? and of swap?

free -h

free swap should be more then used memory.

/sys/power/image_size represents the required amount of space for the image; that said, the machine has 16G RAM total, so even if that were maxed out, it would fit into 97.7G comfortably.

Oh, and `free -h`:

              total used free shared buff/cache available
Mem: 15Gi 6.0Gi 667Mi 567Mi 9.0Gi 8.8Gi
Swap: 98Gi 13Mi 98Gi

> $ cat /sys/power/resume
> 0:0

This was a red herring. What I have found consistently fixes this is:

$ sudo swapoff /dev/sda2
$ sudo swapon -p 1 /dev/sda2

Hibernate then succeeds. However, this is not how I want my system configured: I have a small swap partition on my SSD, which I would like to be used for swap under normal operation; changing the priority to address my hibernation issues means that I'm using my slow HDD as regular swap.

For reference, after the above changes:

$ swapon
NAME TYPE SIZE USED PRIO
/dev/dm-2 partition 980M 0B -2
/dev/sda2 partition 97.7G 0B 1

I enabled systemd-logind debug logging, and I saw:

Jan 06 17:45:18 surprise systemd-logind[73027]: Got message type=method_call sender=:1.264 destination=:1.220 path=/org/freedesktop/login1 interface=org.freedesktop.login1.Manager member=CanHibernate cookie=6 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a
Jan 06 17:45:18 surprise systemd-logind[73027]: Sleep mode "disk" is supported by the kernel.
Jan 06 17:45:18 surprise systemd-logind[73027]: /dev/dm-2: is a candidate device.
Jan 06 17:45:18 surprise systemd-logind[73027]: /dev/sda2: ignoring device with lower priority
Jan 06 17:45:18 surprise systemd-logind[73027]: No swap partitions or files matching resume config were found in /proc/swaps.
Jan 06 17:45:18 surprise systemd-logind[73027]: Sent message type=method_return sender=n/a destination=:1.264 path=n/a interface=n/a member=n/a cookie=185 reply_cookie=6 signature=s error-name=n/a error-message=n/a
Jan 06 17:45:18 surprise systemd-logind[73027]: Got message type=method_call sender=:1.264 destination=:1.220 path=/org/freedesktop/login1 interface=org.freedesktop.login1.Manager member=CanHybridSleep cookie=7 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a
Jan 06 17:45:18 surprise systemd-logind[73027]: Sleep mode "disk" is supported by the kernel.
Jan 06 17:45:18 surprise systemd-logind[73027]: /dev/dm-2: is a candidate device.
Jan 06 17:45:18 surprise systemd-logind[73027]: /dev/sda2: ignoring device with lower priority
Jan 06 17:45:18 surprise systemd-logind[73027]: No swap partitions or files matching resume config were found in /proc/swaps.

However, after a reboot this morning, hibernation does seem to be working, and I instead see:

Jan 07 09:29:37 surprise systemd-logind[1798]: Got message type=method_call sender=:1.125 destination=org.freedesktop.login1 path=/org/freedesktop/login1 interface=org.freedesktop.login1.Manager member=Hibernate cookie=3 reply_cookie=0 signature=b error-name=n/a error-message=n/a
Jan 07 09:29:37 surprise systemd-logind[1798]: Sleep mode "disk" is supported by the kernel.
Jan 07 09:29:37 surprise systemd-logind[1798]: /dev/sda2: device matches configured resume settings.
Jan 07 09:29:37 surprise systemd-logind[1798]: Hibernation will attempt to use swap entry with path: /dev/sda2, device: 8:2, offset: 0, priority: -3
Jan 07 09:29:37 surprise systemd-logind[1798]: Enough swap for hibernation, Active(anon)=3080788 kB, size=102401108 kB, used=0 kB, threshold=98%

So it looks like the issue is that my resume device was not being detected correctly previously, meaning that priority was being used (which resulted in the wrong device being selected).

So, looking at the systemd code at https://github.com/systemd/systemd/blob/c5b6b4b6d08cf4c16a871401358faeb5a186c02a/src/shared/sleep-config.c#L422-L426, perhaps setting /sys/power/resume to the correct device actually was the workaround/fix?

> So, looking at the systemd code at https://github.com/systemd/systemd/blob/c5b6b4b6d08cf4c16a871401358faeb5a186c02a/src/shared/sleep-config.c#L422-L426, perhaps setting /sys/power/resume to the correct device actually was the workaround/fix?

The confusing part about this is that I don't believe I set /sys/power/resume when I first booted, so I guess that was set correctly for some reason this time around? (Or maybe something else is going on entirely, of course.)

these things are very weird indeed.

Also, you do disable secureboot as well right? Because with secureboot on, even though hybernation image is created, it will be ignored and not used upon resume.

On Thu, Jan 07, 2021 at 03:23:36PM -0000, Dimitri John Ledkov wrote:
> Also, you do disable secureboot as well right? Because with secureboot
> on, even though hybernation image is created, it will be ignored and
> not used upon resume.

Yep, Secure Boot is disabled on this system.

And, for clarity, when systemd does hibernate, I haven't had issues
restoring: it's just getting systemd to find the correct swap space to
use that's been the issue.

So I've done a little more testing. On boot, with /sys/power/resume unset (i.e. 0:0), I see this in the logind debug logs:

Jan 08 09:47:11 surprise systemd-logind[1887]: Sleep mode "disk" is supported by the kernel.
Jan 08 09:47:11 surprise systemd-logind[1887]: /dev/dm-2: is a candidate device.
Jan 08 09:47:11 surprise systemd-logind[1887]: /dev/sda2: ignoring device with lower priority
Jan 08 09:47:11 surprise systemd-logind[1887]: /sys/power/resume is not configured; attempting to hibernate with path: /dev/dm-2, device: 253:2, offset: 0, priority: -2
Jan 08 09:47:11 surprise systemd-logind[1887]: Not enough swap for hibernation, Active(anon)=1134696 kB, size=1003516 kB, used=0 kB, threshold=98%

If I set /sys/power/resume correctly, then I see:

Jan 08 09:48:10 surprise systemd-logind[1887]: Sleep mode "disk" is supported by the kernel.
Jan 08 09:48:10 surprise systemd-logind[1887]: /dev/dm-2: is a candidate device.
Jan 08 09:48:10 surprise systemd-logind[1887]: /dev/sda2: ignoring device with lower priority
Jan 08 09:48:10 surprise systemd-logind[1887]: No swap partitions or files matching resume config were found in /proc/swaps.

(I haven't been able to test what happens if I restart logind because I hit https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1910769 if I try; my plan is add an ExecStartPre which sets the resume device to logind's unit file, to see if that does the trick.)

Does anyone know what in the system is _meant_ to be setting /sys/power/resume? Is it possible I'm missing some configuration somewhere to have that setting happen?

> /sys/power/image_size represents the required amount of space for the image

no it doesn't:
https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-power

"The kernel's suspend-to-disk code will do its best to ensure the image size
will not exceed this number. However, if it turns out to be impossible, the
kernel will try to suspend anyway using the smallest image possible."

"Reading from this file will display the current image size limit, which is
set to around 2/5 of available RAM by default."

> /dev/dm-2: is a candidate device.
> /dev/sda2: ignoring device with lower priority
> /sys/power/resume is not configured; attempting to hibernate with path: /dev/dm-2, device: 253:2, offset: 0, priority: -2
> Not enough swap for hibernation, Active(anon)=1134696 kB, size=1003516 kB, used=0 kB, threshold=98%

yeah, this is the problem; the checks in sleep-config.c first look for the highest-priority swap device that is valid as a target, but doesn't actually check if it's large enough for all Active(anon) pages until *after* selecting which swap device to use. Probably need to open an upstream bug.

Status changed to 'Confirmed' because the bug affects multiple users.

I think this was reported upstream as https://github.com/systemd/systemd/issues/19486, which was subsequently fixed (a quick look at the patch makes me think that was the same issue). So this should be fixed in newer releases.

Please re-open if I am mistaken.