When using swap in ZFS, system stops when you start using swap

A swapfile on ZFS is a bad idea. Swapped out pages get pushed through the vfs into zfs and each page of swap will be magnified in the number of free pages required to get this page out to disk.

https://github.com/zfsonlinux/pkg-zfs/wiki/HOWTO-use-a-zvol-as-a-swap-device

..there are known issues with swap on ZFS not working well on heavily memory loaded systems.

@colin-king

> A swapfile on ZFS is a bad idea. Swapped out pages get pushed through the vfs into zfs and each page of swap will be magnified in the number of free pages required to get this page out to disk.

currently, swap uses zvol in default. not a swapfile. (actually I also thinking swapfile is mistake of bad architecture..)

> ..there are known issues with swap on ZFS not working well on heavily memory loaded systems.

and also, i also see that article, and check it.
It was already optimised.

I did the same test for USB flash memory and it works normally.
try create a swap on zvol, it's definitely not working well..

===================================================================

jehos@MacBuntu:~$ sudo fdisk -l /dev/sdc
Disk /dev/sdc: 7.46 GiB, 8002732032 bytes, 15630336 sectors
Disk model: Cruzer Blade
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 91BE0613-2C00-8944-BEFF-12CCBF5DE2B8
jehos@MacBuntu:~$ sudo mkswap -f /dev/sdc
Setting up swapspace version 1, size = 7.5 GiB (8002727936 bytes)
no label, UUID=14bb8b5c-ded5-444e-8a62-a21cbc469d08

jehos@MacBuntu:~$ sudo swapon /dev/sdc

jehos@MacBuntu:~$ free -m
              total used free shared buff/cache available
Mem: 7870 3106 3843 257 920 4260
Swap: 7631 0 7631

jehos@MacBuntu:~$ time stress --vm 100 -t 10
stress: info: [6247] dispatching hogs: 0 cpu, 0 io, 100 vm, 0 hdd
stress: info: [6247] successful run completed in 15s

real 0m15.124s
user 0m0.736s
sys 0m3.536s

Status changed to 'Confirmed' because the bug affects multiple users.

Confirmed this affects me. 16gig of ram, and a swap ZVOL. When memory gets low the system freezes for several minutes.

Also reported this bug here: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1848102

As I believe ubiquity shouldn't be creating a ZVOL for swap when this is a known issue.

Status changed to 'Confirmed' because the bug affects multiple users.

partman-swapfile does not create swapfiles on neither btrfs nor zfs.

when using desktop installer and choosing zfs a zfs zvol is created for swap.

it uses all the settings seen in the guide, but the "com.sun:auto-snapshot=false" do we need to add that or does our zfs not support it?

Or are you saying, that zfs zvol swap shouldn't be created either, since that is known to cause lockups?

> "com.sun:auto-snapshot=false" do we need to add that or does our zfs not support it?

You do not need that. That is used by some snapshot tools, but Ubuntu is doing its own zsys thing.

> Or are you saying, that zfs zvol swap shouldn't be created either,
> since that is known to cause lockups?

That is correct. As per the thread at the attached github zfsonlinux bug report, zfs zvol swap causes system freezes for minutes at a time, and under enough memory pressure will lock up the system completely.

Ubiquity installer should put swap on it's own partition, not a zvol.

This bug was fixed in the package ubiquity - 19.10.21

---------------
ubiquity (19.10.21) eoan; urgency=medium

  [ Dimitri John Ledkov ]
  * scripts/plugininstall.py: skip installing extras with multiarch
    dependencies in the first pass.

  [ Mathieu Trudel-Lapierre ]
  * Updated translations from Launchpad.

  [ Adam Conrad ]
  * Rewrite zsys-setup swap handling to use a swap partition (LP: #1847628)

 -- Adam Conrad <email address hidden> Wed, 16 Oct 2019 17:36:10 +0100

upstream proposal is to force everything through ZIL by creating the swap like this:

zfs create -o sync=always -o primarycache=metadata -o secondarycache=none -o logbias=throughput -b 4K -V 8G rpool/swap

I'll test with our setup on Ubuntu.

@jibel unfortunately that doesn't help, still stalling when writing to swap, even without compression.

Yes this is confirmed, there is currently no way to use swap on zvol without a deadlock. The only solution at the moment is to create a swap partition.

Is there any hope that a swap on zvol with sync=standard at least doesn't hang/deadlock? Like:

zfs create rpool/swap -V 2G -b $(getconf PAGESIZE) -o logbias=throughput -o sync=always -o primarycache=metadata -o secondarycache=none

In my testing this didn't hang when swap got hit.