Comment 92 for bug 1833281

Another resource, quite long but has a tl;dr, and reviewed by some of the cgroups/resource control folks:
https://chrisdown.name/2018/01/02/in-defence-of-swap.html

I'll use somewhat technically sloppy language, but hopefully a useful metaphor: there's incidental swap and heavy swap. Incidental swap is when some file or anonymous page really isn't needed, and it's good to evict it to free up memory. But in the case of heavy swap (or even reclaim), not at all incidental, it becomes a serious performance impediment. The reality is that there are some tasks that just take gobs of memory and swap isn't a good substitute. But some swap is useful for freeing up memory for things that need to stay in memory.

I'm finding for the incidental swap need, swap-on-ZRAM is quite useful. You are exchanging an IO bound task for a memory+CPU task; also it forces such pages to be pinned into memory. So there's no free lunch. Conservative use would be a ZRAM device around 1/4 of RAM up to a max of 50% RAM. It's not bad or wrong to use more, it's just that some workloads, like the webkitgtk example, have such significant need that it'll actually turn 1/2 of memory into swap, which in effect means you have 50% less RAM for that task. It actually makes the problem worse. Really, the near term is a) build with flags that cause fewer resources to be used in the firist place, or b) build in a systemd user session and limit resources that way, or c) buy more RAM, or d) use conventional swap partition, possibly with zswap to leave the most frequent pages in a small RAM cache pool, that's big enough for the task to eventually complete and just suffer with the ensuing lack of GUI responsiveness.

Anyway, it's a bit complicated. And lots of moving parts. And probably needs more sophisticate use of perf and maybe even bpf could be useful in figuring where the various bottlenecks are.