Trashing hell

No way for the kernel to tell if this is what you want or not. It will start
killing applications if the system runs out of memory. However, if you have
plenty of memory, what should it do? Start killing processes that are "too
busy"? I've put machines into this state on purpose. I wouldn't want the kernel
to start killing things. There's no danger in it, so why start randomly booting
processes?

This is a non-bug. Kernel didn't do anything wrong. If you want to control this
sort of things, start looking at process limits (the kernel respects them, but
you have to set them up per user or per process).

I do not question that there are some situations, e.g. servers locked up in a
room, where this behaviour is fine. But I question the value of allowing one
buggy process to cause the system to freeze for more than 10 minutes on a
desktop system intended for ordinary users. Do you really think this is
reasonable behaviour?

Really, how hard can it be to detect this kind of situation?

if have been trashing for 10 seconds
  if one process consumes most of the memory and has grown > 100% in the past 20
seconds
    kill that process

Make it an option so that server people can switch it off. Or instead set
default ulimits in Ubuntu based on the available memory.

I reported this because I was hoping that a Ubuntu guy would be able to see this
from a desktop user's point-of-view. If I fix the process limits on this
particular desktop machine, it won't help the millions of other mere users who
have no idea what a process limit is.

I hope you will reconsider.

Detecting it is not he point. Deciding that you do want to kill that process is
the point. What if that process is a 3D rendering program? That's something a
desktop user would want that could take up massive amounts of CPU and memory,
and likely do a lot of trashing. Do they want us to kill that process by
default? Probably not. In fact, we'd be a rediculous OS if we did do it by default.

What you want is process limits established for the user. These are hard and
soft limits that can be placed on the amount of memory (phys and virtual) and
cpu that a user or process can use. You can set this up yourself now with the
system you already have installed.

As such, if you feel that the default system needs these limits, then by all
means file a bug on the proper package (hint, it's not the kernel, since
userspace sets these limits, and the kernel merely enforces them). Try GDM or
similar.

Coming to think of it, it is probably easier to just deny the process any more
memory and let it die its own horrible death. The thing is that if you are using
your computer for software rendering, then you can probably also find a way to
disable the default setting (although if you end up in this kind of trashing
hell I think it would be faster to get a five-hour job a McDonalds and earn the
money necessary to buy the extra RAM). It is a lot harder to protect the system
yourself than it is to disable a default protection.

But OK, I will file a new bug on UNKNOWN and hope someone else will have a go on
it. I'm not sure the current process limit system is good enough for modeling
this situation, but who knows. Anyway, thanks for a great response time.