Comment 7 for bug 756499

I'm marking this a duplicate of #693594 but if you come across this here are some Workarounds/Fixes:

1)do not move all tasks into a default cgroup
comment CREATE_DEFAULT=yes out and uncomment CREATE_DEFAULT=no in /etc/default/cgconfig

2)putting the following in /etc/cgconfig.conf fixed (a better value as this is kernels default anyway)
group sysdefault {
    cpu {
        cpu.rt_runtime_us = 950000;
    }
}

3) Add a rule so that [kthreadd] is never put in the default group

In case anybody was wondering what is going on:
Suspend fails because it cannot shutdown your extra CPUs ( you can test this with echo 0 > /sys/devices/system/cpu/cpu$x/online ).
The kernel can't shutdown the extra CPUs because some it spawns some threads (ksoftirqd and migration) that need realtime scheduling and because they are in the sysdefault subgroup it will not allow it (you can fix this with echo PID of the stuck thread to /sys/fs/cgroup/cpu/tasks (the thread will show up as state D (Uninterruptible sleep))).

Note that on some kernels: (e.g 3.0.0 that is shipped with 11.10) you can shutdown the CPUs once each before the problem occurs (e.g one suspend) but on others ( e.g 3.3.0-rc4) you cant.