Compiz gets into a state where going fullscreen in apps causes an X11 BadAlloc error

Oh, I should probably also mentions that I am using TwinView.

I've now tried changing the "Legacy Fullscreen Support" and "Unredirect Fullscreen Windows" options while experiencing this problem, but that had no effect. I also checked my memory usage when this happened and it was only at around 50%. Not sure what else to check.

Hi Tristan
you report this issue some time ago and hasn't been any activity in a while, i was wondering if this is still an issue for you with the latest upgrades in Hardy Heron.

Thanks in advance

Yes, this still occurs today, even with the Compiz/X/NVIDIA packages from Hardy Heron. Attaching logs taken with the Hardy packages and changing back to New.

These logs were taken with me using SMPlayer and the 'xv' output module.

Anything I can do to help debug this? This is the last bug preventing me from using Compiz. I'm a computer engineer so you can feel free to ask me to use gdb and stuff.

I haven't seen this in ages. Changing to Invalid.

I take it back, this happened again today. Definitely less common than it used to be, though. Note that I am using pure up-to-date Hardy now.

Perhaps related to Bug #216322?

Seems unlikely to me. The description is very different.

Please create a backtrace and attach it here as a non-compressed text file. Instructions are available at https://wiki.ubuntu.com/DebuggingProgramCrash and https://wiki.ubuntu.com/Backtrace

Relevant debug package may include xserver-xorg-core-dbgsym, compiz-core-dbgsym. libx11-6-dbgsym, libx11-6-dbg, libc6-dbg

This bug is not a crash, it is just an error that makes the program exit. How can you get a backtrace for that?

ahh.. my fault for forgetting to mention setting breakpoint. Could you try to break at exit. Interrupt gdb by ctrl-c and type 'break exit'

Gotcha. I will do this the next time this bug happens. (Could be a while though.)

Happened again today, except this time the bug occurred whenever the video WASN'T in fullscreen mode. I got backtraces for both Totem and MPlayer; they are attached. They don't really say much though, basically just that the program gave a frame to the X server and got back this error.

At the time the bug happened, I was only using 75% RAM and 5% of SWAP. After getting the backtraces, I noticed that I had about a dozen instances of SMPlayer open (probably due to a stuck shortcut key that opened too many). After closing them, the bug went away, even though my RAM usage had barely changed (75% -> 72%). Then I tried opening a dozen instances of SMPlayer again (-> 78% RAM usage) and the bug came back.

Perhaps the problem is that X and/or Compiz reserves video memory for each app that might need it, even when they haven't asked for it? But that the memory for fullscreen mode is shared? Though that wouldn't explain the original situation where the bug happens in fullscreen mode ...

I have the same problem. My card is a nVidia 660GT with 128MB. I'm using Ubuntu Hardy and I tried with the Nvidia restricted drivers 169.12, 173.14.12 (envyng) and the latest 177.80 (from the nvidia web).

I can reproduce the problem whenever I want, I think it's related with the amount of texture memory used. I'm using a resolution of 1920x1200 and when I have a few windows opened I can't watch videos in fullscreen, Totem, Mplayer and VLC just die with a BadAlloc error.

If I have a lot of windows opened I can't even watch the video at all, the players close with the same error before the video is shown. But if I minimize all the windows, I can play the video in fullscreen, and when the video is playing I can re-maximize the previously minimized windows and the video will keep playing (sometimes it fails too).

I'm only having this problem with compiz enabled.

Yay, finally some confirmation. :)

I'm having the same problem using Sabayon 32/64 bit.

FYI, I hadn't seen this for a while, but then today it happened a lot when I was trying to play a 1080p video. So it's probably more likely with larger video resolutions. Which of course would make sense.

I've seen this as well. It seems to be related to the number of windows I have open at the time. If I close a few of them the problem goes away.

This is on Intrepid on a 8Gig 64bit Intel box with a GeForce 7600 GT (256MB Memory) and 2 1920x1200 monitors running in twinview.

Is there a good way to see what Compiz is allocating for textures etc?

Still occurs in Jaunty Jackalope.

I'm using Fedora 11, this is my problem too on my GeForce 7600 GS (256Mb of RAM). It's not Ubuntu specific.

Current versions:
Xorg 1.6.1.901
Nvidia driver 185.18.14 / 180.51
Kernel 2.6.29.4
Compiz 0.7.8
i686

My previous Fedora 9 _working well_ on the same hardware, there's:
Xorg 1.5.2
Nvidia driver 177.82
Kernel 2.6.27.24
Compiz 0.7.6
i686

I have to open many windows to trigger the bug (~10-15 firefox). If I switch to another VT and back / restart compiz with --replace / kill some windows, then I can play videos with XV for a while (5-10 minutes). ALT+TAB in compiz is much slower when the XV doesn't work.

Daniel, I cannot found a method to check used texture memory, googled for hours for nothing.

It also happens to me. Jaunty with a Nividia 8400GS video card 512MB, 4 gig of system ram and a 1670X1050 resolution. Only does it when I run emerald.

Hmm, I'm also using Emerald. Maybe this only occurs when using Compiz _and_ Emerald?

Has anyone seen this when not using Emerald?

I'm using gtk-window-decorator..

NVnews topic: http://www.nvnews.net/vbulletin/showthread.php?t=124410

It happens to me as well, Ubuntu 9.10, nvidia drivers 185.18.36-0ubuntu9.

Mplayer issue can be easily resolved with setting video output to x11, but tvtime application can't change it's output and it doesn't work. Also, screensavers are extremely slow with 100% CPU usage.

It makes compiz unusable, so I had to turn it off.

I see this on my AGP 7600GT (256M); as well as stopping video going full screen, it makes mythtv frontend with the opengl drawing engine unbelievably slow and cpu hungry (presumably it falls back to software 3d rendering or something). Would this be caused by texture memory fragmentation or something? (Does video and/or whatever mythtv does need a large _contiguous_ chunk of it? Or does that not make sense - does texture memory get virtualised by an mmu (gart?) or something?).

Sadly I've had to stop using compiz as a result.

I found solution that works for me.

I deleted whole ~/.config/compiz and ~/.gconf/apps/compiz directories, and compiz is behaving properly afterwards - no more xv issues.

YMMV.

Thank you for taking the time to report this bug and helping to make Ubuntu better. We are sorry that we do not always have the capacity to look at all reported bugs in a timely manner. There have been many changes in Ubuntu since that time you reported the bug and your problem may have been fixed with some of the updates. It would help us a lot if you could test it on a currently supported Ubuntu version. When you test it and it is still an issue, kindly upload the updated logs by running apport-collect <bug #> and any other logs that are relevant for this particular issue.

I no longer have the computer that experienced this issue, and my new one has tons more video RAM so I can't determine whether this issue still exists or not. I'll leave it as Incomplete in case someone else is still experiencing it and can provide the info.

[Expired for compiz (Ubuntu) because there has been no activity for 60 days.]