Miscompilation with gcc 4.8 (segfault in x264_cqm_init)

Bug #1241772 reported by Alex
264
This bug affects 57 people
Affects Status Importance Assigned to Milestone
x264 (Ubuntu)
Fix Released
High
Andrew Starr-Bochicchio
Saucy
Fix Released
High
Andrew Starr-Bochicchio

Bug Description

SRU Justification:

[Impact]

 * A segfault in libx264-123 has caused major regressions in vlc, kazam, avidemux, and avconv among others. Nearly any program that encodes video in H264 is probably affected.

[Test Case]

 * As this impacts many packages, there are a number of possible test cases. A simple one is to use kazam, the screen recording app:

 1) Install the kazam package.
 2) Run kazam
 3) Make sure "file > preferences > screencast > record with" is set to H264/MP4
 4) Attempt to record a screencast.

 With 2:0.123.2189+git35cf912-1ubuntu1 from saucy install, you will experience a segfault. With the version in saucy-proposed, a screencast will be correctly recorded and encoded.

[Regression Potential]

 * There is little chance of regression. In order to make the most minimal change possible, the package is just simply built with -fno-aggressive-loop-optimizations rather than attempting to backport any code changes from upstream. The most likely regression would be a typo causing a FTBFS (though I have built and tested the package on saucy).

[Other Info]

This had not been rebuilt with gcc 4.8 until one day before release when a rebuilt was triggered to fix the arm64 build.

http://gcc.gnu.org/gcc-4.8/changes.html

"GCC now uses a more aggressive analysis to derive an upper bound for the number of iterations of loops using constraints imposed by language standards. This may cause non-conforming programs to no longer work as expected, such as SPEC CPU 2006 464.h264ref and 416.gamess. A new option, -fno-aggressive-loop-optimizations, was added to disable this aggressive analysis. In some loops that have known constant number of iterations, but undefined behavior is known to occur in the loop before reaching or during the last iteration, GCC will warn about the undefined behavior in the loop instead of deriving lower upper bound of the number of iterations for the loop. The warning can be disabled with -Wno-aggressive-loop-optimizations."

The fix has already been uploaded to trusty.

-----------------------
Original bug:

Vlc crashes when encoding h264:

Thread 9 (Thread 0x7fffd9712700 (LWP 3034)):
#0 __memcmp_sse2 () at ../sysdeps/x86_64/multiarch/../memcmp.S:74
#1 0x00007fffc9ec9d9e in x264_cqm_init () from /usr/lib/x86_64-linux-gnu/libx264.so.123
#2 0x00007fffc9f34374 in x264_encoder_open_123 () from /usr/lib/x86_64-linux-gnu/libx264.so.123
#3 0x00007fffca1fef34 in Open (p_this=0x7fffd0000e08) at x264.c:1254
#4 0x00007ffff795ed00 in vlc_module_load (p_this=p_this@entry=0x7fffd0000e08,
    psz_capability=psz_capability@entry=0x7fffe8ceaa31 "encoder", psz_name=<optimized out>, b_strict=b_strict@entry=true,
    probe=probe@entry=0x7ffff795e5d0 <generic_start>) at modules/modules.c:347
#5 0x00007ffff795f1a4 in module_need (obj=obj@entry=0x7fffd0000e08, cap=cap@entry=0x7fffe8ceaa31 "encoder", name=<optimized out>,
    strict=strict@entry=true) at modules/modules.c:437
#6 0x00007fffe8ce8a34 in transcode_video_new (p_stream=p_stream@entry=0x7fffe00059d8, id=id@entry=0x7fffd00008e0) at video.c:241
#7 0x00007fffe8ce9c28 in transcode_video_add (p_stream=p_stream@entry=0x7fffe00059d8, p_fmt=p_fmt@entry=0x7fffe0526990,
    id=id@entry=0x7fffd00008e0) at video.c:832
#8 0x00007fffe8ce54a8 in Add (p_stream=0x7fffe00059d8, p_fmt=0x7fffe0526990) at transcode.c:553
#9 0x00007ffff797f64f in sout_InputNew (p_sout=0x7fffe000a6d8, p_fmt=p_fmt@entry=0x7fffe0526990)
    at stream_output/stream_output.c:184
#10 0x00007ffff791b6e6 in DecoderProcessSout (p_block=0x0, p_dec=0x7fffe0523ea8) at input/decoder.c:1812
#11 DecoderProcess (p_dec=p_dec@entry=0x7fffe0523ea8, p_block=p_block@entry=0x7fffe052e9d0) at input/decoder.c:2040
#12 0x00007ffff791bde4 in DecoderThread (p_data=0x7fffe0523ea8) at input/decoder.c:938
#13 0x00007ffff76c8f6e in start_thread (arg=0x7fffd9712700) at pthread_create.c:311
#14 0x00007ffff71ef9cd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

http://forum.doom9.org/showthread.php?p=1623567:
Configure x264 with "--extra-cflags=-fno-aggressive-loop-optimizations" solves this problem.
Also seems that problem solved in trunk - http://git.videolan.org/?p=x264.git;a=commit;h=89aecb440e2939be7fb72d8362eb12504711b94f

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in x264 (Ubuntu):
status: New → Confirmed
Revision history for this message
Maarten Baert (maarten-baert) wrote :

I can confirm this. It affects anything that uses libx264, not just VLC but also avconv (the command line tool) and any program that encodes video with libavcodec (i.e. most video editors and screencasting software).

The backtrace in avconv is mostly the same:
#0 __memcmp_ssse3 () at ../sysdeps/x86_64/multiarch/memcmp-ssse3.S:1684
#1 0x00007ffff3012d9e in x264_cqm_init () from /usr/lib/x86_64-linux-gnu/libx264.so.123
#2 0x00007ffff307d374 in x264_encoder_open_123 () from /usr/lib/x86_64-linux-gnu/libx264.so.123
#3 0x00007ffff693736c in ?? () from /usr/lib/x86_64-linux-gnu/libavcodec.so.53
#4 0x00007ffff6cc25f5 in avcodec_open2 () from /usr/lib/x86_64-linux-gnu/libavcodec.so.53
#5 0x00000000004051aa in ?? ()
#6 0x00007ffff59aede5 in __libc_start_main (main=0x404c10, argc=12, ubp_av=0x7fffffffe008, init=<optimized out>,
    fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdff8) at libc-start.c:260
#7 0x0000000000407983 in ?? ()

Revision history for this message
3vi1 (launchpad-net-eternaldusk) wrote :

I can confirm both the problem and the workaround (--extra-cflags=-fno-aggressive-loop-optimizations) as well, after testing on 13.10 (64-bit)

Revision history for this message
Doug McMahon (mc3man) wrote : Re: [Bug 1241772] Re: Miscompilation with gcc 4.8 (segfault in x264_cqm_init)

On 10/19/2013 10:13 PM, 3vi1 wrote:
> I can confirm both the problem and the workaround (--extra-cflags=-fno-
> aggressive-loop-optimizations) as well, after testing on 13.10 (64-bit)
>
You all could also just use a newer version of x264, the current is a
bit stale, from late april 2013
(same goes for libav which is even older

Revision history for this message
Maarten Baert (maarten-baert) wrote :

@Doug: x264 can't be updated without recompiling or even updating libav as well, AFAIK.

I've added a patched version of x264 to my PPA:
https://launchpad.net/~maarten-baert/+archive/simplescreenrecorder
Normally I wouldn't mess with official packages, but since the official one is completely broken, I can't possibly make it worse, right? :) It's the original package from Ubuntu with the upstream patch applied to it.

For those who want to try it (at your own risk):

    sudo add-apt-repository ppa:maarten-baert/simplescreenrecorder
    sudo apt-get update
    sudo apt-get upgrade

Revision history for this message
Dave Gilbert (ubuntu-treblig) wrote :

Maarten: Do you have a pointer to some discussion about that patch? Is this really a gcc optimisation bug or is vlc doing something that's not legal/safe but didn't happen to trigger on older gcc?

Revision history for this message
David Newman (drdrnewman) wrote :

This affects vlc, avidemux, and command line use of avconv. All produce segfaults on my 64-bit Kubuntu 13.10 upgraded computer. It affects all ways in which avconv uses libx264.

Revision history for this message
Maarten Baert (maarten-baert) wrote :

I couldn't find any info either. As far as I understand, x264 was doing something that wasn't standard-compliant (array overreads), and the GCC optimization aggressive-loop-optimizations relies on the assumption that all code is standard-compliant. So this is not a GCC bug.

It may be possible to verify this with Valgrind, but I think the version of Valgrind in Ubuntu doesn't support all the instructions that x264 uses yet, so this is requires recompilation without assembly optimizations.

Revision history for this message
Dave Gilbert (ubuntu-treblig) wrote :

Triaged: Upstream fix found
High: Affecting multiple common packages making them unusable.

Changed in x264 (Ubuntu):
importance: Undecided → High
status: Confirmed → Triaged
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "Upstream patch (backported)" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Revision history for this message
Torsten Römer (dode) wrote :

Rebuilding the libx264-123_0.123.2189+git35cf912-1ubuntu2 packages with -fno-aggressive-loop-optimizations indeed solves the problem on 13.10, there is no more segfault: https://www.dropbox.com/sh/0o1bi2vzmp5p11o/vBzFd8nCgx

Revision history for this message
Daniel T Chen (crimsun) wrote :

In light of this being an SRU candidate for 13.10, I recommend we simply compile using -fno-aggressive-loop-optimizations.

Changed in x264 (Ubuntu Saucy):
status: New → Triaged
importance: Undecided → High
Revision history for this message
Lonnie Lee Best (launchpad-startport) wrote :

Andrew Starr-Bochicchio has pointed out that his bug also affects kazam:
https://bugs.launchpad.net/ubuntu/+source/kazam/+bug/1241924

Changed in x264 (Ubuntu):
status: Triaged → In Progress
assignee: nobody → Andrew Starr-Bochicchio (andrewsomething)
Changed in x264 (Ubuntu Saucy):
status: Triaged → In Progress
assignee: nobody → Andrew Starr-Bochicchio (andrewsomething)
description: updated
Revision history for this message
Andrew Starr-Bochicchio (andrewsomething) wrote :
Revision history for this message
Adam Pryor (adam-pryor-1992) wrote :

I can confirm that Maarten Baert's ppa has resolved the issue I was having with x264 on 13.10. Kazam works fine now and mp4 playback is also working. I have had no negative side-effects so far, but I'll be sure to report back if I find any problems.

Revision history for this message
Klaus Kettner (klaus-kettner) wrote :

I can confirm that Maarten Baert's ppa has resolved my problems with avconv on the command-line. Tnx dude!

Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Alex, or anyone else affected,

Accepted x264 into saucy-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/x264/2:0.123.2189+git35cf912-1ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in x264 (Ubuntu Saucy):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package x264 - 2:0.123.2189+git35cf912-1ubuntu2

---------------
x264 (2:0.123.2189+git35cf912-1ubuntu2) trusty; urgency=low

  * Compile using -fno-aggressive-loop-optimizations to work around
    segfault caused by changes in gcc 4.8 (LP: #1241772).
 -- Andrew Starr-Bochicchio <email address hidden> Tue, 22 Oct 2013 12:22:07 -0400

Changed in x264 (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
John D (johndcarmack) wrote :

I tried following the directions in the EnableProposed link, but the aptitude program complains there are no packages for saucy-proposed. Using apt-get specifically worked, though? At any rate, I finally got it installed and no more segfault.

Revision history for this message
Miklos Juhasz (mjuhasz) wrote :

The proposed package resolves this bug for me as well.

tags: added: verification-done
removed: verification-needed
Revision history for this message
enrico (eliboni) wrote :

fixed for me as well.

Revision history for this message
asgard2 (kamp000x) wrote :

proposed sources with apt-get are also working for me

Revision history for this message
Yuv (yuv) wrote :

libx264-123 from saucy-proposed fixed the bug for me as well. It is not yet in the main saucy repository where the segfault still happens. What does it take to move the package over from proposed?

Revision history for this message
Scott Kitterman (kitterman) wrote : Re: [Bug 1241772] Re: Miscompilation with gcc 4.8 (segfault in x264_cqm_init)

It needs a week in -proposed for regression testing.

Revision history for this message
Adam Pryor (adam-pryor-1992) wrote :

Just activated the -proposed updates and installed them. Everything works fine, as it did with Maarten Baert's ppa, and I'm seeing no regressions at all so far.

Revision history for this message
Pieter Hintjens (ph-imatix) wrote :

Fixed for me too... Maarten, thank you.

Revision history for this message
korgman (korgie) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package x264 - 2:0.123.2189+git35cf912-1ubuntu1.1

---------------
x264 (2:0.123.2189+git35cf912-1ubuntu1.1) saucy-proposed; urgency=low

  * Compile using -fno-aggressive-loop-optimizations to work around
    segfault caused by changes in gcc 4.8 (LP: #1241772).
 -- Andrew Starr-Bochicchio <email address hidden> Tue, 22 Oct 2013 12:22:07 -0400

Changed in x264 (Ubuntu Saucy):
status: Fix Committed → Fix Released
Revision history for this message
Scott Kitterman (kitterman) wrote : Update Released

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.