[SRU] mpeg2enc crashes with SIGILL on non-p4 architectures.

This crash disappears if I just rebuild the mjpegtools Ubuntu package from source on the machine in question, and install it.

StacktraceTop:init_fdct () at fdct.c:340
init_transform () at transfrm_ref.c:247
MPEG2Encoder::SIMDInitOnce () at mpeg2encoder.cc:74
MPEG2Encoder (this=0xbfd2ff18, _options=@0xbfd2fd20)
YUV4MPEGEncoder (this=0xbfd2ff18, cmd_options=@0xbfd2fd20)

Confirming here. This seems to be due to a pentium-4 specific optimization done at the build farm, see http://launchpadlibrarian.net/21608730/buildlog_ubuntu-jaunty-i386.mjpegtools_1%3A1.9.0-0.0_FULLYBUILT.txt.gz.

The configure-script passes -march=pentium4 -mtune=pentium4 to gcc which I think is the cause for the failures.

Further investigation by James Westby shows that the configure-script should always be passed -mno-sse2 to prevent the compiler to use instructions specific to one architecture. debdiff attached:

mjpegtools (1:1.9.0-0ubuntu1) jaunty; urgency=low

  * debian/rules
    - add -mno-sse2 to CFLAGS to prevent the package being compiled with
      pentium4 specific optimizations. (LP: #351017)

 -- Andreas Moog <email address hidden> Tue, 21 Apr 2009 01:46:54 +0200

ACK

Uploaded to jaunty-proposed

Accepted mjpegtools into jaunty-proposed-proposed; please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

The attached debdiff FTBFS due to the compile flag being applied to all arches, even if they don't support it. Attached debdiff corrects this issue by using:

ifneq ($(findstring $(DEB_BUILD_ARCH), i386 lpia),)
        CFLAGS+=-mno-sse2
endif

I just noticed that mjpegtools still get built with -march=penitum4, thus still giving a SIGILL. I didn't notice before because the ppa-buildmachine use K8 as architecture and the binaries produced there just worked.

Attached is a new debdiff:

mjpegtools (1:1.9.0-0.0ubuntu3) jaunty; urgency=low

  * debian/control
    - add quilt build-depend for patch
    - reformat to comply with upstream, making future merge easier
  * debian/rules
    - add quilt
  * debian/patches/02_fix_arch_specifics.patch
    - patch configure and configure.ac to never use arch specific
      optimizations, really fixes SIGILL on non-pentium4 archs.

 -- Andreas Moog <email address hidden> Sat, 25 Apr 2009 15:17:36 +0200

ACK 0ubuntu3 and sponsoring.

Got rejected by ScottK, since new debdiff doesn't have a bug reference. Please add it, or use debuild -v to include the previous changelog.

mjpegtools (1:1.9.0-0.0ubuntu3) jaunty; urgency=low

  * debian/control
    - add quilt build-depend for patch
    - reformat to comply with upstream, making future merge easier
  * debian/rules
    - add quilt
  * debian/patches/02_fix_arch_specifics.patch
    - patch configure and configure.ac to never use arch specific
      optimizations, really fixes SIGILL on non-pentium4 archs.
      (LP: #351017)

 -- Andreas Moog <email address hidden> Sat, 25 Apr 2009 15:17:36 +0200

I installed mjpegtools 1:1.9.0-0.0ubuntu2 from jaunty-proposed. jpeg2yuv now works on AMD Athlon XP, but mpeg2enc still returns illegal instruction:

mpeg2enc -n n -f 8 -a 2 -o "/tmp/qdvdtest/Main Menu VMGM/menu.m2v"
   INFO: [mpeg2enc] SETTING EXTENDED MMX for MOTION!
   INFO: [mpeg2enc] SETTING SSE and MMX for TRANSFORM!
Illegal instruction

It sounds like this might be fixed with 1.9.0-0.0ubuntu3, so can this be added to proposed for testing?

I'm using xubuntu 9.04 on an AMD XP 2600 with 1 gig ram.

I can confirm that this is still a problem as of today. I re-installed through synaptic (1:1.9.0-0) and even recompiled mjpegtools-1.9.0 from source with:
cd /opt/software/mjpegtools-1.9.0
./configure
./make
sudo make install

Did I do something incorrectly or is there still a problem?

wperz@wperz-desktop:~/work$ jpeg2yuv -n 30 -I p -f 29.97 -j "/tmp/Hells.Kitchen.S01D01/Main Menu VMGM/background.jpg" 2>/dev/null > tmp.yuv
wperz@wperz-desktop:~/work$ ls -l *yuv
-rw-r--r-- 1 wperz wperz 15552229 2009-05-19 22:45 tmp.yuv
wperz@wperz-desktop:~/work$ mpeg2enc -n n -f 8 -a 3 tmp.yuv -o "/tmp/Hells.Kitchen.S01D01/Main Menu VMGM/menu.m2v"
   INFO: [mpeg2enc] SETTING EXTENDED MMX for MOTION!
   INFO: [mpeg2enc] SETTING SSE and MMX for TRANSFORM!
Illegal instruction

Help?

I'm one admin of the mjpegtools project on SF.

From all the posts I'm nor sour if the problem is caused by the mjpegtools package. Anf from reading alle the posts I'm not sour either.

I have a report from one use wher compiling from source solved the problem.

I have updated to mjpegtools from -proposed repository (1:1.9.0-0.0ubuntu2), and I am still getting the SIGILL from mpeg2enc.
In gdb I get:

Starting program: /usr/bin/mpeg2enc -a 2 -n p -f 8 -o /dev/stdout for_mpeg2enc
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
   INFO: [mpeg2enc] SETTING EXTENDED MMX for MOTION!
   INFO: [mpeg2enc] SETTING SSE and MMX for TRANSFORM!
   INFO: [mpeg2enc] SETTING EXTENDED MMX for PREDICTION!
   INFO: [mpeg2enc] Selecting DVD with dummy navigation packets output profile
   INFO: [mpeg2enc] Progressive input - selecting progressive encoding.
   INFO: [mpeg2enc] Encoding MPEG-2 video to /dev/stdout
   INFO: [mpeg2enc] Horizontal size: 720 pel
   INFO: [mpeg2enc] Vertical size: 576 pel
   INFO: [mpeg2enc] Aspect ratio code: 2 = 4:3 display
   INFO: [mpeg2enc] Frame rate code: 3 = 25.0 (PAL/SECAM VIDEO / converted FILM)
   INFO: [mpeg2enc] Bitrate: 7500 KBit/s
   INFO: [mpeg2enc] Quality factor: 8 (Quantisation = 9) (1=best, 31=worst)
   INFO: [mpeg2enc] Field order for input: none/progressive
   INFO: [mpeg2enc] Sequence unlimited length
   INFO: [mpeg2enc] Search radius: 16
   INFO: [mpeg2enc] DualPrime: no
   INFO: [mpeg2enc] Using one-pass rate controller
   INFO: [mpeg2enc] GOP SIZE RANGE 7 TO 15
   INFO: [mpeg2enc] Setting colour/gamma parameters to "PAL B/G"
   INFO: [mpeg2enc] Progressive format frames = 1
   INFO: [mpeg2enc] Using default unmodified quantization matrices
[New Thread 0xb7c3e6d0 (LWP 29816)]

Program received signal SIGILL, Illegal instruction.
[Switching to Thread 0xb7c3e6d0 (LWP 29816)]
0xb7ef850f in EncoderParams::InitQuantMatrices () from /usr/lib/libmpeg2encpp-1.9.so.0
(gdb) bt
#0 0xb7ef850f in EncoderParams::InitQuantMatrices () from /usr/lib/libmpeg2encpp-1.9.so.0
#1 0xb7ef931f in EncoderParams::Init () from /usr/lib/libmpeg2encpp-1.9.so.0
#2 0x0804ac3b in ?? ()
#3 0x0804ba7a in ?? ()
#4 0xb7c55775 in __libc_start_main () from /lib/tls/i686/cmov/libc.so.6
#5 0x08049ae1 in ?? ()
(gdb)

Seems a bit different from the others here.

I also have a crash with the jaunty-proposed version of (lib)mjpegtools.
jpeg2yuv now works well, but mpeg2enc still crash :

$ dpkg -l|grep mjpegto
ii libmjpegtools-1.9 1:1.9.0-0.0ubuntu2 MJPEG video capture/editting/playback MPEG e
ii mjpegtools 1:1.9.0-0.0ubuntu2 MJPEG video capture/editting/playback MPEG e

$ mpeg2enc -n p -f 8 -a 2 -o "/tmp/test.m2v" < /tmp/test.yuv
   INFO: [mpeg2enc] SETTING EXTENDED MMX for MOTION!
   INFO: [mpeg2enc] SETTING SSE and MMX for TRANSFORM!
   INFO: [mpeg2enc] SETTING EXTENDED MMX for PREDICTION!
   INFO: [mpeg2enc] Selecting DVD with dummy navigation packets output profile
   INFO: [mpeg2enc] Progressive input - selecting progressive encoding.
   INFO: [mpeg2enc] Encoding MPEG-2 video to /tmp/Unnamed/Main Menu VMGM/menu.m2v
   INFO: [mpeg2enc] Horizontal size: 720 pel
   INFO: [mpeg2enc] Vertical size: 576 pel
   INFO: [mpeg2enc] Aspect ratio code: 2 = 4:3 display
   INFO: [mpeg2enc] Frame rate code: 3 = 25.0 (PAL/SECAM VIDEO / converted FILM)
   INFO: [mpeg2enc] Bitrate: 7500 KBit/s
   INFO: [mpeg2enc] Quality factor: 8 (Quantisation = 9) (1=best, 31=worst)
   INFO: [mpeg2enc] Field order for input: none/progressive
   INFO: [mpeg2enc] Sequence unlimited length
   INFO: [mpeg2enc] Search radius: 16
   INFO: [mpeg2enc] DualPrime: no
   INFO: [mpeg2enc] Using one-pass rate controller
   INFO: [mpeg2enc] GOP SIZE RANGE 7 TO 15
   INFO: [mpeg2enc] Setting colour/gamma parameters to "PAL B/G"
   INFO: [mpeg2enc] Progressive format frames = 1
   INFO: [mpeg2enc] Using default unmodified quantization matrices
Instruction non permise

$ grep name /proc/cpuinfo
model name : AMD Athlon(tm) XP 1600+

with a recompiled package, everything is fine. configure gave me : -march=athlon-xp -mno-sse2 -mtune=athlon-xp

Hi,
I also have the same kind of crashes with the mjpegtools, but not only with mpeg2enc.

Precisely, the utility ppmtoy4m crashes in the same way. I noticed, because this utility is used frequently by the DVD authoring tools (QDVDAuthor, KmediaFactory).

Here I cp all the info I have:

$ uname -srm
Linux 2.6.28-13-generic i686

$ grep name /proc/cpuinfo
model name : AMD Athlon(tm) XP 2600+

$ ppmtoy4m image001.pnm -n 675 -r -S 420mpeg2 -F 25:1 -A 59:54 -I t -L >out.y4m
   INFO: [ppmtoy4m] Command-line Parameters:
   INFO: [ppmtoy4m] framerate: 25:1
   INFO: [ppmtoy4m] pixel aspect ratio: 59:54
   INFO: [ppmtoy4m] pixel packing: RGB
   INFO: [ppmtoy4m] interlace: top-field-first (interleaved PPM input)
   INFO: [ppmtoy4m] starting frame: 0
   INFO: [ppmtoy4m] # of frames: 675, repeat last frame until done
   INFO: [ppmtoy4m] chroma subsampling: 4:2:0 MPEG-2 (horiz. cositing)
   INFO: [ppmtoy4m] Output Stream parameters:
   INFO: [ppmtoy4m] frame size: 800x600 pixels (720000 bytes)
   INFO: [ppmtoy4m] chroma: 4:2:0 MPEG-2 (horiz. cositing)
   INFO: [ppmtoy4m] frame rate: 25/1 fps (~25.000000)
   INFO: [ppmtoy4m] interlace: top-field-first
   INFO: [ppmtoy4m] sample aspect ratio: 59:54
Illegal instruction

Debugging information

$ gdb ppmtoy4m
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
(no debugging symbols found)
(gdb) r image001.pnm -n 675 -r -S 420mpeg2 -F 25:1 -A 59:54 -I t -L >out.y4m
Starting program: /usr/bin/ppmtoy4m image001.pnm -n 675 -r -S 420mpeg2 -F 25:1 -A 59:54 -I t -L >out.y4m
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
(no debugging symbols found)
   INFO: [ppmtoy4m] Command-line...

This bug was fixed in the package mjpegtools - 1:1.9.0-0.5ubuntu1

---------------
mjpegtools (1:1.9.0-0.5ubuntu1) karmic; urgency=low

  * merge from debian-multimedia, remaining changes:
  * debian/patches/02_fix_arch_specifics.patch
    - patch configure and configure.ac to never use arch specific
      optimizations, fixes SIGILL on non-pentium4 archs.
      (LP: #351017)

mjpegtools (1:1.9.0-0.5) unstable; urgency=low

  * New patch to fix a memory leak in png2yuv.

mjpegtools (1:1.9.0-0.4) unstable; urgency=low

  * Unset CFLAGS (fix mipsel build).
  * Replace toolame by twolame. Only mpegtranscode and lav2mpeg use toolame.

mjpegtools (1:1.9.0-0.3) unstable; urgency=low

  * Build powerpc packages without altivec support.

mjpegtools (1:1.9.0-0.2) unstable; urgency=low

  * New patch 01_cpu-detection to compile with -march and -mtune set to i486.

mjpegtools (1:1.9.0-0.1) unstable; urgency=low

  * Upload in unstable.

 -- Andreas Moog <email address hidden> Sat, 18 Jul 2009 14:39:52 +0200

This has been released into Karmic, but there has been no fix for Jaunty users.

As this is a bug impacting a lot of Athlon users will there be a fix released for Jaunty?

Hi,

FYI, I installed (manually) the debian i386 packages from http://www.debian-multimedia.org/pool/main/m/mjpegtools/mjpegtools_1.9.0-0.5_i386.deb and http://www.debian-multimedia.org/pool/main/m/mjpegtools/libmjpegtools-1.9_1.9.0-0.5_i386.deb in my Kubuntu Jaunty, and I have tested that the bugs have been solved correctly.

I think a backport to Jaunty is really desirable.

accepted into jaunty-proposed, please test.

How long should this take to appear in the repository?

Currently the most recent version in the repository is

mjpegtools_1.9.0-0.0ubuntu3_i386.deb

This also shows up if you enable the jaunty-proposed repository

I've tried with this update and it still has the same Illegal instruction when processing samples.

Ok something a bit odd here.

I removed mjpegtools and then re-installed.

This picked up mjpegtools_1.9.0-0.0ubuntu3_i386.deb again but now it appears to work.

Can confirm this.
Updated from mjpegtools_1.9.0-0.0ubuntu2_i386.deb (which gave "Illegal instruction" on using mplex) to mjpegtools_1.9.0-0.0ubuntu3_i386.deb and now (at least) mplex works smoothly.

[Ubuntu Jaunty,
Athlon XP 1700+,
Linux 2.6.28-13-generic #45-Ubuntu SMP Tue Jun 30 19:49:51 UTC 2009]

Hello - need help (beginner)

I have the same problem with ppmtoy4m -> Illegal instruction (on my Athlon XP2000+)

but I still don't know what and HAW to do.

1. I had reinstall mjpegtools with Synapptic - no effect (marking package to reinstall + apply)

2. I can't find out, which version of package I have:

- in Synaptic and also with dpkg-query -s mjpegtools - there is just information

"Version: 1:1.9.0-0.0"

also is it ??? mjpegtools_1.9.0-0.0ubuntu3_i386 or mjpegtools_1.9.0-0.0ubuntu2_i386 ???
I don't know.

Can You give me PLEASE detailed instruction, what to do to install updated package mjpegtools_1.9.0-0.0ubuntu2_i386 ?

Thank You

This bug was fixed in the package mjpegtools - 1:1.9.0-0.0ubuntu3

---------------
mjpegtools (1:1.9.0-0.0ubuntu3) jaunty-proposed; urgency=low

  * debian/control
    - add quilt build-depend for patch
    - reformat to comply with upstream, making future merge easier
  * debian/rules
    - add quilt
  * debian/patches/02_fix_arch_specifics.patch
    - patch configure and configure.ac to never use arch specific
      optimizations, really fixes SIGILL on non-pentium4 archs.
      (LP: #351017)

 -- Andreas Moog <email address hidden> Sat, 25 Apr 2009 15:17:36 +0200