We are getting complaints that the Launchpad apport retracers cannot create proper symbolic stack traces for crashes on ARM. It seems gdb-multiarch does not get along with our core dumps on ARM any more. Reproducing the full apport-retrace environment is a bit complicated, so I reduced this to a very simple test case:
Log into an ARM machine, like the Nexus G4, or porter-armhf.canonical.com (shedir), using "schroot -c saucy-armhf".
1. create a core dump of bash:
$ cd /tmp/
$ bash -c 'ulimit -c unlimited; kill -SEGV $$'
2. Run it with gdb:
$ gdb --batch --ex 'file /bin/bash' --ex 'core-file core' --ex 'bt'
Core was generated by `bash -c ulimit -c unlimited; kill -SEGV $$'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007f2eae349257 in kill () at ../sysdeps/unix/syscall-template.S:8181
../sysdeps/unix/syscall-template.S: Datei oder Verzeichnis nicht gefunden.
#0 0x00007f2eae349257 in kill () at ../sysdeps/unix/syscall-template.S:81
#1 0x00000000004418a1 in kill_pid ()
[...]
There are not a lot of symbols as usually one doesn't have bash-dbgsym etc. installed, but it's clearly able to produce a stack trace.
3. Run the same with gdb-multiarch:
$ gdb-multiarch --batch --ex 'file /bin/bash' --ex 'core-file core' --ex 'bt'
warning: wrong size gregset struct in core file
Core was generated by `bash -c ulimit -c unlimited; kill -SEGV $$'.
Program terminated with signal 11, Segmentation fault.
warning: wrong size gregset struct in core file
#0 <unavailable> in ?? ()
#0 <unavailable> in ?? ()
PC not available
FTR, Apport uses gdb-multiarch on amd64, and --ex 'set architecture arm' --ex 'set gnutarget elf32-littlearm' to generate stack traces for ARM coredumps on x86 (using binaries/libaries/debug symbols for ARM that get unpacked into a temporary directory). The full invocation looks something like this:
gdb-multiarch --ex 'set architecture arm' --ex 'set gnutarget elf32-littlearm' --ex 'set debug-file-directory /tmp/s/usr/lib/debug' --ex 'set solib-absolute-prefix /tmp/s' --ex 'file /tmp/s/bin/bash' --ex 'core-file CoreDump' --ex 'bt full' --batch
But as gdb-multiarch does not even work on a native ARM machine for a native ARM core dump of the very same machine/OS, I guess it's something more fundamental which got broken there.
The same test case on amd64 works fine BTW, i. e. gdb-multiarch can process amd64 binaries on amd64.
I checked again when this started:
- precise (7.4-2012.
- quantal/raring (7.5-0ubuntu2): broken
- saucy (7.6-5ubuntu2): broken
So it seems this started failing with 7.5?