GCC emits 3DNow!-specific instruction for __builtin_prefetch

I'm currently trying to build with debugging symbols to see if it might make things a bit more obvious, but mysqld is actually crashing in rec_copy_prefix_to_buf().

Just for fun I tried running Debian's binary, since it's being built from what appears to be the same codebase with the same patch applied, and it didn't crash.

A gdb backtrace with debugging information:

Program received signal SIGILL, Illegal instruction.
[Switching to Thread 47143187332656 (LWP 27473)]
rec_copy_prefix_to_buf (rec=0x2ae06257408d "SYS_FOREIGN", index=0x2ae06231c4b8, n_fields=1, buf=0x7fff4ab2bda0, buf_size=0x7fff4ab2bda8) at rem0rec.c:1139
1139 UNIV_PREFETCH_RW(*buf);
Current language: auto; currently c
(gdb) bt
#0 rec_copy_prefix_to_buf (rec=0x2ae06257408d "SYS_FOREIGN", index=0x2ae06231c4b8, n_fields=1, buf=0x7fff4ab2bda0, buf_size=0x7fff4ab2bda8)
    at rem0rec.c:1139
#1 0x000000000071dd80 in dict_tree_copy_rec_order_prefix (tree=0x2ae06231d0b8, rec=0x2ae06257408d "SYS_FOREIGN", n_fields=0x7fff4ab2bd58,
    buf=0x7fff4ab2bda0, buf_size=0x7fff4ab2bda8) at dict0dict.c:3961
#2 0x00000000007774fd in btr_pcur_store_position (cursor=0x7fff4ab2bcd0, mtr=<value optimized out>) at btr0pcur.c:136
#3 0x0000000000729f32 in dict_check_tablespaces_and_store_max_id (in_crash_recovery=0) at dict0load.c:278
#4 0x000000000071501b in innobase_start_or_create_for_mysql () at srv0start.c:1488
#5 0x0000000000676d05 in innobase_init () at ha_innodb.cc:1417
#6 0x000000000066b5d7 in ha_init () at handler.cc:485
#7 0x00000000005a6679 in init_server_components () at mysqld.cc:3099
#8 0x00000000005aa187 in main (argc=<value optimized out>, argv=<value optimized out>) at mysqld.cc:3425

I can confirm this bug. I've been tearing my hair out for a little while trying to figure out why a reboot screwed up the mysql server.

It does indeed work on the inital run, but fails after that. This is on amd64.

Same problem here after an upgrade from 6.04 to 6.10. Also on AMD64 install.

There are more people with this problem:
http://www.ubuntuforums.org/showthread.php?t=284950

 I have the same problem on IBM server 2x XEON - kernel-amd64 . Temporary I forced version from dapper to work Mysql .

I'm wondering, are all of you who are having problems using an intel chip or are some of you using AMD? MySQL works just fine on my home and work machines which are Athlon 64s running Ubuntu AMD64. My Intel based server is the only one having the problem.

I am not using the AMD, but Intel Pentium D.

Als Intel Pentium

Same issue - with Intel Pentium 4.

I have the same issue too. I recently upgraded from dapper to edgy and mysql refused to start. I too have intel 64 bit processor. It seems that the problem is a architecture dependent one.

Not to post a me too or anything, but I am having the exact same issue, also on an intel 64 bit processor.

well if its about the problem i think i said what i know to strengthen the topic. If its about the language i correct the mistake.

I can confirm this bug, also using an IBM Xeon server.

 I have same issue
on Dell Intel Xeon IA64 (dual CPU) + Ubuntu 6.10 Edgy Eft

=== /var/log/daemon.log ======

Nov 15 21:20:57 ubuntu-dell mysqld_safe[3540]: started
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: mysqld got signal 4;
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: This could be because you hit a bug. It is also possible that this binary
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: or one of the libraries it was linked against is corrupt, improperly built,
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: or misconfigured. This error can also be caused by malfunctioning hardware.
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: We will try our best to scrape up some info that will hopefully help diagnose
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: the problem, but since we have already crashed, something is definitely wrong
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: and this may fail.
Nov 15 21:20:57 ubuntu-dell mysqld[3543]:
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: key_buffer_size=0
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: read_buffer_size=131072
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: max_used_connections=0
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: max_connections=100
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: threads_connected=0
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: It is possible that mysqld could use up to
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 217599 K
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: bytes of memory
Nov 15 21:20:57 ubuntu-dell mysqld[3543]: Hope that's ok; if not, decrease some variables in the equation.
Nov 15 21:20:57 ubuntu-dell mysqld[3543]:
Nov 15 21:20:57 ubuntu-dell mysqld_safe[3549]: ended
Nov 15 21:20:57 ubuntu-dell ntpdate[2957]: step time server 82.211.81.145 offset -0.823444 sec
Nov 15 21:21:10 ubuntu-dell /etc/init.d/mysql[3684]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
Nov 15 21:21:10 ubuntu-dell /etc/init.d/mysql[3684]: ^G/usr/bin/mysqladmin: connect to server at 'localhost' failed
Nov 15 21:21:10 ubuntu-dell /etc/init.d/mysql[3684]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)'
Nov 15 21:21:10 ubuntu-dell /etc/init.d/mysql[3684]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!

This is a GCC bug, it emits the 3DNow! instruction prefetchw instead of the generic prefetcht0 for __builtin_prefetch.

See http://gcc.gnu.org/bugzilla/show_bug.cgi?id=29833 for more info.

I'm getting the same on the same hardware...

dual Xeons...

installs without error, but even an immediate restart spits the above error to daemon.log

Hi,

Using this simple test program:
int
main (int argc, char *argv[])
{
  int i;
  int test[8];

  __builtin_prefetch (test, 1);
  for (i = 0; i < 8; i++)
    {
      test[i] = i;
    }
  return 0;
}

And compiling using gcc -S test.c, the generated .s file includes the following assembly code:
.LCFI1:
        movl %edi, -52(%rbp)
        movq %rsi, -64(%rbp)
        leaq -48(%rbp), %rax
        prefetchw (%rax)
        movl $0, -4(%rbp)
        jmp .L2
.L3:

prefetchw is a 3Dnow! instruction, not available on EMT64.

Thanks for the test case Lucas. Although it's probably obvious, I just verified that your test application generates an illegal instruction error on the original machine I reported the bug for.

Can anyone provide some details on the status of this bug? Is it just a matter of rebuilding the package with a different switch (which seems like a simple fix) or is it really dependent on an upstream package change? Thanks!

fixed in 4.1.1-21ubuntu1

4.1.1-21ubuntu1 is only available in feisty so far. This is a fundamental flaw, so when will this be fixed in edgy?

What other packages were compiled with the buggy compiler? How can I be sure that other packages don't start crashing on me because of that?

Is there a special process we need to go through to request a backport for this? MySQL as packaged in the repository is still nonfunctional on Intel's xeon cpus.

I've also got problem with MySQL on EMT64, I've reported it as #76858
Also the mentioned test program generates on SIGILL on that machine, but not on AMD64. I think it's a fatal bug, since it renders "edgy server" unusable on EMT64 (well at least if you want to use MySQL). Also, other packages may be affected as well compiled with this gcc in the distributin, I think ...

Greg: The backport process is here: https://wiki.ubuntu.com/StableReleaseUpdates

Proposing a SRU for edgy (mysql server doesn't run on EMT64 hardware), fixed on feisty with a mysql rebuild with a fixed compiler. Choosing for edgy a minimal fix to just rebuild the mysql package.

mysql-dfsg-5.0 (5.0.24a-9ubuntu1) edgy-proposed; urgency=low

  * On amd64, build using -march=x86-64, avoiding 3Dnow! instructions.
    Ubuntu #66702.

 -- Matthias Klose <email address hidden> Thu, 15 Feb 2007 17:25:22 +0000

packages for amd64 can be found at

  http://people.ubuntu.com/~doko/mysql-edgy/

validated that the amd64 specific instruction is not generated.

Matthias, this looks fine. Please go ahead and upload to edgy-proposed.

Does this affect mysql in Feisty? Please update the feisty task status accordingly.

Do we need to fix gcc-4.1 in edgy? Please update the task status.

Thanks!

mysql-dfsg-5.0 fixed in feisty.

Martin, in Oslo the SRU team decided not to upload gcc-4.1 to edgy.

Accepted mysql edgy-proposed upload, please go ahead with QA testing.

Closing gcc-4.1/edgy task as per Matthias' comment.

The package is now available in edgy-proposed. QA-Team notified via sflaw; I don't own EMT64 hardware, so I cannot test it myself.

I installed the package from edgy-proposed and it has addressed the original problem of Mysql not starting. So far mysqld is running smoothly. Thank you all very much for getting this fixed.

yes! works for me now too! thanks a lot

ubuntu-sru, please lets update that for edgy-updates

Matthias, patch looks fine. Please go ahead and upload.

uploaded; keeping this report still open, until #100083 is commented.

Accepted into edgy-updates, thank you!