mpicc (openmpi) broken in Ubuntu 7.10 RC

I can add that mpif77 does nothing when trying to compile any sort of fortran file using 7.10 (official release) fresh install, and all the openmpi* packages installed with gcc/gfortran 4.2 working okay. AMD64. This is horrible.

This is still broken in 7.10 release.

I can confirm that this problem persists in 7.10. It looks like something went wrong with the configure process at build time: strace reveals that mpicc and friends attempt to read "${prefix}/share/openmpi/mpicc-wrapper-data.txt". The work-around I've used is to open up these wrapper programs with a hex editor and edit the offending string in place, replacing it with "/usr/share/openmpi/mpicc-wrapper-data.txt\0\0\0\0\0" in the mpicc case for example.

I can confirm that this is broken in 7.10

mpirun has same bug. It is unable to find files from /usr/share/openmpi/
Please fix.

Confirmed for Xubuntu 7.10 amd64.
mpif90, mpicc, mpicxx, all do not work and strace shows the "${prefix}".
I can not confirm russian_knight's bug of mpirun, as it runs all right here after fixing the compilers.

halfflat's workaround worked fine for me. This way I could fix the 3 mpi* and I am sure this should work for other versions, too.
I had to try a bit to get it working, so here is a bit more detailed description of the workaround for those who are as unexperienced as me:

First of all it is important to use an editor that really just manipulates single bytes, as a hex editor does.
strace showed

"${prefix}/share/openmpi/mpicc-wrapper-data.txt"

but the file contains

${prefix}/share/openmpi.%s%s%s-wrapper-data.txt

I just had to change the first part and make it look like

/usr/share/openmpi

Because the "${prefix}" takes 9 Bytes and "/usr" just 4 I had to overwrite the 5 following bytes after the "openmpi" with "00".
Thanks halfflat for that hint.

About mpirun. Catch:

$ strace mpirun 2>&1 | grep open
open("/etc/ld.so.cache", O_RDONLY) = 3
open("/usr/lib/liborte.so.0", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libdl.so.2", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libnsl.so.1", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libutil.so.1", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libm.so.6", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libpthread.so.0", O_RDONLY) = 3
open("/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = 3
open("/usr/lib/libopal.so.0", O_RDONLY) = 3
open("/etc/openmpi/openmpi-mca-params.conf", O_RDONLY) = 3
open("/home/knight/.openmpi/mca-params.conf", O_RDONLY) = -1 ENOENT (No such file or directory)
open("${prefix}/share/openmpi/help-orterun.txt", O_RDONLY) = -1 ENOENT (No such file or directory)
open("${prefix}/share/openmpi/help-orterun.txt.txt", O_RDONLY) = -1 ENOENT (No such file or directory)

So same "prefix" error here.

You are right, sorry.

I thought your mpirun does not work, so I just tried if does. strace shows me the same prefix error, but mpirun works fine, though.

The bug is confirmed.

I fixed it with an hex editor (ghex2), but there is no problem with the ${prefix} thing.
The binariy contains:

${prefix}/share/openmpi.%s%s%s-wrapper-data.txt

i just changed that dot after the "share/openmpi" to a slash, so now it looks like:

${prefix}/share/openmpi/%s%s%s-wrapper-data.txt

And it works fine:
$ mpicc.openmpi
$ gcc: no input files

Sorry, i was wrong in my last post.

The solution mentioned halfflat did not work for me. I have to do the following.

Replace:

${prefix}/share/openmpi.%s%s%s-wrapper-data.txt

for:

/usr/share/openmpi......%s%s%s-wrapper-data.txt

the dot means a zero byte, so there is two strings in that sequence, and the extra 5 bytes have to be added to the end of the first one.
But, the problem continued, because now the binary tried to open /usr/share/openmpi/mpicc.openmpi-wrapper-data.txt
So i smply made a symlink to the real file, without the ".openmpi".
If you use fortran or c++ compilers, the same thing has to be done for them.

Regards and sorry for the mistake before.

And now, it works for real ;-)

Guys, what's wrong with you?
Why hexedit the file, if it is possible for the package maintainer just to fix the damn package.
How many months should it take?

Apparently, at least two months. And it's still not even triaged. Why bother? If you needed MPI by now, you've recompiled openmpi yourself, hex edited it, or switched to a distro that makes sure these things work, like Fedora. I chose the latter route.

On 12/12/07, Brian Barnes <email address hidden> wrote:
>
> Apparently, at least two months. And it's still not even triaged. Why
> bother? If you needed MPI by now, you've recompiled openmpi yourself,
> hex edited it, or switched to a distro that makes sure these things
> work, like Fedora. I chose the latter route.

You are right. There was a time I loved Ubuntu, because it just worked.
Now it is getting more and more like windows. Nice to use for a while, but
afterwards - problem after problem.

Unfortunately I don't like most linux distros and other OSes I've tried
recently.
Everithing becomes worse and worse.

Yavor

The problem seems to be caused by the autoconf patch that has been added by debian
changing debian/patches/00list to remove the mention of the 99_autoconf and removing the 99_autoconf.dpatch file creates a source package that builds correctly

This is a debdiff to remove the 99_autoconf patch

the hardy version of openmpi works correctly, so this debdiff is only to be applied to gutsy

http://wiki.ubuntu.com/StableReleaseUpdates is the process to get changes into a stable release.

Unsubscribing ubuntu-universe-sponsors for now.

Impact of bug : mpicc, mpirun etc are unusable
Already corrected in hardy as hardy uses a different debian version of the openmpi package (1.2-4.3 and not 1.1-2.5)

I've changed the debdiff so that the target is gutsy-proposed in stead of gutsy

Hi,

Can you please provide a justification for the removal of 99_autoconf and why its presence breaks the package?

Thanks,

MOTU SRU team

I've pinpointed the problem to one line in the 99_autoconf patch

these are the changes around line 1760 in the current 99_autoconf.patch file

1759 -datadir='${prefix}/share'
1760 +datarootdir='${prefix}/share'
1761 +datadir='${datarootdir}'

line 1761 should be +datadir='${prefix}/share'

it seems that the dataroordir does not get expanded correctly

I'll attach two new debdiffs with this minor change only

result of debdiff openmpi_1.1-2.5.dsc openmpi_1.1-2.5ubuntu1.dsc > openmpi-1.1-2.5--1.1-2.5ubuntu1.debdiff

changes file

dsc file

Ok, this debdiff looks a lot better. Please update the original description to reflect your discovery in comment #22 and then I'd say it's good enough for upload.

Also, is this fixed in Hardy? If not, debdiffs against Hardy would be needed before Gutsy can be SRU'ed.

Hi John,

  Hardy is OK already as hardy has been synced with a newer debian version that does not have this problem (I believe the 99_autoconf patch disappeared in subsequent versions)

    Koen

Marking Hardy fixed release due to last comment. The description and debdiff proposed here looks reasonable to me, so please proceed to find a sponsor to get this into gutsy-proposed.

How do I find a sponsor ?

Subscribe the "ubuntu-universe-sponsors" team to this bug report and leave it unassigned and Confirmed (as it currently is)

What about Feisty? Could you please check if it is affected too and eventually prepare a debdiff for feisty-proposed?

feisty uses version 1.1-2.3 which doesn't have the 99_autoconf patch that causes problems in gutsy

as far as I know the only ubuntu version that has this problem is gutsy (it's the only version in ubuntu that has this 99_autoconf patch)

Uploaded to gutsy-proposed, now waiting for an archive-admin to accept it.

Accepted into gutsy-proposed, please test.

Hi,
Seems to work using openmpi 1.1-2.5ubuntu1 packages from gutsy-proposed (I'm running Gutsy 7.10):

$ mpicc
gcc: no input files

I did this:
System -> Administration -> Software Sources
Go to Updates tab and check Pre-released updates (gutsy-proposed)
Click on Close and the Reload
System -> Administration -> Synaptic Package Manager
Search for openmpi and then I did a complete removal (Mark for Complete Removal) of all openmpi packages
Apply changes
And then (re)install (I used Synaptic) openmpi-bin, openmpi-common, openmpi-libs0, openmpi-dbg, openmpi-dev

There is probably a cute one-liner for these steps as well.

Cheers,
Johan

It works.

GUTSY VERIFICATION

1.1-2.5:
$ mpicc -showme
$ mpicc -showme:compile
$ mpicc -showme:link
$ ls hello*
hello.c
$ mpicc -Wall -O2 hello.c -o hello
$ ls hello*
hello.c
$

1.1-2.5ubuntu1:
$ mpicc -showme
gcc -I/usr/include/openmpi -pthread -lmpi -lorte -lopal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl
$ mpicc -showme:compile
-I/usr/include/openmpi -pthread
$ mpicc -showme:link
-pthread -lmpi -lorte -lopal -ldl -Wl,--export-dynamic -lnsl -lutil -lm -ldl
$ ls hello*
hello.c
$ mpicc -Wall -O2 hello.c -o hello
$ ls hello*
hello hello.c
$ ./hello
SRU candidate works!
$

It runs fine now.

Copied to gutsy-updates. Thank you!

Hi,

I have updated to the latest version, but I still have problems. After installing openmpi-bin, there is no executable mpicc, only mpicc.openmpi. This executable outputs nothing whatsoever.

ilmar

$dpkg -l | grep openmpi
ii openmpi-bin 1.1-2.5ubuntu1 high performance message passing library
ii openmpi-common 1.1-2.5ubuntu1 high performance message passing library
ii openmpi-dbg 1.1-2.5ubuntu1 high performance message passing library
ii openmpi-dev 1.1-2.5ubuntu1 high performance message passing library
ii openmpi-libs0 1.1-2.5ubuntu1 high performance message passing library

$ mpicc.openmpi
$ mpicc.openmpi -showme
$ mpicc.openmpi -showme:link
$ mpicc.openmpi -showme:compile

Update:
When creating the link
$ cd /usr/bin/
$ sudo ln -s mpicc.openmpi mpicc
 it works as it should. So the problem seems to be that mpicc is not linked to mpicc.openmpi under /usr/bin

ilmar

Sorry for all the posts, but I figured out what the problem was. Or at least how to solve it. Even though I made sure that no other MPI-libraries were installed (LAM, MPICH), they earlier presence contributed to this error. After removing the file /var/lib/dpkg/alternatives/mpicc the installation worked fine.

ilmar

Hi,
yes it works fine.
However, I am trying to install a software for modeling that needs both mpi and hdf5. The last reported lines in the configure file were:
checking for pthread_create in -lpthread... yes
checking for MPI_Init in -lmpich... no
checking for MPI_Init in -lmpi... yes
checking for H5open in -lhdf5... no
configure: error: HDF5 is strongly recommended for paraview support.
                  Re-run configure --with-hdf5=/path/to/hdf5.
                  Re-run configure --without-hdf5 to get rid of this message

This, despite I installed hdf5 after your tip. Perhaps there is a similar bug?
Thanks and sorry for the cross posting

I confirm that tis package stills broken in Hardy.

This is mad. It seems that newer versions of ubuntu are based on previous released versions plenty of bugs. In this way we never will obtain a stable version that make us proud of Linux.
However, and despite of that, I always prefer Linux! I never loss my hopes.
 Gustavo
PS.: the package mpich has the same problem...

With every release, we hope that the problems we are having are solved, but they never are. We keep creating our own packages of, amongst others, openmpi based on Debias Sid. I see now that Gutsy will not see version 1.2.6. This goes for openmpi in general, not any specific bugs.