wrong quoting in aclocal's python.m4 check

this is by design. in python2.5, the directory /usr/local/lib/python2.5/site-packages was the standard site-packages of an default local python installation (which goes into /usr/local). with the new behaviour, locally installed packages installed with the "system" python go into /usr/local/lib/python2.6/dist-packages.

If you do want to have the old behaviour, move all stuff in /usr/local/lib/python2.6/dist-packages to /usr/local/lib/python2.6/site-packages and make /usr/local/lib/python2.6/dist-packages a symlink to /usr/local/lib/python2.6/site-packages.

it seems that this change has caused some trouble in autotools (m4.python maybe, but that is a wild guess)

i'll try to describe it in detail:
app that has suffered is hamster-applet [1]

when ./autogen.sh is run and autotools are called, python extension module is detected wrongly:

checking for python script directory... ${prefix}/lib/python2.6/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.6/site-packages

but
python -c "import sys; print sys.path"

is looking in /usr/lib/python2.6/dist-packages

which results, as one would expect, failure

I'm no expert in autotools, so would appreciate some feedback or verification that indeed is a bug.

[1] http://projecthamster.wordpress.com/

the aclocal check is broken.

checking for python platform... linux2
+ printf '%s\n' 'configure:14520: checking for python script directory'
+ printf %s 'checking for python script directory... '
checking for python script directory... + test '' = set
+ echo 'from distutils import sysconfig; print sysconfig.get_python_lib(0,0,prefix='\''${prefix}'\'')'
from distutils import sysconfig; print sysconfig.get_python_lib(0,0,prefix='${prefix}')
++ /usr/bin/python -c 'from distutils import sysconfig; print sysconfig.get_python_lib(0,0,prefix='\''${prefix}'\'')'
XXX: ${prefix} False
+ echo '${prefix}/lib/python2.6/site-packages'
${prefix}/lib/python2.6/site-packages
++ /usr/bin/python -c 'from distutils import sysconfig; print sysconfig.get_python_lib(0,0,prefix='\''${prefix}'\'')'
+ am_cv_python_pythondir='${prefix}/lib/python2.6/site-packages'
+ printf '%s\n' 'configure:14530: result: ${prefix}/lib/python2.6/site-packages'
+ printf '%s\n' '${prefix}/lib/python2.6/site-packages'
${prefix}/lib/python2.6/site-packages
+ pythondir='${prefix}/lib/python2.6/site-packages'
+ pkgpythondir='${pythondir}/hamster-applet'
+ set +x

the XXX line just prints out the prefix argument from get_python_lib. so automake does get the quoting wrong, which isn't noticed as long as we have some default script dir.

well, we could hack get_python_lib to interpret '${prefix}' ...

> well, we could hack get_python_lib to interpret '${prefix}' ...

no, doesn't work, because we cannot decide in get_python_lib() if this should expand into /usr or /usr/local, but maybe /usr/local would be the safe choice.

This bug was fixed in the package python2.6 - 2.6.2~rc1-0ubuntu1

---------------
python2.6 (2.6.2~rc1-0ubuntu1) jaunty; urgency=low

  * Update to the 2.6.2 release candidate.
  * Work around an automake m4 expansion bug ('${prefix}' and '${exec_prefix}'
    are passed unexpanded to get_python_lib()). Assume in this case, that the
    expansion is '/usr/local', it's not safe to assume '/usr' in this case.
    This is really, really ugly, just hoping that the automake maintainers are
    punished for this bug by one hell level deeper than me for this hack.
    LP: #350016.

 -- Matthias Klose <email address hidden> Wed, 08 Apr 2009 02:29:05 +0200

Thanks, this fixes the problem with "site-packages" vs "dist-packages",

but what about those cases when the prefix has been given when calling autotools
./autogen.sh --prefix=/usr/

now the script seems to ignore the setting and goes to /usr/local/ in all cases

@doko:

We need to back this workaround out and figure out an alternative solution....if one exists.

I confirmed that this modification install now python bindings in /usr/local/lib when you expect them in /usr/lib. You can take for examples avant-window-navigator and awn-extras-applets packages, which build python bindings. The .install files wait the .so in /usr/lib but the installation want to put it in /usr/local/lib, so FTBFS with the same sources.
I'm sure others packages are affected if you want to rebuild them.

Yes Julien, I can't update gnome-panels to 2.26.1 because of the same issue.

The buggy python change has been reverted with the upload of python2.6 2.6.2~rc1-0ubuntu2. Changelog:

python2.6 (2.6.2~rc1-0ubuntu2) jaunty; urgency=low

  * Update to 20090414, taken from the 2.6 release branch.
    - No numbered table of contents for the docs, not in sphinx-0.5.
    - Fix issues #5731, #5741.
  * Revert the change to interpret an unexpanded prefix in get_python_lib().
  * Fix build failure on ia64.
  * Move the bdist_wininst files into the -dev package (only needed to build
    windows installers).
  * Document changes to the site directory name in the installation manual.

This bug was fixed in the package automake1.10 - 1:1.10.2-0ubuntu3

---------------
automake1.10 (1:1.10.2-0ubuntu3) jaunty; urgency=low

  * python.m4: Don't call the get_python_lib() with the unexpanded arguments.
    LP: #350016.

 -- Matthias Klose <email address hidden> Tue, 14 Apr 2009 08:53:28 +0200

This bug was fixed in the package automake1.9 - 1.9.6+nogfdl-3ubuntu2

---------------
automake1.9 (1.9.6+nogfdl-3ubuntu2) jaunty; urgency=low

  * python.m4: Don't call the get_python_lib() with the unexpanded arguments.
    LP: #350016.

 -- Matthias Klose <email address hidden> Tue, 14 Apr 2009 09:06:15 +0200

As far as I can tell, the manifestation of this issue hasn't been fixed.

>>> print sys.path
['', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/Numeric', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/python2.6/dist-packages/gst-0.10', '/var/lib/python-support/python2.6', '/usr/lib/python2.6/dist-packages/gtk-2.0', '/var/lib/python-support/python2.6/gtk-2.0', '/usr/local/lib/python2.6/dist-packages']

ezyang@javelin:~$ aptitude show automake
Package: automake
State: installed
Automatically installed: yes
Version: 1:1.10.2-0ubuntu3

ezyang@javelin:~$ aptitude show python2.6
Package: python2.6
New: yes
State: installed
Automatically installed: no
Version: 2.6.2-0ubuntu1

Maybe Python needs to be repackaged?

Its still happening. I just upgraded from intrpid to jaunty and all python apps no need a manual
PYTHONPATH=/usr/lib/python2.6/site-packages

Is there a preferred workaround for users who wish to install Python modules from source like this:

  sudo python setup.py install --prefix=/usr/local

Even after setting and exporting PYTHONPATH to be '/usr/local/lib/python2.6/site-packages' (which exists) this failed with an error message that claims I have a "bad install directory or PYTHONPATH".

Getting this same problem in labyrinth... LP: #327174

Configure output during build:

checking for python version... 2.6
checking for python platform... linux2
checking for python script directory... ${prefix}/lib/python2.6/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.6/site-packages

python -c "import sys; print sys.path" output:

['', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/python2.6/dist-packages/gst-0.10', '/usr/lib/pymodules/python2.6', '/usr/lib/python2.6/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.6/gtk-2.0', '/usr/lib/python2.6/dist-packages/wx-2.8-gtk2-unicode', '/usr/local/lib/python2.6/dist-packages']