Unable to start pump server because of python version mismatch

Bug #511585 reported by Ihor Kaharlichenko
74
This bug affects 22 people
Affects Status Importance Assigned to Milestone
distcc
Invalid
Undecided
Unassigned
distcc (Ubuntu)
Fix Released
Medium
Dustin Kirkland 
Karmic
Won't Fix
Medium
Unassigned
Lucid
Won't Fix
Medium
Unassigned
Maverick
Invalid
Medium
Unassigned
Natty
Fix Released
Medium
Dustin Kirkland 

Bug Description

[Impact]

Using distcc-pump is a significant speed gain with a large compile farm.

Ubuntu users can not use distcc-pump due to problems with python version and module path hardcoding. The upstream build system and Ubuntu packaging attempt various hacks to detect the python version and installed location of include_server modules. These have proved quite unreliable.

Subsequent “fixes” to bump the version only fail later, or sometimes don't work at all (see comments on bug #767066).

Instead of constantly hardcoding newer python versions, distcc/3.1-4.2 includes a fix where the default version is detected in debian/rules and passed along to the build configure as “/usr/bin/pythonN.M”. Further, it also installs the modules to a fixed, private path and bypasses the problems with detecting the installed path during the package build process.

[Test Case]

* Install distcc and distcc-pump packages:

$ sudo apt-get install -y distcc distcc-pump

* Start a local distcc server and setup environment:

$ distccd --daemon --allow 127.0.1.1
$ export DISTCC_HOSTS=localhost/4,cpp

* Attempt to start distcc-pump, observe the the include_server module is not found:

# on lucid:
$ pump --startup
__________Using distcc-pump from /usr/bin
__________Using 3 distcc servers, of which only 2 support(s) pump mode
/usr/bin/python: can't open file '/usr/lib/pymodules/python2.5/include_server/include_server.py': [Errno 2] No such file or directory
__________Expected a socket at '/tmp/distcc-pump.I3DsMc/socket'
__________Could not start distcc-pump include server

# on precise (the command and error message are different):
$ distcc-pump --startup
__________Using 2 distcc servers, of which only 1 support(s) pump mode
/usr/bin/distcc-pump: 538: /usr/bin/find_c_extension.sh: not found
/usr/bin/python: can't open file '/usr/bin/include_server/include_server.py': [Errno 2] No such file or directory
__________Expected a socket at '/tmp/distcc-pump.Hlfnoz/socket'
__________Could not start distcc-pump include server

[Regression Potential]

Although this will still require a rebuild when the default version changes, it will not require any further modifications to the distcc packaging.

Small patch.

The python modules for the include_server were previously installed to a public path. They are now moved to a private path. This is not expected to impact any users, because these modules have never been usable by anything other than the official distcc-pump script. In the extra rare situation that a user has developed some custom program using them, they will have to add the new path (which is fixed and well-defined) to their python load path.

The proposed fix has been in sid since March 2012, and quantal since August. No regressions have been reported.

Related branches

Revision history for this message
Ihor Kaharlichenko (madkinder) wrote :
tags: added: patch
Revision history for this message
Sergii Pylypenko (x-pelya-x) wrote :

Also there in Lucid

Revision history for this message
Michael Casadevall (mcasadevall) wrote :

Ugh, this would fix the issue, but the behavior in Debian of hardcoding in a path is broken by design (especially since Ubuntu and Debian support concurrent installations of python side by side). Looking at Debian, I don't see a specific fix for this bug in the changelog. I'm testing to see if this is an issue in Debian, but I'm flagging the patch needswork as we shouldn't be hardcoding in a python version.

tags: added: patch-needswork
removed: patch
Revision history for this message
Michael Casadevall (mcasadevall) wrote :

I managed to confirm the bug still exists in Lucid. Tasks opened and bug marked as confirmed.

Changed in distcc (Ubuntu Lucid):
status: New → Confirmed
Changed in distcc (Ubuntu Karmic):
importance: Undecided → Medium
Changed in distcc (Ubuntu Maverick):
importance: Undecided → Medium
Changed in distcc (Ubuntu Lucid):
importance: Undecided → Medium
Changed in distcc (Ubuntu Maverick):
status: New → Confirmed
Changed in distcc (Ubuntu Karmic):
status: New → Triaged
Changed in distcc (Ubuntu Lucid):
status: Confirmed → Triaged
Changed in distcc (Ubuntu Maverick):
status: Confirmed → Triaged
Joel Ebel (jbebel)
tags: added: glucid
Revision history for this message
Justin Lebar (jlebar) wrote :

I agree that changing the hardcoded Python version number isn't the Right Way to fix this, but it would be nice if you guys could release a fix for this package while you work on a better solution. It's been broken for six months now.

Revision history for this message
Justin Lebar (jlebar) wrote :

For those looking for a workaround, one was posted in bug 472201: Modify /usr/bin/pump and replace "python2.5" with "python2.6".

Revision history for this message
Ihor Kaharlichenko (madkinder) wrote :

I totally agree with you, guys, - I hate hardcoding myself. But meanwhile shouldn't we have a hackish fix that "just works (tm)" while Debian/Ubuntu Python team proposes a robust policy on how to deal with such problems?

Seen from my perspective ignoring this issue because "hardcoding is bad" is not a way to go either - casual users need distcc to work out of the box regardless of how it is packaged and don't bother creating dirty symlinks/patching the source on their own.

I suggest the following:
1. fix this issue with a hack (either with my patch or any other way)
2. create another one describing the python version hardcoding problem and handle it separately; this issue can address several packages at once (I don't think pump-distcc is that special with this problem)

Meanwhile users can enjoy the power of pupm-distcc.

papukaija (papukaija)
tags: added: karmic lucid maverick
Revision history for this message
marcobra (Marco Braida) (marcobra) wrote :

For those looking for a workaround for Natty: Modify /usr/bin/pump and replace "python2.5" with "python2.7".

tags: added: natty
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Attaching a trivial patch to get this fixed in Natty. (Could be SRU'd easily, too, if anyone wants to run with this).

Changed in distcc (Ubuntu Natty):
status: Triaged → In Progress
assignee: nobody → Dustin Kirkland (kirkland)
Revision history for this message
Dustin Kirkland  (kirkland) wrote :
Changed in distcc (Ubuntu Natty):
status: In Progress → Fix Committed
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Uploaded to Natty. Awaiting release-team approval.

Changed in distcc (Ubuntu Karmic):
status: Triaged → Won't Fix
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package distcc - 3.1-4ubuntu1

---------------
distcc (3.1-4ubuntu1) natty; urgency=low

  * debian/patches: 04_fix_pumps_include_server_path.dpatch, fix determination
    of pymodules path, LP: #511585
 -- Dustin Kirkland <email address hidden> Tue, 19 Apr 2011 10:25:04 -0400

Changed in distcc (Ubuntu Natty):
status: Fix Committed → Fix Released
Revision history for this message
marcobra (Marco Braida) (marcobra) wrote :
Revision history for this message
JC Hulce (soaringsky) wrote :

This bug affects Ubuntu 10.10, Maverick Meerkat. Maverick has reached end-of-life and is no longer supported, so I am closing the bugtask for Maverick. Please upgrade to a newer version of Ubuntu.
More information here: https://lists.ubuntu.com/archives/ubuntu-announce/2012-April/000158.html

Changed in distcc (Ubuntu Maverick):
status: Triaged → Invalid
Revision history for this message
Daniel Hartwig (wigs) wrote :

Only an issue with the packaged software. Installing from source is ok.

Changed in distcc:
status: New → Invalid
Daniel Hartwig (wigs)
description: updated
Daniel Hartwig (wigs)
description: updated
Revision history for this message
Daniel Hartwig (wigs) wrote :

This is the patch used in 3.1-4.2.

Revision history for this message
Rudd-O (rudd-o) wrote :

Still not fixed in precise.

I can't believe this.

Look at the source of the /usr/bin/distcc-pump file:

# This is where include_server.py lives. It's set to the empty string
# before pump is installed into its final location. At install-time,
# the 'make install' command will rewrite this line of pump to point
# to the installed include_server.py.
# NOTE: DO NOT CHANGE THE LINE BELOW WITHOUT CHANGING THE SED IN
# Makefile.in:install-include-server.
include_server=''

why is this empty?

Revision history for this message
Rudd-O (rudd-o) wrote :

This bug is in precise too.

Rudd-O (rudd-o)
tags: added: precise
Revision history for this message
Daniel Hartwig (wigs) wrote : Re: [Bug 511585] Re: Unable to start pump server because of python version mismatch

On 11 December 2012 08:05, Rudd-O <email address hidden> wrote:
> Still not fixed in precise.

Please contact a bug supervisor to nominate this task for precise
then. You can find such people on #ubuntu-bugs.

The description already contains appropriate SRU fields, and the
upstream patch is identified. The upstream patch may not be suitable,
as it changes installed file locations. However, it would be easy
enough to just "update" the hardcoding in this file.

Revision history for this message
Thomas A. F. Thorne (tafthorne) wrote :

Maverick has the bug, it could have been fixed. I think this state should have been "Won't Fix".

I'll see if I can navigate the DistCC project, Debian system and Ubuntu system in the next few weeks. These are all helpful tools for a small/medium business or even any where that has offices of a dozen developers.

Revision history for this message
Thomas A. F. Thorne (tafthorne) wrote :

I am still seeing this bug in what I forgot was a very old version of Mint. My appologies for making noises on an old thread. I can confirm that I have not seen this on my updated 14.04 trusty machine.

Revision history for this message
Rolf Leggewie (r0lf) wrote :

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".

Changed in distcc (Ubuntu Lucid):
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.