Python build from source lack some modules due to non-standard libraries placement

Reproduce: try to install python from sources (after doing `apt-get build-dep python 2.7` of course).
You'll see warnings about missing modules:

Python build finished, but the necessary bits to build these modules were not found:
_bsddb _sqlite3 bsddb185
dbm dl gdbm
imageop sunaudiodev zlib
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

Failed to build these modules:
crypt nis

Please importance critical, as you'll lose half of your archive on next rebuild.
BTW, there's build failure with setuptools:

Traceback (most recent call last):
  File "setup.py", line 94, in <module>
    scripts = scripts,
  File "/tmp/ibd3-1/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/tmp/ibd3-1/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/tmp/ibd3-1/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/tmp/ibd3-setup-bJCwNZ/setuptools-0.6c11/setuptools/command/install.py", line 76, in run
    self.do_egg_install()
  File "/tmp/ibd3-setup-bJCwNZ/setuptools-0.6c11/setuptools/command/install.py", line 96, in do_egg_install
    self.run_command('bdist_egg')
  File "/tmp/ibd3-1/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/tmp/ibd3-1/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/tmp/ibd3-setup-bJCwNZ/setuptools-0.6c11/setuptools/command/bdist_egg.py", line 236, in run
    dry_run=self.dry_run, mode=self.gen_header())
  File "/tmp/ibd3-setup-bJCwNZ/setuptools-0.6c11/setuptools/command/bdist_egg.py", line 527, in make_zipfile
    z = zipfile.ZipFile(zip_filename, mode, compression=compression)
  File "/tmp/ibd3-1/lib/python2.7/zipfile.py", line 679, in __init__
    "Compression requires the (missing) zlib module"
RuntimeError: Compression requires the (missing) zlib module

The change in location of the libraries is an intended transition; marking as invalid for those tasks. The focus is on getting python to build against the new location.

And what the hell should I do with the official Python distribution???

For my job I have to run bootstrap script that has to install custom Python. It just cannot be done with Natty. So I have to either manually setup symlinks or downgrade.

Sorry for being rude, but the possibility for working under Natty is at risk for me, and I'm quite used to Ubuntu=)

The patch in the linked branch corrects the previous deb-setup patch and gives a correct build in natty again with the same set of modules.

For custom python builds that don't use the Debian/Ubuntu packaging, I believe it is possible to configure additional library paths using configure options such as --libdir, but I'm afraid I'm not familiar enough with the python build system to speak to this authoritatively.

I guest I can adopt your deb-setup.diff for my purposes... Can it be used with patch utility?

I ended up with the following (leaving here for someone with the same trouble):

--- setup.py.orig 2011-03-23 11:30:58.000000000 +0300
+++ setup.py 2011-03-23 11:34:46.000000000 +0300
@@ -349,6 +349,14 @@
         # Ensure that /usr/local is always used
         add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib')
         add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')
+ try:
+ triplet = os.popen('dpkg-architecture -qDEB_HOST_MULTIARCH').readline()[:-1]
+ except EnvironmentError:
+ pass
+ else:
+ add_dir_to_list(self.compiler.library_dirs, '/usr/lib/%s' % triplet)
+ add_dir_to_list(self.compiler.include_dirs, '/usr/include/%s' % triplet)
+

         # Add paths specified in the environment variables LDFLAGS and
         # CPPFLAGS for header and library files.

Apply:
cd Python-2.7.1
patch ./setup.py /path/to/patches/python-setup.py-natty.diff

This bug was fixed in the package python2.7 - 2.7.1-5ubuntu1

---------------
python2.7 (2.7.1-5ubuntu1) natty; urgency=low

  * debian/patches/deb-setup.diff: DEB_GNU_TYPE was never being set, so
    the expected multiarch path support wasn't working. Consolidate to
    a single call to dpkg-architecture -qDEB_HOST_MULTIARCH in setup.py.
    LP: #738213.
 -- Steve Langasek <email address hidden> Wed, 23 Mar 2011 15:02:47 -0700

This bug was fixed in the package python2.6 - 2.6.6-6ubuntu7

---------------
python2.6 (2.6.6-6ubuntu7) natty; urgency=low

  * Search libraries in multiarch directories too. LP: #738213.
 -- Matthias Klose <doko@timbuktu> Fri, 25 Mar 2011 17:45:24 +0100

This bug was fixed in the package python3.2 - 3.2-1ubuntu1

---------------
python3.2 (3.2-1ubuntu1) natty; urgency=low

  * Search libraries in multiarch directories too. LP: #738213.
 -- Matthias Klose <email address hidden> Fri, 25 Mar 2011 18:00:03 +0100

This bug was fixed in the package python3.1 - 3.1.3-1ubuntu1

---------------
python3.1 (3.1.3-1ubuntu1) natty; urgency=low

  * Search libraries in multiarch directories too. LP: #738213.
 -- Matthias Klose <email address hidden> Fri, 25 Mar 2011 18:05:31 +0100

The question is: what should we do about this upstream?

From the mercurial repo, the following modules fail to build in 2.7:

_bsddb
_sqlite3
bsddb185 (don't care)
dbm
dl
gdbm
imageop (don't care)
sunaudiodev (don't care)
zlib

I don't know if it would be acceptable to include the dpkg-architecture call in upstream, though definitely not as given in deb-setup.diff (since that's too platform specific). I also don't know how far back we can or should fix upstream.

Upstream bug: http://bugs.python.org/issue1294959

Scratch that. Use http://bugs.python.org/issue11715 instead.