XEN depends on Python 2.5

I should add that I was using the AMD64 platform and the alternate installer.

It's really unfortunate that this was discovered so late in the release process. The attached debdiff might fix the issue, but I really have no idea what the implications of switching to Python 2.6 will be.

I'm working on a build with that patch in my PPA (http://launchpad.net/~broder/+archive/ppa) right now, but I won't have time to test it myself.

ubuntu-release - only one of the xen-3.3 binary packages is in main (libxen3), and I find it very unlikely that my change will affect that. I really have no sense of the regression potential for the python-xen-3.3 package, though.

After some discussion with Steve Langasek on IRC, he suggested that we're really too close to release to be considering non-trivial changes like this where we don't fully understand the implications, so I'm going to start working on this as an SRU.

Here's a new patch targetted at a post-release SRU. It also fixes prevents this from being a problem in the future by using ${python:Depends} instead of just depending on python.

Build should be running in my PPA any minute now.

Hi Evan,

Yeah I was surprised something this obvious was broken in an RC - but
I guess Xen is not used much with Ubuntu.
Actually that might make me drop Ubuntu for now and stick with Debian
... although that sucks too since Deb is notoriously ancient with
everything.

As an alternative quick fix, can'y you just depend on python AND python2.5?

Regards,
-Jeppe

On Thu, Apr 16, 2009 at 9:53 PM, Evan Broder <email address hidden> wrote:
> After some discussion with Steve Langasek on IRC, he suggested that
> we're really too close to release to be considering non-trivial changes
> like this where we don't fully understand the implications, so I'm going
> to start working on this as an SRU.
>
> Here's a new patch targetted at a post-release SRU. It also fixes
> prevents this from being a problem in the future by using
> ${python:Depends} instead of just depending on python.
>
> Build should be running in my PPA any minute now.
>
> ** Attachment added: "xen-3.3_3.3.0-1ubuntu9.1.debdiff"
>   http://launchpadlibrarian.net/25586832/xen-3.3_3.3.0-1ubuntu9.1.debdiff
>
> --
> XEN depends on Python 2.5
> https://bugs.launchpad.net/bugs/362691
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “xen-3.3” source package in Ubuntu: New
>
> Bug description:
> On a clean Ubuntu 9.04 Desktop installation, I installed ubuntu-xen-desktop.
>
> After installing and rebooting, it still doesn't work because it requires Python 2.5 which was not in the dependency list.
>
> Installing Python 2.5 fixed the initial error (but now I get Bug #349987)
>

As far as I know, anybody who's using Ubuntu as a Xen host is running Hardy instead of the latest release. Note that running Ubuntu as a Xen guest should work fine using the -generic or -server kernels, so long as you're running a Xen 3.3 host.

I will note that since Lenny just released recently, it is still fairly current. If you're concerned about how out of date the release is, running Lenny and then switching back to Ubuntu when dom0 support is reincorporated could be better than using Ubuntu Hardy for now.

As for the quick fix, unfortunately any change requires rebuilding the package, and it would be rebuilt in an environment where the default Python is 2.6, which means that it would build using 2.6. It would be an even more substantial change than the one I've posted to make that not the case, and I would judge it equally risky during the RC period.

Subscribed sponsors for uploading and sru team.

Jeppe -
   I think that the build currently in my PPA should fix this issue (https://launchpad.net/~broder/+archive/ppa, see "Adding a PPA to your Ubuntu repositories" on https://help.launchpad.net/Packaging/PPA for the documentation on how to enable it). Could you test that and see if it works?

Actually, scratch that. That version apparently has no Python dependency at all. Blargh

Ok - using dh_pycentral instead of dh_python seems to have fixed that. The version currently in my PPA depends on python (<< 2.7), python (>= 2.6).

Debdiff attached.

Please add some documentation how to test this, since this is quite an invasive change.

SRU reviewed.

Accepted xen-3.3 into jaunty-proposed-proposed; please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Hmm...having throught through this some, I think I've changed my mind about this patch. I agree that it's invasive, and would argue that it's too invasive for an SRU.

I'll try to get a patch together soon for Karmic (there are some other changes to the Xen packaging that should be incorporated as well). In the mean time, here's another patch for Jaunty that includes the necessary python2.5 dependency, and forces everything to use it.

Does that mean I should remove the current -proposed package with your original patch?

Yes, please. And thanks, as always.

2009-05-06 10:14:41 INFO Removing candidates:
2009-05-06 10:14:41 INFO xen-3.3 3.3.0-1ubuntu9.1 in jaunty
2009-05-06 10:14:41 INFO libxen3 3.3.0-1ubuntu9.1 in jaunty amd64
2009-05-06 10:14:41 INFO libxen3 3.3.0-1ubuntu9.1 in jaunty i386
2009-05-06 10:14:41 INFO libxen3-dev 3.3.0-1ubuntu9.1 in jaunty amd64
2009-05-06 10:14:41 INFO libxen3-dev 3.3.0-1ubuntu9.1 in jaunty i386
2009-05-06 10:14:41 INFO python-xen-3.3 3.3.0-1ubuntu9.1 in jaunty amd64
2009-05-06 10:14:41 INFO python-xen-3.3 3.3.0-1ubuntu9.1 in jaunty i386
2009-05-06 10:14:41 INFO xen-docs-3.3 3.3.0-1ubuntu9.1 in jaunty amd64
2009-05-06 10:14:41 INFO xen-docs-3.3 3.3.0-1ubuntu9.1 in jaunty i386
2009-05-06 10:14:41 INFO xen-hypervisor-3.3 3.3.0-1ubuntu9.1 in jaunty amd64
2009-05-06 10:14:41 INFO xen-hypervisor-3.3 3.3.0-1ubuntu9.1 in jaunty i386
2009-05-06 10:14:41 INFO xen-utils-3.3 3.3.0-1ubuntu9.1 in jaunty amd64
2009-05-06 10:14:41 INFO xen-utils-3.3 3.3.0-1ubuntu9.1 in jaunty i386
2009-05-06 10:14:41 INFO Removed-by: Martin Pitt
2009-05-06 10:14:41 INFO Comment: retracted SRU
2009-05-06 10:14:41 INFO 13 packages successfully removed.

Evan, please rewrite the changelog to not refer to a previous one (since that just got eradicated), document it properly standing on its own. You also need to refer to the bug number.

Also, FWIW, the new patch is much more invasive than the previous one, and much more prone to errors. The first one looked okay, the bug just misses testing instructions.

Here's a new debdiff against the package currently in Jaunty.

As for the invasiveness of the patch, I think it's sort of a means vs. ends type of thing - this one certainly makes more changes to the packaging, but the behavior is closer to how the package was intended to work when it was uploaded - i.e. with /usr/bin/python as 2.5 instead of 2.6.

With the other patch, I could certainly spit out a few commands to test that basic Xen operations still worked, but I am absolutely not comfortable making any assertions that the entire Python interface is as functional as it was under Python 2.5.

As for error verification, I've done some prodding at the test build in my PPA (which should be equivalent to this patch, modulo the changelog alterations). I've checked that all of the pycentral bits install only python2.5 variants, and that the shebang line on everything is either "#!/usr/bin/python2.5" or "#!/usr/bin/env python2.5".

Re: [Xen-devel] Attempt to build xen-unstable on Ubuntu 9.04 Server (Ubuntu Desktop installed)
Friday, May 8, 2009 1:40 PM From: "Boris Derzhavets" <email address hidden>
Add sender to Contacts To: "Alex Zeffertt" <email address hidden>
Cc: "xen-devel" <email address hidden>, "Ian Jackson" <email address hidden>
     Notice, that direct port Xen 3.4 ( with 2.6.30-rc3-tip) to F11 (preview) has been tested with no problems. Just no xen-lib and xen-lib dependent packages: libvirt,libvirt-python,.. is a must . Problem been described seems to me as python 2.6 implementation issue on Ubuntu 9.04 Server.

Boris.

--- On Fri, 5/8/09, Boris Derzhavets <email address hidden> wrote:

    From: Boris Derzhavets <email address hidden>
    Subject: Re: [Xen-devel] Attempt to build xen-unstable on Ubuntu 9.04 Server (Ubuntu Desktop installed) To: "Alex Zeffertt" <email address hidden>
    Cc: "xen-devel" <email address hidden>, "Ian Jackson" <email address hidden>
    Date: Friday, May 8, 2009, 11:45 AM

    Follow your suggestion i was able to build Xen Host on Ubuntu 9.04 Server
    with the most recent Xen-3.4-rc4-pre and 2.6.30-rc3-tip pvops kernel via
    Jeremy's git repo. Due to newly built Xen Host dual boot with similar one on top of Ubuntu 8.10 Server i just copied F10 PV profile to new Xen Host and attempted to run
    F10 PV :-
    # xm create -c f10.pyrun
    Name ID Mem VCPUs State Time(s)
    Domain-0 0 6034 4 r----- 138.5
    F10PV 5 2048 1 --p--- 0.0
    root@ServerJaunty:/home/boris/vm# xm unpause F10PV
    /usr/lib/python2.6/site-packages/xen/util/acmpolicy.py:20: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
      import sha
    # xm unpause F10PV
    PCI: Fatal: No config space access function found
    IO APIC resources could be not be allocated.
    Loading /lib/kbd/keymaps/i386/qwerty/us.map
    root@ServerJaunty:/home/boris/vm# Error: Device 0 (vif) could not be connected. Hotplug scripts not working.
    Then i've tried to comment out vif in profile:
    PCI: Fatal: No config space access function found
    IO APIC resources could be not be allocated.
    Loading /lib/kbd/keymaps/i386/qwerty/us.map
    BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
    IP: [<ffffffffa00010e5>] backend_changed+0x2d5/0x3af [xen_blkfront]
    PGD 7d1a0067 PUD 7d1a1067 PMD 7d1b0067 PTE 0
    Oops: 0000 [1] SMP
    CPU 1
    Modules linked in: xen_blkfront
    Pid: 19, comm: xenwatch Not tainted 2.6.27.19-170.2.35.fc10.x86_64 #1
    RIP: e030:[<ffffffffa00010e5>] [<ffffffffa00010e5>] backend_changed+0x2d5/0x3af [xen_blkfront]
    RSP: e02b:ffff88007f40fdf0 EFLAGS: 00010246
    RAX: 0000000000000000 RBX: ffff88007d09b2c0 RCX: 0000000000000000
    RDX: ffff88007f40fe28 RSI: 0000000000000005 RDI: ffff88007ff5f400
    RBP: ffff88007f40fe80 R08: ffff88007f40fdd0 R09: ffff88007f40fcc0
    R10: 0000000000000001 R11: ffff880000000018 R12: ffff88007ff5f400
    R13: ffff88007d0be000 R14: 000000000000000a R...

Accepted into jaunty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Please get this fixed in Karmic ASAP.

PLease be aware of changeset 19594 in Xen Unstable
http://xenbits.xensource.com/xen-unstable.hg?rev/bf946cc3008b
With this patch applied Xen Unstable been built on top of Ubuntu 9.04 Server with python 2.6 (or python 2.5) now is able to start xend daemon, but appears unable to start any DomU due to "Hotplug scripts failure to work". Kernel 2.6.30-rc3-tip (supporting Xen in Dom0) just sends one bakend driver's message to dmesg report and report output freeze.
I am not python expert,but it seems like Python 2.6 ( 2.5 as well) setup is incorrect on 9.04. All issues described above disappear if Xen Unstable along with kernel 2.6.30-rc3-tip were built on top of Ubuntu 8.10 Server (Python 2.5). Suse's xen-ified kernel 2.6.27.5 behaves absolutely same way
as pvops enabled kernel .

In meantime attempt to install ubuntu-xen-desktop via proposed ( following
instructions) fails.

Could you be more specific about how it fails?

Please, view attachment.

Sorry,
I had to clean up /usr/lib/python2.5/sites-packages
manually and was able to commit Xen 3.3.0 Hypervisor Install:-
root@boris-desktop:/boot# ls -l
total 79624
-rw-r--r-- 1 root root 525592 2009-04-17 07:34 abi-2.6.28-11-generic
-rw-r--r-- 1 root root 524602 2009-05-02 01:44 abi-2.6.28-12-server
-rw-r--r-- 1 root root 90584 2009-04-17 07:34 config-2.6.28-11-generic
-rw-r--r-- 1 root root 90560 2009-05-02 01:44 config-2.6.28-12-server
-rw-r--r-- 1 root root 95380 2009-05-13 20:51 config-2.6.30-rc3-tip
drwxr-xr-x 2 root root 4096 2009-05-14 20:32 grub
-rw-r--r-- 1 root root 47093352 2009-05-13 20:52 initrd-2.6.30-rc3-tip.img
-rw-r--r-- 1 root root 7947390 2009-05-13 17:22 initrd.img-2.6.28-11-generic
-rw-r--r-- 1 root root 7953737 2009-05-14 18:19 initrd.img-2.6.28-12-server
-rw-r--r-- 1 root root 128796 2009-03-27 23:12 memtest86+.bin
-rw-r--r-- 1 root root 1871601 2009-04-17 07:34 System.map-2.6.28-11-generic
-rw-r--r-- 1 root root 1863093 2009-05-02 01:44 System.map-2.6.28-12-server
-rw-r--r-- 1 root root 1945381 2009-05-13 20:51 System.map-2.6.30-rc3-tip
-rw-r--r-- 1 root root 1170 2009-04-17 07:39 vmcoreinfo-2.6.28-11-generic
-rw-r--r-- 1 root root 1169 2009-05-02 01:48 vmcoreinfo-2.6.28-12-server
-rw-r--r-- 1 root root 3522336 2009-04-17 07:34 vmlinuz-2.6.28-11-generic
-rw-r--r-- 1 root root 3506816 2009-05-02 01:44 vmlinuz-2.6.28-12-server
-rw-r--r-- 1 root root 3728672 2009-05-13 20:51 vmlinuz-2.6.30-rc3-tip
-rw-r--r-- 1 root root 470077 2009-05-14 14:06 xen-3.3.gz
I've tried gub entry :-
title Xen 3.3 / Ubuntu 9.04 (2.6.28-12-server)
uuid 3d186041-b458-4eb0-952c-30a273b49100
kernel /boot/xen-3.3.gz
module /boot/vmlinuz-2.6.28-12-server root=/dev/sdb9 ro console=tty0
module /boot/initrd.img-2.6.28-12-server
and obvously failed.
Xen-ified kernel seems to be missing

Seems to be done. Synaptic Manager Screenshot attached.

Loading Suse's 2.6.27.5 under your Xen Hypervisor:-
View : dmesg.log
Dom0 loaded.

Attempt to start xend. Old bug is still here (default path has been changed to dist-packages, what causes this errors) :-

root@boris-desktop:/etc/python2.5# /etc/init.d/xend start
Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
.Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
.Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
.Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
.Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
.Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
.Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
.Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
.Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDaemon.py", line 20, in <module>
    import xen.lowlevel.xc
ImportError: No module named xc
.Traceback (most recent call last):
  File "/usr/sbin/xend", line 44, in <module>
    from xen.xend.server import SrvDaemon
  File "/usr/lib/python2.5/site-packages/xen/xend/serv...

I'm...a bit confused by what your seeing, and by what your setup is.

You're running a Suse domU under a Jaunty dom0?

What was the status of the dom0 before you upgraded to the version that's in jaunty-proposed? Were you running the version in jaunty? intrepid?

I just installed a fresh Jaunty machine with xen-3.3 3.3.0-1ubuntu9.2 and was able to create and boot a VM. I generally couldn't find anything wrong. If the upgrade path doesn't work, though, I can see what there is to be done about that.

I was running Jaunty Desktop. After porting Xen 3.3 via proposed i didn't find any kind
of xenified kernel to load Dom0 and at the same time i saw that Hypervisor and tools got
installed . It wasn't surprising after my experience with Ubuntu Intrepid Server. Because that was exactly what happened on Ubuntu 8.10 Server. So i just ported Suse's 2.6.27.5 xenified kernel from xensource.org . It works fine on Ubuntu Intrepid Server under Xen 3.3 Hypervisor:-
       http://lxer.com/module/newswire/view/114358/index.html
I wonder what xenified kernel you were running for your Dom0. Could you post a grub entry working for your Xen Host ? I suspect failure when install Xen 3.3 via proposed on my machine.

You're correct - neither Intrepid nor Jaunty ship a Xen dom0-capable kernel. I've been using the Hardy xen kernel in my testing. If you want to replicate that, you can duplicate the Jaunty entries in your /etc/apt/sources.list, replacing jaunty with hardy. This should only affect packages that were present in Hardy but removed before Jaunty. Once you've done that, you can install the linux-xen package to get a Xen dom0 kernel.

Your error about xen.lowlevel.xc makes me think that python-xen-3.3 isn't installed properly - xen.lowlevel.xc is one of the 5 or so C extensions included in that package. Did you somehow force the installation of xen-utils-3.3 without python-xen-3.3?

Attached is my current work-in-progress patch for Karmic, that also closes bug #286450 and bug #346003, but it's not ready to be uploaded yet.

Both the attached patch and the patch currently in jaunty-proposed will break on further upgrades, because for some reason the prerm for python-xen-3.3 is being run before the prerm for xen-utils-3.3. I'm not sure why this is - xen-utils-3.3 depends on python-xen-3.3, but because of what's happening now, the Python module symlinks are being removed by pycentral before xen-utils-3.3's prerm can stop xend, which results in the prerm failing, leaving the package broken:

> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Reading extended state information
> Initializing package states... Done
> The following packages will be upgraded:
> libxen3 python-xen-3.3 xen-docs-3.3 xen-hypervisor-3.3 xen-utils-3.3
> 5 packages upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
> Need to get 1945kB of archives. After unpacking 0B will be used.
> Do you want to continue? [Y/n/?]
> Writing extended state information... Done
> Get:1 http://ppa.launchpad.net karmic/main libxen3 3.3.0-1ubuntu10~broder8 [151kB]
> Get:2 http://ppa.launchpad.net karmic/main python-xen-3.3 3.3.0-1ubuntu10~broder8 [378kB]
> Get:3 http://ppa.launchpad.net karmic/main xen-docs-3.3 3.3.0-1ubuntu10~broder8 [13.5kB]
> Get:4 http://ppa.launchpad.net karmic/main xen-hypervisor-3.3 3.3.0-1ubuntu10~broder8 [514kB]
> Get:5 http://ppa.launchpad.net karmic/main xen-utils-3.3 3.3.0-1ubuntu10~broder8 [888kB]
> Fetched 1945kB in 4s (407kB/s)
> (Reading database ... 31548 files and directories currently installed.)
> Preparing to replace libxen3 3.3.0-1ubuntu10~broder7 (using .../libxen3_3.3.0-1ubuntu10~broder8_amd64.deb) ...
> Unpacking replacement libxen3 ...
> Preparing to replace python-xen-3.3 3.3.0-1ubuntu10~broder7 (using .../python-xen-3.3_3.3.0-1ubuntu10~broder8_amd64.deb) ...
> Unpacking replacement python-xen-3.3 ...
> Preparing to replace xen-docs-3.3 3.3.0-1ubuntu10~broder7 (using .../xen-docs-3.3_3.3.0-1ubuntu10~broder8_amd64.deb) ...
> Unpacking replacement xen-docs-3.3 ...
> Preparing to replace xen-hypervisor-3.3 3.3.0-1ubuntu10~broder7 (using .../xen-hypervisor-3.3_3.3.0-1ubuntu10~broder8_amd64.deb) ...
> Unpacking replacement xen-hypervisor-3.3 ...
> Preparing to replace xen-utils-3.3 3.3.0-1ubuntu10~broder7 (using .../xen-utils-3.3_3.3.0-1ubuntu10~broder8_amd64.deb) ...
> Cannot find Xen Python modules.
> invoke-rc.d: initscript xend, action "stop" failed.
> dpkg: warning - old pre-removal script returned error exit status 1
> dpkg - trying script from the new package instead ...
> Cannot find Xen Python modules.
> invoke-rc.d: initscript xend, action "stop" failed.

I'm still working to figure out why the prerms are getting run in the wrong order; I just wanted to keep everyone following this bug updated on progress.

In case I wasn't sufficiently clear above, PLEASE don't upload the patch on the above comment just yet. It still needs more work.

Honestly, i don't see much sense in forward porting Xen 3.3 , because it doesn't support ( http://wiki.xensource.com/xenwiki/XenParavirtOps) :-

1. The ability for the Xen hypervisor to load and boot bzImage pv_ops dom0 kernel
2. The ability for the Xen tools to use the sysfs memory ballooning support provided by pv_ops dom0 kernel

Coming soon F11 (and even F10 in meantime) has Xen 3.3.1 with mentioned feature back ported from Xen Unstable (3.4). View for instance:-

http://lxer.com/module/newswire/view/120005/index.html.

My major concern in details follows bellow has been posted on xen-devel mailing list :-

Re: [Xen-devel] Changeset 19594 for Xen 3.4 build on Ubuntu Server 9.04

From:"Boris Derzhavets" <email address hidden> Friday, May 15, 2009 12:38 PM Add sender to Contacts
To:"Ian Jackson" < ... @eu.citrix.com>
Cc:"<email address hidden>" <email address hidden>, "Keir Fraser" < .... @eu.citrix.com>

Per /usr/lib/python2.6/site.py:-

For Debian and derivatives, this sys.path is augmented with directories
for packages distributed within the distribution. Local addons go
into /usr/local/lib/python<version>/dist-packages, Debian addons
install into /usr/{lib,share}/python<version>/dist-packages.
/usr/lib/python<version>/site-packages is not used.

It is different from /usr/lib/python2.5/site.py

With Config.mk:-

PYTHON = python
PYTHON_PREFIX_ARG =

Packages gets installed into /usr/local/lib/python2.6/dist-packages:-

root@ServerJaunty:/usr/local/lib/python2.6/dist-packages# ls -ld xen
drwxr-sr-x 9 root staff 4096 2009-05-15 10:35 xen

Xend starts fine, pygrub doesn't fail. Xen host seems to be OK.
But, "xm create -c DomU.cfg" doesn't work any more,even with xenified kernel. DomU gets paused at startup and no matter of been unpaused or no, finally crashes with message:-

"Hotplug scripts not working"

I believe Xen 3.4 has problem been installed on Ubuntu 9.04 Server (64-bit)
with python 2.6 installed with default settings. Same situation will happen on any Debian derivative with Python 2.6.
If i am wrong about that, please advise.

Boris.

Boris, I'd appreciate if you kept the discussion on this bug focused on this bug - if you see other issues with the Xen packaging, then you should open another bug. Upgrading to 3.3.1, for example, while an excellent idea, is not related to this bug, and I in particular would prefer to get my version of 3.3.0-1ubuntu10 uploaded before I look into upgrading to 3.3.1, which is a non-trivial change from the current version:

fanty:~/Downloads evan$ diff -ur xen-3.3.0 xen-3.3.1 | diffstat | tail -n 1
 185 files changed, 2874 insertions(+), 1467 deletions(-)

For 9.04, neither the Xen packages in jaunty or the ones in jaunty-proposed are using Python 2.6, so the switch to dist-packages isn't relevant. For Karmic, one of my changes is to patch the xen-python-patch script to look in dist-packages as well as site-packages.

Using my test builds of the Karmic packages in my PPA (https://launchpad.net/~broder/+archive/ppa), I've been able to successfully build a domU and boot it up completely, so I believe the issues with hotplug scripts may be specific to your configuration.

Deployment Xen 3.4 (testing) with 2.6.30-rc5-tip kernel to Ubuntu 9.04 Server (64-bit)
http://bderzhavets.wordpress.com/2009/05/19/deployment-xen-3-4-testing-with-2-6-30-rc5-tip-kernel-to-ubuntu-9-04-server-64-bit/

Boris, that is still off-topic for this bug. See bug #378240 if you want to discuss the upgrade.

Here is a patch for Karmic that fixes this bug, as well as a few other long-standing bugs.

I intend to work on packaging for Xen 3.4 once this is done.

Due to some strange dependency interactions that nobody I've talked to has been able to explain, the prerm for python-xen-3.3 removes the pycentral-managed Python modules before the prerm for xen-utils-3.3 stops xend, which means that stopping xend fails.

The SRU for Jaunty needs to be updated to the attached patch, which holds off on restarting xend until after the upgrade. While this won't show up on the upgrade from 3.3.0-1ubuntu9 to 3.3.0-1ubuntu9.2, it would mean that any subsequent upgrades would break.

>I intend to work on packaging for Xen 3.4 once this is done.

Porting Xen 3.4 Hyperviser&tools to Ubuntu Karmic Server would be a good job. It would make some sense i guess due to ongoing development Xen Dom0 support for vanilla kernel. Even if it wouldn't merge mainline Linux in next merge window. PV_ops kernel is alive and stable enough (2.6.30-rc5-tip).
Regarding Xen, view:-
http://www.xen.org/download/
Sorry, for post off-topic.

Updated xen-3.3 package accepted, please test:

 xen-3.3 (3.3.0-1ubuntu9.3) jaunty-proposed; urgency=low
 .
   * Force the build tree to explicitly use python2.5 instead of python,
     since that's 2.6 now. (LP: #362691)
   * Wait until the end of an upgrade to restart xend. Otherwise, the
     pycentral-managed python modules won't be available when trying to
     stop the old xend.

Ok, I've tested 3.3.0-1ubuntu9.3 on a Jaunty machine. I was able to install a VM (using ubuntu-vm-builder) and then boot it up and connect to the console, so I think this one is all set.

The patch in https://bugs.launchpad.net/ubuntu/+source/xen-3.3/+bug/362691/comments/37 is still good for Karmic.

This bug was fixed in the package xen-3.3 - 3.3.0-1ubuntu9.3

---------------
xen-3.3 (3.3.0-1ubuntu9.3) jaunty-proposed; urgency=low

  * Force the build tree to explicitly use python2.5 instead of python,
    since that's 2.6 now. (LP: #362691)
  * Wait until the end of an upgrade to restart xend. Otherwise, the
    pycentral-managed python modules won't be available when trying to
    stop the old xend.

 -- Evan Broder <email address hidden> Tue, 19 May 2009 13:01:38 -0400

This bug was fixed in the package xen-3.3 - 3.3.0-1ubuntu10

---------------
xen-3.3 (3.3.0-1ubuntu10) karmic; urgency=low

  * debian/patches/use-python-2.6.diff: Pass --install-layout=deb for the
    brave new Python 2.6 world.
  * Also rebuild for Python 2.6. (LP: #362691)
  * debian/patches/suppress-python-2.6-warnings.diff: Patch out a few
    DeprecationWarnings introduced in Python 2.6
  * debian/patches/gcc-4.4-fixes.diff: Fix some inline assembly to build
    with gcc 4.4
  * debian/control: Make sure a dependency on a specific Python version
    gets added.
  * debian/control: Fix the Replaces line for libxen3 and libxen3-dev (LP:
    #286450)
  * debian/control: Fix a lintian warning in libxen3-dev's dependencies.
  * debian/rules: Don't clean out debian/tmp between install stages. This
    reunites some manpages with their long-lost packages. (LP: #346003)
  * debian/rules: Clean up the debian/xen-utils-3.3.udev file created as
    part of the build process.
  * debian/rules: Restart xend after the upgrade is complete to be sure
    pycentral installed the modules.

 -- Evan Broder <email address hidden> Sat, 16 May 2009 18:01:24 -0400