Our hudson server still had trouble building nova trunk with clean virtual environment "./run_tests.sh -f". I hacked in the following into our hpc_trunk branch as a workaround. It looks like the dependencies are getting triggered out of order.
bschott@ironman:~/Source/nova/hpc-trunk$ bzr diff -r 465 tools/install_venv.py
=== modified file 'tools/install_venv.py'
--- tools/install_venv.py 2011-01-11 18:11:30 +0000
+++ tools/install_venv.py 2011-01-24 16:23:31 +0000
@@ -93,6 +93,17 @@
# get it in stalled in the right order
run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, 'greenlet'], redirect_output=False)
+ #
+ # BFS workaround for glance build error, remove when fixed
+ #
+ run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, 'sphinx'],
+ redirect_output=False)
+ run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, 'argparse'],
+ redirect_output=False)
+ #
+ # BFS end workaround
+ #
+
run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, '-r', PIP_REQUIRES], redirect_output=False)
run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, TWISTED_NOVA],
--- log ---
bschott@ironman:~/Source/nova/trunk$ bzr pull
Using saved parent location: bzr+ssh://bazaar.launchpad.net/%2Bbranch/nova/
M nova/virt/xenapi/vmops.py
M tools/pip-requires
All changes applied successfully.
Now on revision 604.
bschott@ironman:~/Source/nova/trunk$ ls -al
total 144
drwxr-xr-x 30 bschott staff 1020 Jan 22 12:29 .
drwxr-xr-x 9 bschott staff 306 Jan 24 11:33 ..
drwxr-xr-x 7 bschott staff 238 Jan 11 11:24 .bzr
-rw-r--r-- 1 bschott staff 200 Jan 11 11:24 .bzrignore
-rw-r--r-- 1 bschott staff 125 Jan 11 11:24 .gitignore
-rw-r--r-- 1 bschott staff 1646 Jan 22 12:29 .mailmap
-rw-r--r-- 1 bschott staff 2072 Jan 22 12:29 Authors
drwxr-xr-x 11 bschott staff 374 Jan 11 11:24 CA
-rw-r--r-- 1 bschott staff 1345 Jan 11 11:24 HACKING
-rw-r--r-- 1 bschott staff 10143 Jan 11 11:24 LICENSE
-rw-r--r-- 1 bschott staff 1070 Jan 22 12:29 MANIFEST.in
-rw-r--r-- 1 bschott staff 837 Jan 22 12:29 README
-rw-r--r-- 1 bschott staff 17 Jan 11 11:24 babel.cfg
drwxr-xr-x 19 bschott staff 646 Jan 22 12:29 bin
-rwxr-xr-x 1 bschott staff 787 Jan 11 11:24 builddeb.sh
drwxr-xr-x 3 bschott staff 102 Jan 11 11:24 bzrplugins
drwxr-xr-x 5 bschott staff 170 Jan 22 12:29 contrib
drwxr-xr-x 9 bschott staff 306 Jan 11 11:24 doc
drwxr-xr-x 3 bschott staff 102 Jan 22 12:29 etc
-rw-r--r-- 1 bschott staff 41 Jan 22 12:29 krm_mapping.json.sample
drwxr-xr-x 3 bschott staff 102 Jan 11 11:24 locale
drwxr-xr-x 33 bschott staff 1122 Jan 22 12:29 nova
drwxr-xr-x 3 bschott staff 102 Jan 11 11:24 plugins
-rw-r--r-- 1 bschott staff 829 Jan 11 11:24 pylintrc
-rw-r--r-- 1 bschott staff 2240 Jan 22 12:29 run_tests.py
-rwxr-xr-x 1 bschott staff 2205 Jan 22 12:29 run_tests.sh
-rw-r--r-- 1 bschott staff 372 Jan 11 11:24 setup.cfg
-rw-r--r-- 1 bschott staff 3481 Jan 22 12:29 setup.py
drwxr-xr-x 10 bschott staff 340 Jan 22 12:29 smoketests
drwxr-xr-x 10 bschott staff 340 Jan 24 11:51 tools
bschott@ironman:~/Source/nova/trunk$ ./run_tests.sh -f
Cleaning virtualenv...
No virtual environment found...create one? (Y/n)
done.
Creating venv... done.
Installing pip in virtualenv... done.
Installing dependencies with pip (this can take a while)...
Downloading/unpacking greenlet
Downloading greenlet-0.3.1.tar.gz
Running setup.py egg_info for package greenlet
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'repository'
warnings.warn(msg)
Installing collected packages: greenlet
Running setup.py install for greenlet
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'repository'
warnings.warn(msg)
building 'greenlet' extension
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c greenlet.c -o build/temp.macosx-10.6-universal-2.6/greenlet.o
gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch ppc -arch x86_64 build/temp.macosx-10.6-universal-2.6/greenlet.o -o build/lib.macosx-10.6-universal-2.6/greenlet.so
Successfully installed greenlet
Cleaning up...
Downloading/unpacking zope.interface==3.6.1 (from -r /Users/bschott/Source/nova/trunk/tools/pip-requires (line 19))
Downloading zope.interface-3.6.1.tar.gz (125Kb): 125Kb downloaded
Running setup.py egg_info for package zope.interface
Downloading/unpacking Twisted>=10.1.0 (from -r /Users/bschott/Source/nova/trunk/tools/pip-requires (line 25))
Downloading Twisted-10.2.0.tar.bz2 (2.7Mb): 2.7Mb downloaded
Running setup.py egg_info for package Twisted
Downloading/unpacking SQLAlchemy==0.6.3 (from -r /Users/bschott/Source/nova/trunk/tools/pip-requires (line 1))
Downloading SQLAlchemy-0.6.3.tar.gz (1.9Mb): 1.9Mb downloaded
Running setup.py egg_info for package SQLAlchemy
no previously-included directories found matching 'doc/build/output'
Downloading/unpacking Cheetah==2.4.2.1 (from -r /Users/bschott/Source/nova/trunk/tools/pip-requires (line 5))
Downloading Cheetah-2.4.2.1.tar.gz (189Kb): 189Kb downloaded
Running setup.py egg_info for package Cheetah
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py:266: UserWarning: Unknown distribution option: 'line'
warnings.warn(msg)
warning: no files found matching 'examples'
warning: no files found matching 'docs'
warning: no files found matching 'bin'
warning: no files found matching '*' under directory 'docs'
warning: no files found matching '*' under directory 'examples'
warning: no previously-included files matching '*.pyc' found under directory 'cheetah'
warning: no previously-included files matching '*~' found under directory 'cheetah'
warning: no previously-included files matching '*.aux' found under directory 'cheetah'
warning: no previously-included files matching '*~' found under directory 'docs'
warning: no previously-included files matching '*.aux' found under directory 'docs'
Downloading/unpacking glance (from -r /Users/bschott/Source/nova/trunk/tools/pip-requires (line 31))
Downloading glance-0.1.3.tar.gz (67Kb): 67Kb downloaded
Running setup.py egg_info for package glance
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/Users/bschott/Source/nova/trunk/.nova-venv/build/glance/setup.py", line 22, in <module>
from sphinx.setup_command import BuildDoc
ImportError: No module named sphinx.setup_command
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/Users/bschott/Source/nova/trunk/.nova-venv/build/glance/setup.py", line 22, in <module>
from sphinx.setup_command import BuildDoc
ImportError: No module named sphinx.setup_command
----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in /Users/bschott/.pip/pip.log
Traceback (most recent call last):
File "/Users/bschott/Source/nova/trunk/.nova-venv/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/runner.py", line 16, in <module>
exit = run()
File "/Users/bschott/Source/nova/trunk/.nova-venv/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/runner.py", line 12, in run
return pip.main()
File "/Users/bschott/Source/nova/trunk/.nova-venv/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/__init__.py", line 116, in main
return command.main(initial_args, args[1:], options)
File "/Users/bschott/Source/nova/trunk/.nova-venv/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/basecommand.py", line 149, in main
log_fp = open_logfile(log_fn, 'w')
File "/Users/bschott/Source/nova/trunk/.nova-venv/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg/pip/basecommand.py", line 178, in open_logfile
log_fp = open(filename, mode)
IOError: [Errno 13] Permission denied: '/Users/bschott/.pip/pip.log'
Command "tools/with_venv.sh pip install -E /Users/bschott/Source/nova/trunk/.nova-venv -r /Users/bschott/Source/nova/trunk/tools/pip-requires" failed.
None
bschott@ironman:~/Source/nova/trunk$
Our hudson server still had trouble building nova trunk with clean virtual environment "./run_tests.sh -f". I hacked in the following into our hpc_trunk branch as a workaround. It looks like the dependencies are getting triggered out of order.
bschott@ ironman: ~/Source/ nova/hpc- trunk$ bzr diff -r 465 tools/install_ venv.py install_ venv.py' venv.py 2011-01-11 18:11:30 +0000 venv.py 2011-01-24 16:23:31 +0000 command( ['tools/ with_venv. sh', 'pip', 'install', '-E', venv, 'greenlet'],
redirect_ output= False) ['tools/ with_venv. sh', 'pip', 'install', '-E', venv, 'sphinx'], output= False) ['tools/ with_venv. sh', 'pip', 'install', '-E', venv, 'argparse'], output= False) command( ['tools/ with_venv. sh', 'pip', 'install', '-E', venv, '-r', PIP_REQUIRES],
redirect_ output= False) command( ['tools/ with_venv. sh', 'pip', 'install', '-E', venv, TWISTED_NOVA],
=== modified file 'tools/
--- tools/install_
+++ tools/install_
@@ -93,6 +93,17 @@
# get it in stalled in the right order
run_
+ #
+ # BFS workaround for glance build error, remove when fixed
+ #
+ run_command(
+ redirect_
+ run_command(
+ redirect_
+ #
+ # BFS end workaround
+ #
+
run_
run_
--- log ---
bschott@ ironman: ~/Source/ nova/trunk$ bzr pull //bazaar. launchpad. net/%2Bbranch/ nova/ xenapi/ vmops.py ironman: ~/Source/ nova/trunk$ ls -al json.sample ironman: ~/Source/ nova/trunk$ ./run_tests.sh -f unpacking greenlet 0.3.1.tar. gz Library/ Frameworks/ Python. framework/ Versions/ 2.6/lib/ python2. 6/distutils/ dist.py: 266: UserWarning: Unknown distribution option: 'repository' warn(msg) Library/ Frameworks/ Python. framework/ Versions/ 2.6/lib/ python2. 6/distutils/ dist.py: 266: UserWarning: Unknown distribution option: 'repository' warn(msg) aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -I/System/ Library/ Frameworks/ Python. framework/ Versions/ 2.6/include/ python2. 6 -c greenlet.c -o build/temp. macosx- 10.6-universal- 2.6/greenlet. o macosx- 10.6-universal- 2.6/greenlet. o -o build/lib. macosx- 10.6-universal- 2.6/greenlet. so unpacking zope.interface= =3.6.1 (from -r /Users/ bschott/ Source/ nova/trunk/ tools/pip- requires (line 19)) 3.6.1.tar. gz (125Kb): 125Kb downloaded unpacking Twisted>=10.1.0 (from -r /Users/ bschott/ Source/ nova/trunk/ tools/pip- requires (line 25)) 10.2.0. tar.bz2 (2.7Mb): 2.7Mb downloaded unpacking SQLAlchemy==0.6.3 (from -r /Users/ bschott/ Source/ nova/trunk/ tools/pip- requires (line 1)) 0.6.3.tar. gz (1.9Mb): 1.9Mb downloaded unpacking Cheetah==2.4.2.1 (from -r /Users/ bschott/ Source/ nova/trunk/ tools/pip- requires (line 5)) 2.4.2.1. tar.gz (189Kb): 189Kb downloaded Library/ Frameworks/ Python. framework/ Versions/ 2.6/lib/ python2. 6/distutils/ dist.py: 266: UserWarning: Unknown distribution option: 'line' warn(msg) unpacking glance (from -r /Users/ bschott/ Source/ nova/trunk/ tools/pip- requires (line 31)) bschott/ Source/ nova/trunk/ .nova-venv/ build/glance/ setup.py" , line 22, in <module> setup_command import BuildDoc setup_command
Using saved parent location: bzr+ssh:
M nova/virt/
M tools/pip-requires
All changes applied successfully.
Now on revision 604.
bschott@
total 144
drwxr-xr-x 30 bschott staff 1020 Jan 22 12:29 .
drwxr-xr-x 9 bschott staff 306 Jan 24 11:33 ..
drwxr-xr-x 7 bschott staff 238 Jan 11 11:24 .bzr
-rw-r--r-- 1 bschott staff 200 Jan 11 11:24 .bzrignore
-rw-r--r-- 1 bschott staff 125 Jan 11 11:24 .gitignore
-rw-r--r-- 1 bschott staff 1646 Jan 22 12:29 .mailmap
-rw-r--r-- 1 bschott staff 2072 Jan 22 12:29 Authors
drwxr-xr-x 11 bschott staff 374 Jan 11 11:24 CA
-rw-r--r-- 1 bschott staff 1345 Jan 11 11:24 HACKING
-rw-r--r-- 1 bschott staff 10143 Jan 11 11:24 LICENSE
-rw-r--r-- 1 bschott staff 1070 Jan 22 12:29 MANIFEST.in
-rw-r--r-- 1 bschott staff 837 Jan 22 12:29 README
-rw-r--r-- 1 bschott staff 17 Jan 11 11:24 babel.cfg
drwxr-xr-x 19 bschott staff 646 Jan 22 12:29 bin
-rwxr-xr-x 1 bschott staff 787 Jan 11 11:24 builddeb.sh
drwxr-xr-x 3 bschott staff 102 Jan 11 11:24 bzrplugins
drwxr-xr-x 5 bschott staff 170 Jan 22 12:29 contrib
drwxr-xr-x 9 bschott staff 306 Jan 11 11:24 doc
drwxr-xr-x 3 bschott staff 102 Jan 22 12:29 etc
-rw-r--r-- 1 bschott staff 41 Jan 22 12:29 krm_mapping.
drwxr-xr-x 3 bschott staff 102 Jan 11 11:24 locale
drwxr-xr-x 33 bschott staff 1122 Jan 22 12:29 nova
drwxr-xr-x 3 bschott staff 102 Jan 11 11:24 plugins
-rw-r--r-- 1 bschott staff 829 Jan 11 11:24 pylintrc
-rw-r--r-- 1 bschott staff 2240 Jan 22 12:29 run_tests.py
-rwxr-xr-x 1 bschott staff 2205 Jan 22 12:29 run_tests.sh
-rw-r--r-- 1 bschott staff 372 Jan 11 11:24 setup.cfg
-rw-r--r-- 1 bschott staff 3481 Jan 22 12:29 setup.py
drwxr-xr-x 10 bschott staff 340 Jan 22 12:29 smoketests
drwxr-xr-x 10 bschott staff 340 Jan 24 11:51 tools
bschott@
Cleaning virtualenv...
No virtual environment found...create one? (Y/n)
done.
Creating venv... done.
Installing pip in virtualenv... done.
Installing dependencies with pip (this can take a while)...
Downloading/
Downloading greenlet-
Running setup.py egg_info for package greenlet
/System/
warnings.
Installing collected packages: greenlet
Running setup.py install for greenlet
/System/
warnings.
building 'greenlet' extension
gcc-4.2 -fno-strict-
gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch ppc -arch x86_64 build/temp.
Successfully installed greenlet
Cleaning up...
Downloading/
Downloading zope.interface-
Running setup.py egg_info for package zope.interface
Downloading/
Downloading Twisted-
Running setup.py egg_info for package Twisted
Downloading/
Downloading SQLAlchemy-
Running setup.py egg_info for package SQLAlchemy
no previously-included directories found matching 'doc/build/output'
Downloading/
Downloading Cheetah-
Running setup.py egg_info for package Cheetah
/System/
warnings.
warning: no files found matching 'examples'
warning: no files found matching 'docs'
warning: no files found matching 'bin'
warning: no files found matching '*' under directory 'docs'
warning: no files found matching '*' under directory 'examples'
warning: no previously-included files matching '*.pyc' found under directory 'cheetah'
warning: no previously-included files matching '*~' found under directory 'cheetah'
warning: no previously-included files matching '*.aux' found under directory 'cheetah'
warning: no previously-included files matching '*~' found under directory 'docs'
warning: no previously-included files matching '*.aux' found under directory 'docs'
Downloading/
Downloading glance-0.1.3.tar.gz (67Kb): 67Kb downloaded
Running setup.py egg_info for package glance
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/Users/
from sphinx.
ImportError: No module named sphinx.
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 14, in <module>
File "/Users/ bschott/ Source/ nova/trunk/ .nova-venv/ build/glance/ setup.py" , line 22, in <module>
from sphinx. setup_command import BuildDoc
ImportError: No module named sphinx. setup_command
------- ------- ------- ------- ------- ----- bschott/ .pip/pip. log bschott/ Source/ nova/trunk/ .nova-venv/ lib/python2. 6/site- packages/ pip-0.8. 1-py2.6. egg/pip/ runner. py", line 16, in <module> bschott/ Source/ nova/trunk/ .nova-venv/ lib/python2. 6/site- packages/ pip-0.8. 1-py2.6. egg/pip/ runner. py", line 12, in run bschott/ Source/ nova/trunk/ .nova-venv/ lib/python2. 6/site- packages/ pip-0.8. 1-py2.6. egg/pip/ __init_ _.py", line 116, in main main(initial_ args, args[1:], options) bschott/ Source/ nova/trunk/ .nova-venv/ lib/python2. 6/site- packages/ pip-0.8. 1-py2.6. egg/pip/ basecommand. py", line 149, in main log_fn, 'w') bschott/ Source/ nova/trunk/ .nova-venv/ lib/python2. 6/site- packages/ pip-0.8. 1-py2.6. egg/pip/ basecommand. py", line 178, in open_logfile bschott/ .pip/pip. log' bschott/ Source/ nova/trunk/ .nova-venv -r /Users/ bschott/ Source/ nova/trunk/ tools/pip- requires" failed. ironman: ~/Source/ nova/trunk$
Command python setup.py egg_info failed with error code 1
Storing complete log in /Users/
Traceback (most recent call last):
File "/Users/
exit = run()
File "/Users/
return pip.main()
File "/Users/
return command.
File "/Users/
log_fp = open_logfile(
File "/Users/
log_fp = open(filename, mode)
IOError: [Errno 13] Permission denied: '/Users/
Command "tools/with_venv.sh pip install -E /Users/
None
bschott@