On  2 Jan 2009, Toni Mueller <email address hidden> wrote:
> 
> Hi Martin,
> 
> On Mon, 29.12.2008 at 19:24:02 +0100, Toni Mueller <email address hidden> wrote:
> > I tried to run the self-test suite for bzr-1.10 on OpenBSD 4.4/amd64,
> 
> I've re-tried, and got the results that you find attached. I didn't
> check whether all tests were actually performed, but simply ship
> everything that the tests yieled, to you. The test run (probably) ended
> when Python (2.5.2p4) segfaulted. I have the core lying around, so if
> you want me to do some surgery on it, just ask for it (I don't really
> know what to do). I created the "...cpio.." archive using "pax -x
> sv4cpio". Rolling a tarball was impossible because of the 100 char
> pathname limit in the tar archive format.


Hi,

Thanks for testing this.  I opened
<https://bugs.edge.launchpad.net/bzr/+bug/314247> to make the results
available to other people.

The problems here seem to be:

Firstly, if bzr selftest doesn't run correctly without a terminal or in
nohup mode, that seems to be a bug in itself.  Run under Linux with
nohup it does not write any output to the log file while it's running, I
guess because it's relying on a progress bar and that will be off if
there's no terminal, but I do get a list of failed, xfail etc tests at
the end of the run.

On Ubuntu Intrepid all the tests pass in a nohup environment pass except
for some problems in bzr-gtk.

bzr is supposed to delete temporary directories as it finishes each
test, so it shouldn't be using a large amount of disk space to run.  It
is possible that particular tests will need several megabytes.  As I'm
running the tests I see all of /tmp is about 12MB.  If you can isolate
what is using too much space we may find a bug.  Otherwise you can run
them with TMPDIR=~/tmp to put it on another filesystem.

So the substantial bugs in here are:

> blackbox.test_breakin.TestBreakin.test_breakin                                                                       FAIL                 619ms
>     pattern "entering debugger" not found in
> """\
> No handlers could be found for logger "bzr"
> """

The logger message might be a problem with not having a console. 
breakin.py seems to unconditionally, write to stderr so should not be
affected by that.  It would be useful if you could try pressing C-\
while bzr is running normally, to see if there's a difference in sigquit
handling on openbsd.

> blackbox.test_check.ChrootedCheckTests.test_check_missing_branch
> ERROR                  48ms
> 
>    (26, 'Failed to open/read local data from file/application')

> ^^^^[log from bzrlib.tests.blackbox.test_check.ChrootedCheckTests.test_check_missing_branch]
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/sw/local/bzr-1.10/bzrlib/tests/blackbox/test_check.py", line 104, in test_check_missing_branch
>     'check --branch %s' % self.get_readonly_url(''))
>   File "/home/sw/local/bzr-1.10/bzrlib/tests/__init__.py", line 1486, in run_bzr
>     working_dir=working_dir,
>   File "/home/sw/local/bzr-1.10/bzrlib/tests/__init__.py", line 1400, in _run_bzr_autosplit
>     encoding=encoding, stdin=stdin, working_dir=working_dir,
>   File "/home/sw/local/bzr-1.10/bzrlib/tests/__init__.py", line 1430, in _run_bzr_core
>     args)
>   File "/home/sw/local/bzr-1.10/bzrlib/tests/__init__.py", line 1719, in apply_redirected
>     return a_callable(*args, **kwargs)
>   File "/home/sw/local/bzr-1.10/bzrlib/commands.py", line 912, in run_bzr_catch_user_errors
>     return run_bzr(argv)
>   File "/home/sw/local/bzr-1.10/bzrlib/commands.py", line 839, in run_bzr
>     ret = run(*run_argv)
>   File "/home/sw/local/bzr-1.10/bzrlib/commands.py", line 539, in run_argv_aliases
>     return self.run(**all_cmd_args)
>   File "/home/sw/local/bzr-1.10/bzrlib/builtins.py", line 2513, in run
>     check_dwim(path, verbose, do_branch=branch, do_repo=repo, do_tree=tree)
>   File "/home/sw/local/bzr-1.10/bzrlib/check.py", line 275, in check_dwim
>     BzrDir.open_containing_tree_branch_or_repository(path)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 919, in open_containing_tree_branch_or_repository
>     bzrdir, relpath = klass.open_containing(location)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 828, in open_containing
>     return BzrDir.open_containing_from_transport(transport)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 849, in open_containing_from_transport
>     result = BzrDir.open_from_transport(a_transport)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 803, in open_from_transport
>     redirected)
>   File "/home/sw/local/bzr-1.10/bzrlib/transport/__init__.py", line 1643, in do_catching_redirections
>     return action(transport)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 780, in find_format
>     transport, _server_formats=_server_formats)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 1638, in find_format
>     return format.probe_transport(transport)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 2621, in probe_transport
>     server_version = medium.protocol_version()
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 593, in protocol_version
>     client_protocol.query_version()
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/protocol.py", line 769, in query_version
>     self.call('hello')
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/protocol.py", line 617, in call
>     self._request.finished_writing()
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 421, in finished_writing
>     self._finished_writing()
>   File "/home/sw/local/bzr-1.10/bzrlib/transport/http/__init__.py", line 575, in _finished_writing
>     data = self._medium.send_http_smart_request(self._buffer)
>   File "/home/sw/local/bzr-1.10/bzrlib/transport/http/__init__.py", line 552, in send_http_smart_request
>     code, body_filelike = t._post(bytes)
>   File "/home/sw/local/bzr-1.10/bzrlib/transport/http/_pycurl.py", line 267, in _post
>     self._curl_perform(curl, header, ['Expect: '])
>   File "/home/sw/local/bzr-1.10/bzrlib/transport/http/_pycurl.py", line 341, in _curl_perform
>     curl.perform()
> error: (26, 'Failed to open/read local data from file/application')
> 

This occurs in many of the tests, and it looks like a problem in pycurl,
or in curl.  It might be best to uninstall it, or disable it by putting
a file named 'pycurl.py' containing 'raise ImportError()' in a directory
on your $PYTHONPATH. 


> blackbox.test_diff.TestExternalDiff.test_external_diff                                                               FAIL                 495ms
>     not equal:
> a = ''
> b = "diff: unknown option -- binary\nusage: diff [-abdilpqTtw] [-I pattern] [-c | -e | -f | -n | -u]\n            [-L label] file1 file2\n       diff [-abdilpqTtw] [-I pattern] [-L label] -C number file1 file2\n       diff [-abdilqtw] [-I pattern] -D string file1 file2\n       diff [-abdilpqTtw] [-I pattern] [-L label] -U number file1 file2\n       diff [-abdilNPpqrsTtw] [-I pattern] [-c | -e | -f | -n | -u]\n            [-L label] [-S name] [-X file] [-x pattern] dir1 dir2\nbzr: ERROR: external diff failed with exit code 2; command: ['diff', '--label', u'goodbye\\t1970-01-01 00:00:00 +0000', '/tmp/bzr-diff-old-D3V3cy', '--label', u'goodbye\\t2008-12-31 00:10:49 +0000', '/tmp/bzr-diff-new-laJYMs', '--binary', '-ub']\n"
> 
> 

We seem to be unnecessarily relying on gnu diff here in
bzrlib.diff.external_diff, and on bsd at least we should not.

> blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir
> ERROR                 344ms
>    [Errno 2] No such file or directory: '..'


> ======================================================================
> ERROR: test_remove_changed_files_from_child_dir (bzrlib.tests.blackbox.test_remove.TestRemove)
> 
> vvvv[log from bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir]
> 546.905  creating repository in file:///tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir/work/.bzr/.
> 546.942  creating branch <bzrlib.branch.BzrBranchFormat6 object at 0x21c09b750> in file:///tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir/work/.bzr/
> 546.993  trying to create missing lock '/tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir/work/.bzr/checkout/dirstate'
> 546.994  opening working tree '/tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir/work'
> 547.047  run bzr: ['commit', '-m', 'added files']
> 547.048  bzr arguments: ['commit', '-m', 'added files']
> 547.049  encoding stdout as sys.stdout encoding '646'
> 547.052  opening working tree '/tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir/work'
> 547.068  preparing to commit
>     INFO  Committing to: /tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir/work/
> 547.072  Selecting files for commit with filter []
>     INFO  added a
>     INFO  added b
>     INFO  added d
>     INFO  added b/c
>     INFO  Committed revision 1.
> 547.128  errors:
> 'Committing to: /tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir/work/\nadded a\nadded b\nadded d\nadded b/c\nCommitted revision 1.\n'
> 547.129  run bzr: ['remove', '../a', 'c', '.', '../d']
> 547.129  bzr arguments: ['remove', '../a', 'c', '.', '../d']
> 547.130  encoding stdout as sys.stdout encoding '646'
> 547.133  opening working tree '/tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir/work'
> 547.164  Traceback (most recent call last):
>   File "/home/sw/local/bzr-1.10/bzrlib/commands.py", line 912, in run_bzr_catch_user_errors
>     return run_bzr(argv)
>   File "/home/sw/local/bzr-1.10/bzrlib/commands.py", line 839, in run_bzr
>     ret = run(*run_argv)
>   File "/home/sw/local/bzr-1.10/bzrlib/commands.py", line 539, in run_argv_aliases
>     return self.run(**all_cmd_args)
>   File "/home/sw/local/bzr-1.10/bzrlib/builtins.py", line 1239, in run
>     force=file_deletion_strategy=='force')
>   File "/home/sw/local/bzr-1.10/bzrlib/mutabletree.py", line 52, in tree_write_locked
>     return unbound(self, *args, **kwargs)
>   File "/home/sw/local/bzr-1.10/bzrlib/workingtree.py", line 1932, in remove
>     raise errors.BzrRemoveChangedFilesError(tree_delta)
> BzrRemoveChangedFilesError: Can't safely remove modified or unknown files:
> modified:
>   a
>   b/c
> Use --keep to not delete them, or --force to delete them regardless.
> 
> 547.164  errors:
> "bzr: ERROR: Can't safely remove modified or unknown files:\nmodified:\n  a\n  b/c\nUse --keep to not delete them, or --force to delete them regardless.\n"
> 547.164  run bzr: ['remove', '--force', '../a', 'c', '.', '../d']
> 547.164  bzr arguments: ['remove', '--force', '../a', 'c', '.', '../d']
> 547.165  encoding stdout as sys.stdout encoding '646'
> 547.168  opening working tree '/tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir/work'
>     INFO  deleted d
>     INFO  deleted b/c
>     INFO  deleted b
>     INFO  deleted a
> 547.206  errors:
> 'deleted d\ndeleted b/c\ndeleted b\ndeleted a\n'
> 547.257  opening working tree '/tmp/testbzr-qdLfKY.tmp'
> 
> ^^^^[log from bzrlib.tests.blackbox.test_remove.TestRemove.test_remove_changed_files_from_child_dir]
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/sw/local/bzr-1.10/bzrlib/tests/blackbox/test_remove.py", line 202, in test_remove_changed_files_from_child_dir
>     os.chdir('..')
> OSError: [Errno 2] No such file or directory: '..'
> 
> 
> 

This seems like a genuine portability bug, that once a directory has
been deleted on bsd you can no longer see it's .. entry.  On Linux I
think the directory remains alive in memory while it's in use.  We
should fix the test.

> blackbox.test_serve.TestBzrServe.test_bzr_connect_to_bzr_ssh                                                        ERROR                 407ms
>     Connection closed: please check connectivity and permissions (and try -Dhpss if further diagnosis is required)

ERROR: test_bzr_connect_to_bzr_ssh
(bzrlib.tests.blackbox.test_serve.TestBzrServe)

> vvvv[log from
> bzrlib.tests.blackbox.test_serve.TestBzrServe.test_bzr_connect_to_bzr_ssh]
> 611.394  creating repository in
> file:///tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_serve.TestBzrServe.test_bzr_connect_to_bzr_ssh/work/a_branch/.bzr/.
> 611.430  creating branch <bzrlib.branch.BzrBranchFormat6 object at
> 0x21c005d50> in
> file:///tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_serve.TestBzrServe.test_bzr_connect_to_bzr_ssh/work/a_branch/.bzr/
> 611.466  failed to load system host keys: [Errno 2] No such file or
> directory:
> '/tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_serve.TestBzrServe.test_bzr_connect_to_bzr_ssh/home/.ssh/known_hosts'
> 611.466  failed to load bzr host keys: [Errno 2] No such file or
> directory:
> '/tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_serve.TestBzrServe.test_bzr_connect_to_bzr_ssh/home/.bazaar/ssh_host_keys'
> 611.466  creating config directory:
> '/tmp/testbzr-qdLfKY.tmp/bzrlib.tests.blackbox.test_serve.TestBzrServe.test_bzr_connect_to_bzr_ssh/home/.bazaar'
>    DEBUG  starting thread (client mode): 0x1c1baa90L
>     INFO  Connected (version 2.0, client paramiko_1.6.2)
>    DEBUG  kex algos:['diffie-hellman-group1-sha1'] server
> key:['ssh-rsa'] client encrypt:['aes128-cbc', 'blowfish-cbc',
> 'aes256-cbc', '3des-cbc'] server encrypt:['aes128-cbc', 'blowfish-cbc',
> 'aes256-cbc', '3des-cbc'] client mac:['hmac-sha1', 'hmac-md5',
> 'hmac-sha1-96', 'hmac-md5-96'] server mac:['hmac-sha1', 'hmac-md5',
> 'hmac-sha1-96', 'hmac-md5-96'] client compress:['none'] server
> compress:['none'] client lang:[''] server lang:[''] kex follows?False
>    DEBUG  Ciphers agreed: local=aes128-cbc, remote=aes128-cbc
>    DEBUG  using kex diffie-hellman-group1-sha1; server key type ssh-rsa;
> cipher: local aes128-cbc, remote aes128-cbc; mac: local hmac-sha1,
> remote hmac-sha1; compression: local none, remote none
>    DEBUG  Switch to new keys ...
>  WARNING  Adding ssh-rsa host key for localhost:
> 60733844CB5186657FDEDAA22B5A57D5
>    DEBUG  userauth is OK
>     INFO  Authentication (password) successful!
>    DEBUG  Max packet in: 34816 bytes
>    DEBUG  Max packet out: 34816 bytes
>     INFO  Secsh channel 1 opened.
>    DEBUG  Sesch channel 1 request ok
>    DEBUG  EOF received
>    DEBUG  EOF sent
> 611.925  opening working tree '/tmp/testbzr-qdLfKY.tmp'
> 
> ^^^^[log from
> bzrlib.tests.blackbox.test_serve.TestBzrServe.test_bzr_connect_to_bzr_ssh]
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/sw/local/bzr-1.10/bzrlib/tests/blackbox/test_serve.py",
> line 213, in test_bzr_connect_to_bzr_ssh
>     'bzr+ssh://fred:secret@localhost:%d%s' % (port, path_to_branch))
>   File "/home/sw/local/bzr-1.10/bzrlib/branch.py", line 123, in open
>     possible_transports=possible_transports)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 766, in open
>     return BzrDir.open_from_transport(t, _unsupported=_unsupported)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 808, in
> open_from_transport
>     return format.open(transport, _found=True)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 1770, in open
>     return self._open(transport)
>   File "/home/sw/local/bzr-1.10/bzrlib/bzrdir.py", line 2646, in _open
>     return remote.RemoteBzrDir(transport)
>   File "/home/sw/local/bzr-1.10/bzrlib/remote.py", line 93, in __init__
>     response = self._call('BzrDir.open', path)
>   File "/home/sw/local/bzr-1.10/bzrlib/remote.py", line 51, in _call
>     return self._client.call(method, *args)
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/client.py", line 122, in
> call
>     result, protocol = self.call_expecting_body(method, *args)
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/client.py", line 135, in
> call_expecting_body
>     method, args, expect_response_body=True)
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/client.py", line 83, in
> _call_and_read_response
>     expect_body=expect_response_body)
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/message.py", line 258, in
> read_response_tuple
>     self._wait_for_response_args()
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/message.py", line 224, in
> _wait_for_response_args
>     self._read_more()
>   File "/home/sw/local/bzr-1.10/bzrlib/smart/message.py", line 247, in
> _read_more
>     "(and try -Dhpss if further diagnosis is required)")
> ConnectionReset: Connection closed: please check connectivity and
> permissions (and try -Dhpss if further diagnosis is required)
> 

It's not obvious to me why this failed; perhaps an unexpected error on
the server side.


> blackbox.test_serve.TestBzrServe.test_bzr_serve_port_readonly                                                        FAIL                 140ms
>     not equal:
> a = 'bzr: interrupted\n'
> b = '<bzrlib.smart.medium.SmartServerSocketStreamMedium object at 0x209201590> terminating on exception (4, \'Interrupted system call\')\nException in thread smart-server-child:\nTraceback (most recent call last):\n  File "/usr/local/lib/python2.5/threading.py", line 486, in __bootstrap_inner\n    self.run()\n  File "/usr/local/lib/python2.5/threading.py", line 446, in run\n    self.__target(*self.__args, **self.__kwargs)\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 202, in serve\n    server_protocol = self._build_protocol()\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 217, in _build_protocol\n    bytes = self._get_line()\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 161, in _get_line\n    line, excess = _get_line(self.read_bytes)\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 99, in _get_line\n    new_bytes = read_bytes_func(1)\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 148, in read_bytes\n    return self._read_bytes(bytes_to_read)\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 277, in _read_bytes\n    return self.socket.recv(_MAX_READ_SIZE)\nerror: (4, \'Interrupted system call\')\n\nbzr: interrupted\n'
> 
> blackbox.test_serve.TestBzrServe.test_bzr_serve_port_readwrite                                                       FAIL                 270ms
>     not equal:
> a = 'bzr: interrupted\n'
> b = '<bzrlib.smart.medium.SmartServerSocketStreamMedium object at 0x20f639a50> terminating on exception (4, \'Interrupted system call\')\nException in thread smart-server-child:\nTraceback (most recent call last):\n  File "/usr/local/lib/python2.5/threading.py", line 486, in __bootstrap_inner\n    self.run()\n  File "/usr/local/lib/python2.5/threading.py", line 446, in run\n    self.__target(*self.__args, **self.__kwargs)\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 202, in serve\n    server_protocol = self._build_protocol()\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 217, in _build_protocol\n    bytes = self._get_line()\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 161, in _get_line\n    line, excess = _get_line(self.read_bytes)\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 99, in _get_line\n    new_bytes = read_bytes_func(1)\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 148, in read_bytes\n    return self._read_bytes(bytes_to_read)\n  File "/home/sw/local/bzr-1.10/bzrlib/smart/medium.py", line 277, in _read_bytes\n    return self.socket.recv(_MAX_READ_SIZE)\nerror: (4, \'Interrupted system call\')\n\nbzr: interrupted\n'
> 

This seems to be because we have some io code that should retry on
eintr, but that does not.  This could be a problem on any unix platform.

That's as far as I've got through the log, down to about line 800.

-- 
Martin