Failure in lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeout

Bug #681770 reported by Jonathan Lange
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
Critical
Unassigned

Bug Description

In devel r11984 and stable r11979, this test fails:

$ ./bin/test -cvv lp.services.job.tests.test_runner test_timeout
Running tests at level 1
Running canonical.testing.layers.LaunchpadZopelessLayer tests:
  Set up canonical.testing.layers.BaseLayer in 0.003 seconds.
  Set up canonical.testing.layers.ZopelessLayer in 6.549 seconds.
  Set up canonical.testing.layers.DatabaseLayer in 0.332 seconds.
  Set up canonical.testing.layers.LibrarianLayer in 8.466 seconds.
  Set up canonical.testing.layers.MemcachedLayer in 0.121 seconds.
  Set up canonical.testing.layers.LaunchpadLayer in 0.000 seconds.
  Set up canonical.testing.layers.LaunchpadScriptLayer in 0.014 seconds.
  Set up canonical.testing.layers.LaunchpadZopelessLayer in 0.000 seconds.
  Running:
 lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeoutTraceback (most recent call last):
  File "/home/jml/src/launchpad/devel/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/internet/defer.py", line 249, in addCallbacks
    self._runCallbacks()
  File "/home/jml/src/launchpad/devel/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/internet/defer.py", line 441, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "/home/jml/src/launchpad/devel/lib/lp/services/twistedsupport/task.py", line 172, in got_task
    task_consumer.taskStarted(task)
  File "/home/jml/src/launchpad/devel/lib/lp/services/twistedsupport/task.py", line 306, in taskStarted
    d = defer.maybeDeferred(task)
--- <exception caught here> ---
  File "/home/jml/src/launchpad/devel/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/internet/defer.py", line 125, in maybeDeferred
    result = f(*args, **kw)
  File "/home/jml/src/launchpad/devel/lib/lp/services/job/runner.py", line 396, in <lambda>
    yield lambda: self.runJobInSubprocess(job)
  File "/home/jml/src/launchpad/devel/lib/lp/services/job/runner.py", line 370, in runJobInSubprocess
    RunJobCommand, job_id = job_id, _deadline=deadline)
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/pool.py", line 310, in doWork
    self.startAWorker()
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/pool.py", line 196, in startAWorker
    ampChildArgs=self.ampChildArgs)
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/main.py", line 243, in startAMPProcess
    return self.startPythonProcess(prot, *args)
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/main.py", line 259, in startPythonProcess
    usePTY=self.usePTY, packages=self.packages)
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/main.py", line 275, in spawnProcess
    p = os.path.split(imp.find_module(pkg)[1])[0]
exceptions.ImportError: No module named _pythonpath
Traceback (most recent call last):
  File "/home/jml/src/launchpad/devel/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/internet/defer.py", line 249, in addCallbacks
    self._runCallbacks()
  File "/home/jml/src/launchpad/devel/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/internet/defer.py", line 441, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "/home/jml/src/launchpad/devel/lib/lp/services/twistedsupport/task.py", line 172, in got_task
    task_consumer.taskStarted(task)
  File "/home/jml/src/launchpad/devel/lib/lp/services/twistedsupport/task.py", line 306, in taskStarted
    d = defer.maybeDeferred(task)
--- <exception caught here> ---
  File "/home/jml/src/launchpad/devel/eggs/Twisted-10.1.0-py2.6-linux-x86_64.egg/twisted/internet/defer.py", line 125, in maybeDeferred
    result = f(*args, **kw)
  File "/home/jml/src/launchpad/devel/lib/lp/services/job/runner.py", line 396, in <lambda>
    yield lambda: self.runJobInSubprocess(job)
  File "/home/jml/src/launchpad/devel/lib/lp/services/job/runner.py", line 370, in runJobInSubprocess
    RunJobCommand, job_id = job_id, _deadline=deadline)
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/pool.py", line 310, in doWork
    self.startAWorker()
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/pool.py", line 196, in startAWorker
    ampChildArgs=self.ampChildArgs)
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/main.py", line 243, in startAMPProcess
    return self.startPythonProcess(prot, *args)
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/main.py", line 259, in startPythonProcess
    usePTY=self.usePTY, packages=self.packages)
  File "/home/jml/src/launchpad/devel/eggs/ampoule-0.2.0-py2.6.egg/ampoule/main.py", line 275, in spawnProcess
    p = os.path.split(imp.find_module(pkg)[1])[0]
exceptions.ImportError: No module named _pythonpath

Failure in test lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeout
Traceback (most recent call last):
_StringException: Text attachment: librarian-log
------------
2010-11-26 18:20:04+0530 [-] Log opened.
2010-11-26 18:20:04+0530 [-] twistd 10.1.0 (/usr/bin/python 2.6.6) starting up.
2010-11-26 18:20:04+0530 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2010-11-26 18:20:04+0530 [-] canonical.librarian.libraryprotocol.FileUploadFactory starting on 59090
2010-11-26 18:20:04+0530 [-] Starting factory <canonical.librarian.libraryprotocol.FileUploadFactory instance at 0x3d4d440>
2010-11-26 18:20:04+0530 [-] twisted.web.server.Site starting on 58000
2010-11-26 18:20:04+0530 [-] Starting factory <twisted.web.server.Site instance at 0x3d4d710>
2010-11-26 18:20:04+0530 [-] canonical.librarian.libraryprotocol.FileUploadFactory starting on 59095
2010-11-26 18:20:04+0530 [-] Starting factory <canonical.librarian.libraryprotocol.FileUploadFactory instance at 0x3d4d8c0>
2010-11-26 18:20:04+0530 [-] twisted.web.server.Site starting on 58005
2010-11-26 18:20:04+0530 [-] Starting factory <twisted.web.server.Site instance at 0x3d4d7a0>
2010-11-26 18:20:04+0530 [-] Not using upstream librarian
2010-11-26 18:20:04+0530 [-] daemon ready!
2010-11-26 18:20:04+0530 [HTTPChannel,0,127.0.0.1] 127.0.0.1 - - [26/Nov/2010:12:50:04 +0000] "GET / HTTP/1.0" 200 340 "-" "Python-urllib/1.17"
2010-11-26 18:20:05+0530 [HTTPChannel,1,127.0.0.1] 127.0.0.1 - - [26/Nov/2010:12:50:04 +0000] "GET / HTTP/1.0" 200 340 "-" "Python-urllib/1.17"
------------
Text attachment: traceback
------------
Traceback (most recent call last):
  File "/home/jml/src/launchpad/devel/eggs/testtools-0.9.8_r137-py2.6.egg/testtools/runtest.py", line 143, in _run_user
    return fn(*args)
  File "/home/jml/src/launchpad/devel/eggs/testtools-0.9.8_r137-py2.6.egg/testtools/testcase.py", line 539, in _run_test_method
    return self._get_test_method()()
  File "/home/jml/src/launchpad/devel/lib/lp/services/job/tests/test_runner.py", line 390, in test_timeout
    self.assertEqual(1, len(runner.completed_jobs))
  File "/home/jml/src/launchpad/devel/eggs/testtools-0.9.8_r137-py2.6.egg/testtools/testcase.py", line 309, in assertEqual
    self.assertThat(observed, matcher)
  File "/home/jml/src/launchpad/devel/eggs/testtools-0.9.8_r137-py2.6.egg/testtools/testcase.py", line 385, in assertThat
    % (matchee, matcher, mismatch.describe()))
AssertionError: Match failed. Matchee: "0"
Matcher: Equals(1)
Difference: 1 != 0

------------

  Ran 1 tests with 1 failures and 0 errors in 3.487 seconds.
Tearing down left over layers:
  Tear down canonical.testing.layers.LaunchpadZopelessLayer in 0.000 seconds.
  Tear down canonical.testing.layers.LaunchpadScriptLayer in 0.002 seconds.
  Tear down canonical.testing.layers.ZopelessLayer ... not supported
  Tear down canonical.testing.layers.LaunchpadLayer in 0.000 seconds.
  Tear down canonical.testing.layers.LibrarianLayer in 0.002 seconds.
  Tear down canonical.testing.layers.MemcachedLayer in 0.101 seconds.
  Tear down canonical.testing.layers.DatabaseLayer in 0.028 seconds.
  Tear down canonical.testing.layers.BaseLayer in 0.001 seconds.

Tests with failures:
   lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeout (subunit.RemotedTestCase)

I have run 'make clean' and 'make schema' prior to this.

I'm not sure whether this fails on the buildbot. It is *not* failing on ec2.

Revision history for this message
Gary Poster (gary) wrote :

Duped.

Changed in launchpad-foundations:
importance: Undecided → Medium
status: New → Triaged
Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

This just happened in buildbot (though without the _pythonpath messages). Given the number of builds we've had to force, it may have gone wrong regularly and numbed us to the signal.

Bumping priority because we need to clear the foreign objects off our runway.

Changed in launchpad:
importance: Medium → Critical
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.