Celery branch scan jobs intermittantly fail to show revisions for new branches
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Launchpad itself |
Fix Released
|
High
|
Aaron Bentley |
Bug Description
Repeated pushes of the same local branch to new names on staging show that sometimes new revisions are shown, and sometimes not. ~abentley/
This appears to be due to races. The last scan of branch 52 reads:
[2012-04-16 19:23:55,669: INFO/PoolWorker-4] Adding 0 new revisions.
[2012-04-16 19:23:55,669: INFO/PoolWorker-4] Deleting 12 branchrevision records.
[2012-04-16 19:23:55,674: INFO/PoolWorker-4] Inserting 12 branchrevision records.
[2012-04-16 19:23:55,937: INFO/PoolWorker-4] Updating branch scanner status: 12 revs
[2012-04-16 19:23:55,982: INFO/MainProcess] Task lp.services.
The last scan of foo51 reads:
[2012-04-16 19:23:38,794: INFO/PoolWorker-2] Adding 0 new revisions.
[2012-04-16 19:23:38,794: INFO/PoolWorker-2] Deleting 0 branchrevision records.
[2012-04-16 19:23:38,794: INFO/PoolWorker-2] Inserting 12 branchrevision records
.
[2012-04-16 19:23:39,088: INFO/PoolWorker-2] Updating branch scanner status: 12
revs
This issue is not critical because it affects only staging, not production.
This issue is likely visible because an initial push will create two BranchScanJobs, increasing the likelihood of concurrent jobs. The current scan_branches script does not run jobs in parallel.
Related branches
- Aaron Bentley (community): Abstain
- Abel Deuring (community): Approve (code)
-
Diff: 220 lines (+93/-11)6 files modifiedlib/lp/code/model/branchjob.py (+14/-2)
lib/lp/services/database/locking.py (+51/-0)
lib/lp/services/database/stormexpr.py (+16/-0)
lib/lp/services/job/runner.py (+5/-2)
lib/lp/services/job/tests/test_runner.py (+6/-6)
versions.cfg (+1/-1)
description: | updated |
Changed in launchpad: | |
status: | New → Triaged |
importance: | Undecided → High |
tags: | added: celery-jobs |
tags: |
added: fast-slow-lane-jobs removed: celery-jobs |
tags: | added: branch-scanner |
description: | updated |
description: | updated |
description: | updated |
Changed in launchpad: | |
assignee: | nobody → Aaron Bentley (abentley) |
status: | Triaged → In Progress |
Changed in launchpad: | |
status: | Fix Committed → Fix Released |
status: | Fix Released → Fix Committed |
tags: |
added: qa-ok removed: qa-needstesting |
Changed in launchpad: | |
status: | Fix Committed → Fix Released |
Fixed in stable r15136 <http:// bazaar. launchpad. net/~launchpad- pqm/launchpad/ stable/ revision/ 15136>.