Celery branch scan jobs intermittantly fail to show revisions for new branches

Bug #983420 reported by Aaron Bentley
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Aaron Bentley

Bug Description

Repeated pushes of the same local branch to new names on staging show that sometimes new revisions are shown, and sometimes not. ~abentley/+junk/foo49-foo52 all have the same contents, but 49-51 show new revisions, while 52 does not.

This appears to be due to races. The last scan of branch 52 reads:
[2012-04-16 19:23:55,669: INFO/PoolWorker-4] Adding 0 new revisions.
[2012-04-16 19:23:55,669: INFO/PoolWorker-4] Deleting 12 branchrevision records.
[2012-04-16 19:23:55,674: INFO/PoolWorker-4] Inserting 12 branchrevision records.
[2012-04-16 19:23:55,937: INFO/PoolWorker-4] Updating branch scanner status: 12 revs
[2012-04-16 19:23:55,982: INFO/MainProcess] Task lp.services.job.celeryjob.CeleryRunJob[26ab04ab-a295-46bb-9245-14af1af88ed4] succeeded in 0.483397006989s: None

The last scan of foo51 reads:
[2012-04-16 19:23:38,794: INFO/PoolWorker-2] Adding 0 new revisions.
[2012-04-16 19:23:38,794: INFO/PoolWorker-2] Deleting 0 branchrevision records.
[2012-04-16 19:23:38,794: INFO/PoolWorker-2] Inserting 12 branchrevision records
.
[2012-04-16 19:23:39,088: INFO/PoolWorker-2] Updating branch scanner status: 12
revs

This issue is not critical because it affects only staging, not production.

This issue is likely visible because an initial push will create two BranchScanJobs, increasing the likelihood of concurrent jobs. The current scan_branches script does not run jobs in parallel.

Related branches

Aaron Bentley (abentley)
description: updated
Changed in launchpad:
status: New → Triaged
importance: Undecided → High
tags: added: celery-jobs
tags: added: fast-slow-lane-jobs
removed: celery-jobs
tags: added: branch-scanner
description: updated
description: updated
Aaron Bentley (abentley)
description: updated
Aaron Bentley (abentley)
Changed in launchpad:
assignee: nobody → Aaron Bentley (abentley)
status: Triaged → In Progress
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :
tags: added: qa-needstesting
Changed in launchpad:
status: In Progress → Fix Committed
William Grant (wgrant)
Changed in launchpad:
status: Fix Committed → Fix Released
status: Fix Released → Fix Committed
Aaron Bentley (abentley)
tags: added: qa-ok
removed: qa-needstesting
William Grant (wgrant)
Changed in launchpad:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.