Launchpad itself

Reprioritize package build scores based on packageset

Bug #990219 reported by Scott Kitterman
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
Low
Colin Watson

Bug Description

As proposed in https://lists.ubuntu.com/archives/ubuntu-devel/2012-April/035049.html it would help with keeping daily ISOs buildable if seeded packages were built before unseeded packages (as an implementation detail, Launchpad knows about packagesets, not seeds, so the solution to this problem should be packageset based. The current design is https://help.launchpad.net/Packaging/BuildScores. As an initial proposal, packages in seeds that are used for ISO builds should get +750 in their build priority.

lib/lp/soyuz/model/buildpackagejob.py:BuildPackageJob.score()

<cjwatson> ScottK: Did we think it should be different for different packagesets, or just a bonus for being in a packageset at all? (The latter is a lot easier since I don't have to figure out where to put the scores.)
<ScottK> I think it's got to be specific packagesets get a bonus.
<ScottK> There are pakcagesets like mono that are completely unrelated to this question.
<cjwatson> ScottK: It might be as simple as adding a score column to Packageset, defaulting to zero, and adding an API method to set it.
<cjwatson> That wouldn't be desperately hard.
<cjwatson> Then we could tweak the scores fairly freely and it wouldn't have to be analysed all that carefully in advance.
<cjwatson> I'd also rather change a relatively little-used table like Packageset than a table that gets hit all the time like DistributionSourcePackage ...
<wgrant> ScottK, cjwatson: I'd go with the priority on packageset.
<wgrant> Pretty simple.
<cjwatson> You reckon my assessment above (add column on packageset, export on webservice, add in .score) is about right?
<wgrant> cjwatson: Yeah
<cjwatson> Right, that's well within my capacity then
<cjwatson> ScottK: file bug, assign to me, I'll take it from there
<ScottK> cjwatson: Thanks. Will do.

Related branches

lp:~cjwatson/launchpad/db-packageset-score
Stuart Bishop (community): Approve (db)
Robert Collins: Pending (db) requested
Launchpad code reviewers: Pending requested
Diff: 12 lines (+8/-0)
1 file modified
database/schema/patch-2209-18-0.sql (+8/-0)
lp:~cjwatson/launchpad/packageset-score
j.c.sackett (community): Approve
Diff: 89 lines (+12/-10)
4 files modified
lib/lp/soyuz/interfaces/packageset.py (+1/-1)
lib/lp/soyuz/model/buildpackagejob.py (+1/-1)
lib/lp/soyuz/model/packageset.py (+1/-1)
lib/lp/soyuz/tests/test_buildpackagejob.py (+9/-7)
William Grant (wgrant)
Changed in launchpad:
assignee: nobody → Colin Watson (cjwatson)
importance: Undecided → Low
status: New → Triaged
tags: added: feature packagesets soyuz-build
Colin Watson (cjwatson)
Changed in launchpad:
status: Triaged → In Progress
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r11582 in db-stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-stable/revision/11582) is part of this bug's fix.

tags: added: qa-needstesting
Revision history for this message
Colin Watson (cjwatson) wrote :

07:12 <wgrant> 2012-05-10 04:24:20 INFO 2209-18-0 applied just now in 0.6 seconds

And stuff related to packagesets (like +queue and edit_acl.py from lp:ubuntu-archive-tools) still seems to work fine.

tags: added: qa-ok
removed: qa-needstesting
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

Fixed in stable r15264 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/15264>.

tags: added: qa-needstesting
removed: qa-ok
Changed in launchpad:
status: In Progress → Fix Committed
Revision history for this message
Colin Watson (cjwatson) wrote :

On dogfood, I checked that the "hello" package was already in the "core" packageset, gave core a relative_build_score of 2000, and uploaded a test build of hello. The score was correctly set to 4505.

I then deactivated ~techboard as a member of ~launchpad-buildd-admins, thus temporarily removing my own membership of that team, and verified that now I can see the build score but not set it, even though I still have upload privileges:

  >>> core.relative_build_score = 0
  >>> core.lp_save()
  Traceback (most recent call last):
  [...]
  Unauthorized: HTTP Error 401: Unauthorized
  [...]
  (<lp.soyuz.model.packageset.Packageset object at 0xf847dcc>, 'relative_build_score', 'launchpad.Moderate')

So this seems to be behaving as intended now.

tags: added: qa-ok
removed: qa-needstesting
William Grant (wgrant)
Changed in launchpad:
status: Fix Committed → Fix Released
Revision history for this message
Colin Watson (cjwatson) wrote :

I've applied some initial scoring in production. I made a rather more conservative increase than Scott's initial suggestion, because +750 would cause test rebuilds to score many things in main/release above zero (5 + 1000 + 1500 + 750 - 2600 == 655) and that would have undesirable effects. While we could change the copy archive penalty, I think it's sufficient to just apply a rather smaller bonus of +50 to package sets that contribute to image builds. This keeps test rebuilds below zero, but still allows anything in universe that contributes to an image build to beat anything else in universe, which is the principal desired effect here.

  >>> quantal_ps = {ps.name: ps for ps in quantal_ps}
  >>> for image_ps_name in "core desktop-core edubuntu kubuntu lubuntu mythbuntu ubuntu-desktop ubuntu-server ubuntustudio xubuntu".split():
  ... image_ps = quantal_ps[image_ps_name]
  ... image_ps.relative_build_score = 50
  ... image_ps.lp_save()
  ...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.