BugTask:+index timeouts - distributions and milestones being late evaluation loaded - repeatedly - on bug 230350

Bug #724033 reported by Robert Collins
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
Critical
Unassigned

Bug Description

OOPS-1895K1459
OOPS-1895C1447
 OOPS-1927B436

https://launchpad.net/bugs/230350

74 milestone lookups:
2 74 723 9 714 SQL-launchpad-main-slave
SELECT Milestone.active, Milestone.codename, Milestone.dateexpected, Milestone.distribution, Milestone.distroseries, Milestone.id, Milestone.name, Milestone.product, Milestone.productseries, Milestone.summary
FROM Milestone
WHERE Milestone.distribution = $INT
  AND Milestone.active = TRUE
ORDER BY milestone_sort_key(Milestone.dateexpected, Milestone.name) DESC

73 distribution lookups:
3 73 496 6 490 SQL-launchpad-main-slave
SELECT Distribution.answers_usage,
       Distribution.blueprints_usage,
       Distribution.translations_usage,
       Distribution.bug_reported_acknowledgement,
       Distribution.bug_reporting_guidelines,
       Distribution.bug_supervisor,
       Distribution.date_created,
       Distribution.description,
       Distribution.displayname,
       Distribution.domainname,
       Distribution.driver,
       Distribution.enable_bug_expiration,
       Distribution.homepage_content,
       Distribution.icon,
       Distribution.id,
       Distribution.language_pack_admin,
       Distribution.logo,
       Distribution.max_bug_heat,
       Distribution.members,
       Distribution.mirror_admin,
       Distribution.mugshot,
       Distribution.name,
       Distribution.official_answers,
       Distribution.official_blueprints,
       Distribution.official_malone,
       Distribution.official_rosetta,
       Distribution.OWNER, Distribution.security_contact,
                           Distribution.summary,
                           Distribution.title,
                           Distribution.translation_focus,
                           Distribution.translationgroup,
                           Distribution.translationpermission
FROM Distribution
WHERE id IN ($INT)
ORDER BY Distribution.name

Tags: timeout

Related branches

description: updated
Revision history for this message
Robert Collins (lifeless) wrote :
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :
Changed in launchpad:
assignee: nobody → Robert Collins (lifeless)
milestone: none → 11.03
tags: added: qa-needstesting
Changed in launchpad:
status: Triaged → Fix Committed
Revision history for this message
Robert Collins (lifeless) wrote :

The bug in question still does not render - 10 seconds of python time to track down, but the late loading issue is resolved, and other expensive bugs - like bug 1 - still render fine. Will profile later and either build on this bug or file a new one depending on the cause.

tags: added: qa-ok
removed: qa-needstesting
William Grant (wgrant)
Changed in launchpad:
status: Fix Committed → Fix Released
Revision history for this message
Robert Collins (lifeless) wrote :

OOPS-1888C48 with this deployed.

Changed in launchpad:
status: Fix Released → In Progress
Revision history for this message
Robert Collins (lifeless) wrote :

Remaining milestone and distro lookups appear to be due to BugTaskTableRowView's _visible_milestones property, which does a milestone vocab per row, and that triggers a lookup of the milestones for the bugtask context, and a lookup of the contexts found by the vocabulary. Some refactoring is in order.

Curtis Hovey (sinzui)
Changed in launchpad:
milestone: 11.03 → none
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :
Changed in launchpad:
milestone: none → 11.03
tags: added: qa-needstesting
removed: qa-ok
Changed in launchpad:
status: In Progress → Fix Committed
Revision history for this message
Robert Collins (lifeless) wrote :

So, the changes in this branch so far still don't get the reported bug rendering, but we're driving the sql time down:

SQL time: 2166 ms
Non-sql time: 12334 ms
Total time: 14500 ms
Statement Count: 286

tags: added: qa-ok
removed: qa-needstesting
Curtis Hovey (sinzui)
Changed in launchpad:
status: Fix Committed → Fix Released
Changed in launchpad:
status: Fix Released → Triaged
assignee: Robert Collins (lifeless) → nobody
milestone: 11.03 → none
tags: removed: qa-ok
description: updated
description: updated
description: updated
Revision history for this message
William Grant (wgrant) wrote :

OOPS-1943DR70 has a 5s gap between query 61 and 62. Not sure what's going on there.

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 724033] Re: BugTask:+index timeouts - distributions and milestones being late evaluation loaded - repeatedly - on bug 230350

On Wed, Apr 27, 2011 at 6:21 PM, William Grant <email address hidden> wrote:
> OOPS-1943DR70 has a 5s gap between query 61 and 62. Not sure what's
> going on there.

Some possibilities (excluding cosmic rays):
 - uninstrumented expensive code - check the code between the queries
 - thread lockout with a silly-expensive xmlrpc call
 - gc kicking in (particularly if we were slightly in swap) - check
machine graphs
 - possibly log rotate signal kicking in? - check the appserver trace.log
 - cosmic rays

Revision history for this message
William Grant (wgrant) wrote :

The late evaluation mentioned here is gone, but excessive CPU time remains (bug #724025).

Changed in launchpad:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.