Need better log_debug output from lib/db/upgrade.php

Bug #1450680 reported by Robert Lyon
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mahara
Fix Released
Medium
Robert Lyon
15.10
Fix Released
Undecided
Unassigned

Bug Description

One of the most annoying things about db locking up during upgrade is you don't know where abouts in the upgrade you are - because it doesn't error out.

It would be most useful if there was:

1) At least a log_debug line at the start of every if version block - there are often comments there so just turn them into log_debug() comments.

2) Every time we loop over a bunch of results we should have a loop counter, so upgrader knows things are still running eg:
  if ($results) {
    $count = 0;
    $limit = 1000; // where limit is some appropriate number
    $total = count($results);
    foreach ($results as $result) {

       ... the purpose happens here ...

      $count++;
      if (($count % $limit) == 0 || $count == $total) {
          log_debug("$count/$total");
          set_time_limit(30);
      }
  }

Robert Lyon (robertl-9)
Changed in mahara:
status: New → Confirmed
Revision history for this message
Aaron Wells (u-aaronw) wrote :

While we're at it, it would also be good to save the upgrade state after each of those blocks as well. So that if you make it through three blocks and fail on the fourth, you can try again and start at the fourth.

Which I think is sort of how Moodle does it. Maybe it's time we refactor our upgrade a little? Have a method you call at the start of each of these blocks, that logs and updates the version number.

Revision history for this message
Robert Lyon (robertl-9) wrote :

If we were to go that way we would have to stop wrapping the whole upgrade in a db_begin/db_commit but somehow call it for each block

Revision history for this message
Aaron Wells (u-aaronw) wrote :

Yes, currently we do the entire upgrade within a single database transaction. It would probably improve performance to break that up into smaller transactions. When you have a huge amount of changes inside of a transaction, it can eat up a lot of database resources, and it also creates the potential for blocking & locking.

Revision history for this message
Mahara Bot (dev-mahara) wrote : A patch has been submitted for review

Patch for "master" branch: https://reviews.mahara.org/4724

Robert Lyon (robertl-9)
Changed in mahara:
status: Confirmed → In Progress
assignee: nobody → Robert Lyon (robertl-9)
milestone: none → 15.10.0
importance: Undecided → Medium
Revision history for this message
Mahara Bot (dev-mahara) wrote : A change has been merged

Reviewed: https://reviews.mahara.org/4724
Committed: http://gitorious.org/mahara/mahara/commit/4757df6fcbbba4afbeada6be8c4b23c9fe98284b
Submitter: Son Nguyen (<email address hidden>)
Branch: master

commit 4757df6fcbbba4afbeada6be8c4b23c9fe98284b
Author: Robert Lyon <email address hidden>
Date: Fri May 1 13:38:22 2015 +1200

Bug 1450680 - adding in more debugging info to upgrade.php

This will help us better identify problems when upgrading sites and
also allow the community to post more helpful info about their
problems.

Currently patch has only the info I used to help debug a site I was
trying to upgrade from 1.9 -> 15.04

Change-Id: I4918e2e76477028d1bef338d84e662d367378020
Signed-off-by: Robert Lyon <email address hidden>

Revision history for this message
Aaron Wells (u-aaronw) wrote :

The patch that was merged just adds a log_debug() message to the top of each upgrade block. I suppose that is a quick and fairly lightweight way to get the job done. :)

I'll think some more about whether we actually want to split the upgrade up into multiple transactions as I was discussing. If we do, I'll file a separate bug for that.

Revision history for this message
Robert Lyon (robertl-9) wrote :

Yeah - I left this bug as 'in progress' as it didn't deal with the idea of multiple transactions mentioned earlier.

Revision history for this message
Aaron Wells (u-aaronw) wrote :

Let's call this one "fix committed", and open a separate bug if we decide to go with multiple transactions.

Changed in mahara:
status: In Progress → Fix Committed
Robert Lyon (robertl-9)
Changed in mahara:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.