parallel_pg_loader output fails if mfr switch is used and bbb_simple_rec_trigger exists

Bug #1022692 reported by Justin Hopkins
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Evergreen
Won't Fix
Medium
Unassigned

Bug Description

The output sql file will fail if the mfr switches are used. parallell_pg_loader writes the following to the end of the sql script:

SELECT reporter.enable_materialized_simple_record_trigger();
SELECT reporter.disable_materialized_simple_record_trigger();

This fails because the enable step tries to create the bbb_simple_rec_trigger which already exists. I believe the purpose of these two statements is/was to speed up the loading of records in the COPY statement by removing the trigger action that would otherwise run on each row being added. I moved the script bits around so that it first disables the trigger, then runs the COPY statement, then re-enables the trigger and rebuilds the table.

Committed a possible fix here:
http://git.evergreen-ils.org/?p=working/Evergreen.git;a=commit;h=f837691c132802faaf3306e8ea4c93b5f052c033

Revision history for this message
Warren Layton (warren-layton) wrote :

This bug is present in the 2.3.0 release tarball and will trip up anyone trying to import MARC records by following the standard import instructions.

tags: added: pullrequest
Changed in evergreen:
status: New → Confirmed
Ben Shum (bshum)
Changed in evergreen:
importance: Undecided → Medium
milestone: none → 2.4.0-rc
Ben Shum (bshum)
Changed in evergreen:
milestone: 2.4.0-rc → none
Ben Shum (bshum)
Changed in evergreen:
milestone: none → 2.5.0-m1
Dan Wells (dbw2)
Changed in evergreen:
milestone: 2.5.0-m1 → 2.5.0-m2
Revision history for this message
Dan Wells (dbw2) wrote :

I think the given commit is on the right track. One thing that concerns me is the history of this code. It looks like we are more or less reverting to what was there before, see here:

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=commitdiff;h=8595342b1153b7d562fb2d3f5a775e80b9c8d030

For more explanation of this change, read this:

http://libmail.georgialibraries.org/pipermail/open-ils-dev/2009-September/005081.html

It doesn't look like the zzz_update_materialized_simple_record_tgr exists anymore, so I don't think that but is valid, but I am stuck trying to understand how the fix ever made sense, and feel like I am missing something. In any case, if we do this fix, we probably want to do the same in pg_loader.pl (if anyone still uses that).

Dan Wells (dbw2)
Changed in evergreen:
milestone: 2.5.0-m2 → 2.5.0-alpha1
Revision history for this message
Mike Rylander (mrylander) wrote :

I think I'm missing something, here. In what situations should [parallel_]pg_loader.pl be used to generate COPY statements for anything but 'bre' objects in modern Evergreen?

I ask because the code that could generate 'mfr's is frighteningly out of date, and replaced by in-db ingest. Sorry if I'm misunderstanding ...

I think the fix is probably to warn on, and ignore the effect of, anything but '-or bre' and '-a bre' from the command line, WRT objects we should try to load.

Revision history for this message
Justin Hopkins (hopkinsju) wrote :

We were doing our first migrations a year ago when this was submitted, and have since dropped all switches other than '-or bre'.

Revision history for this message
Mike Rylander (mrylander) wrote :

Since the code in question is essentially dead (and, worse, possibly harmful) I've un-pullrequested this and marked it incomplete. Currently proposed plan of action is to trim *pg_loader.pl down to support only 'bre', 'are' and 'sre' objects, warn and ignore others.

Changed in evergreen:
status: Confirmed → Incomplete
tags: removed: pullrequest
Changed in evergreen:
milestone: 2.5.0-alpha1 → none
no longer affects: evergreen/2.3
no longer affects: evergreen/2.4
Changed in evergreen:
status: Incomplete → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.