Launchpad itself

Rosetta PO export is broken

Bug #81278 reported by Данило Шеган
16
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
Critical
Carlos Perelló Marín

Bug Description

Rosetta PO export is broken.

1. We don't know what's going on since we are hiding all the exceptions in rosetta-export-queue: to unhide them, we've got fix in r4437.

After this getting this cherrypicked, we found out this was due to incomplete merge of bug 73509 fix, missing security.cfg change (allow select on pomsgid for poexport user).

2. With exceptions disabled, When PO files contain invalid header fields which contain semicolon (':') in them, we may break with KeyError in poparser.py when reconstructing the header on export.

Revision history for this message
Данило Шеган (danilo) wrote :

Candidate for cherrypick (PO export queue is broken).

Changed in rosetta:
assignee: nobody → danilo
importance: Undecided → Critical
status: Unconfirmed → In Progress
Revision history for this message
Данило Шеган (danilo) wrote :

Case 2 above fixed in rev 4439.

Changed in rosetta:
status: In Progress → Fix Committed
Revision history for this message
Данило Шеган (danilo) wrote :

New issue: we need to catch MOCompilationError as well, which occurs when someone asks for MO export and there's a problem with generated PO file (i.e. "<stdin>:123: `msgid' and `msgstr[2]' entries do not both end with '\n'").

Changed in rosetta:
status: Fix Committed → In Progress
Revision history for this message
Stuart Bishop (stub) wrote :

launchpad@gangotri:/home/stub$ python /srv/launchpad.net/production/launchpad/cronscripts/rosetta-export-queue.py -vvv
04:39:02 INFO creating lockfile
04:39:04 DEBUG Exporting objects for person 1022515, PO template 13362
04:39:09 DEBUG Removing lock file: /var/lock/rosetta-export-queue.lock
Traceback (most recent call last):
  File "/srv/launchpad.net/production/launchpad/cronscripts/rosetta-export-queue.py", line 42, in ?
    sys.exit(main(sys.argv))
  File "/srv/launchpad.net/production/launchpad/cronscripts/rosetta-export-queue.py", line 35, in main
    process_queue(ztm, logger)
  File "/srv/launchpad.net/production/launchpad/cronscripts/../lib/canonical/launchpad/scripts/po_export_queue.py", line 459, in process_queue
    process_request(person, objects, format)
  File "/srv/launchpad.net/production/launchpad/cronscripts/../lib/canonical/launchpad/scripts/po_export_queue.py", line 436, in process_request
    result = process_multi_object_request(objects, format)
  File "/srv/launchpad.net/production/launchpad/cronscripts/../lib/canonical/launchpad/scripts/po_export_queue.py", line 400, in process_multi_object_request
    contents = handler.get_contents()
  File "/srv/launchpad.net/production/launchpad/cronscripts/../lib/canonical/launchpad/scripts/po_export_queue.py", line 109, in get_contents
    return compiler.compile(po_contents)
  File "/srv/launchpad.net/production/launchpad/lib/canonical/launchpad/components/poexport.py", line 942, in compile
    raise MOCompilationError("PO file compilation failed:\n" + stdout)
canonical.launchpad.components.poexport.MOCompilationError: PO file compilation failed:
<stdin>:123: `msgid' and `msgstr[2]' entries do not both end with '\n'
/usr/bin/msgfmt: found 1 fatal error

Revision history for this message
Carlos Perelló Marín (carlos) wrote :

Danilo had a patch for it + another one to log any other exception so we can still get the errors without having that script broken right now.

Seems like it didn't land last week so it should land today and be cherrypicked.

Revision history for this message
Matthew East (mdke) wrote :

My export request came through this morning :)

Revision history for this message
Данило Шеган (danilo) wrote :

Fixed in RF 4456.

Changed in rosetta:
status: In Progress → Fix Committed
Revision history for this message
Carlos Perelló Marín (carlos) wrote :

RF 4456 is a workaround, I'm going to prepare tests and, when possible, prevent those exceptions to be raised and if it cannot be avoided, notify the user.

Changed in rosetta:
assignee: danilo → carlos
status: Fix Committed → In Progress
Revision history for this message
Stuart Bishop (stub) wrote :

Please confirm that the tests are running as the correct db user while you are at it, as Bug 82118 demonstrated that more permissions need to be given to the poexport db user.

Revision history for this message
Carlos Perelló Marín (carlos) wrote :

Fixed in RF 4508

Changed in rosetta:
status: In Progress → Fix Released
William Grant (wgrant)
visibility: private → public
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.