Trac status imports can't cope with Trac instances that don't export individual bugs in CSV format

Bug #134604 reported by Graham Binns
2
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Graham Binns

Bug Description

Some Trac instances don't allow the exports of individual bugs in CSV format, either due to age or because the owner of the Trac instance has disabled this functionality. At present these instances cause the Trac status import code to fail because it can't extract a status from the remote tracker, though this only happens when the bug tracker has few er bug watches needing to be checked than would trigger the batch status import code.

In order to fix this the Trac status import code needs to be able to detect these Trac instances and act appropriately. Since most of them still export the status field when returning all of their bugs as a CSV (as opposed to just one at a time), the status import code can use its batch import code to retrieve bugs from these trackers instead of its
one-bug-at-a-time code.

There are other Trac instances which do not export bugs as a CSV at all. The status import code must deal with these, too.

Tags: lp-bugs
Graham Binns (gmb)
Changed in malone:
assignee: nobody → gmb
importance: Undecided → High
status: New → Confirmed
description: updated
Graham Binns (gmb)
Changed in malone:
status: Confirmed → In Progress
Revision history for this message
Graham Binns (gmb) wrote : Re: [Bug 134604] Re: Trac status imports can't cope with Trac instances that don't export individual bugs in CSV format

The problem occurs because some Tracs do not behave properly when failing to export individual bugs in CSV format.

Ordinarily, a Trac instance which doesn't support CSV exports would return an HTTP 500 error when a request to a URL of the form /ticket/[0-9]+\?format=csv was made. The Trac status import code deals with these correctly.

However, the Tracs that were causing the import code to fail (http://trac.galago-project.org/ and http://hackage.haskell.org/trac/ghc are examples) don't do this; they simply return the ticket in HTML format. The import code assumes that the data it receives is valid because it receives an HTTP 200. However it cannot parse the HTML code and so fails when the status for a given bug is retrieved.

Changed in malone:
status: In Progress → Triaged
Revision history for this message
James Henstridge (jamesh) wrote :

How about checking the content-type of the HTTP response? If it is a text/html mime type, then it probably isn't CSV ...

Revision history for this message
Graham Binns (gmb) wrote :

> How about checking the content-type of the HTTP response? If it is a text/html mime type, then it probably isn't CSV ...

Exactly my thinking :)

Graham Binns (gmb)
Changed in malone:
status: Triaged → In Progress
Revision history for this message
Graham Binns (gmb) wrote :

Fix in review.

Revision history for this message
Graham Binns (gmb) wrote :

 status fixcommitted

Changed in malone:
status: In Progress → Fix Committed
Graham Binns (gmb)
Changed in malone:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.