many packages fail to import due to HTTPError talking to Launchpad (eg ubuntu:transmission)

This error is caused by:

    lp_url = lpapi.SERVICE_ROOT \
              + lp_side_branch_path[lp_side_branch_path.index("~"):]
    lp_branch = lpapi.lp_call(lp.load, lp_url)

I think the problem is that SERVICE_ROOT no longer includes the /beta/ or whatever version.

Perhaps we are using the lplib version that allows URLs without the host? Then we could
just do the url splitting and pass it to lp.load without putting the SERVICE_ROOT in
front.

Thanks,

James

Thanks for analyzing that James, it sounds like it's the same as <https://bugs.launchpad.net/bzr/+bug/707075/comments/11>.

This probably broke recently when we upgraded jubany to lucid.

Max put a workaround for this into bzr in <http://bazaar.launchpad.net/~maxb/bzr/launchpadlib-service-root-api-compat/revision/4875> and perhaps the easiest thing is just to get the correct version from bzr.

I rolled out my attempted fix, but that's apparently not fixed it:

2011-08-11 06:44:57,918 - __main__ - WARNING - Importing transmission failed:
Object: <canonical.launchpad.webapp.servers.WebServiceClientRequest instance URL=https://api.launchpad.net>, name: ''
Traceback (most recent call last):
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 1124, in <module>
    only_before=options.only_before))
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 1000, in main
    revid_db.cleanup_last_run(cleanup, push_back)
  File "/srv/package-import.canonical.com/new/scripts/udd/icommon.py", line 1107, in cleanup_last_run
    push_cb()
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 994, in push_back
    possible_transports, local_branches)
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 734, in push_branches_back
    bstore, lp_side_branch_path)
  File "/srv/package-import.canonical.com/new/scripts/import_package.py", line 504, in file_mp
    lp_branch = lpapi.lp_call(lp.load, lp_url)
  File "/srv/package-import.canonical.com/new/scripts/udd/lpapi.py", line 60, in lp_call
    return callable(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/lazr/restfulclient/resource.py", line 459, in load
    document = self._browser.get(url)
  File "/usr/lib/pymodules/python2.6/lazr/restfulclient/_browser.py", line 316, in get
    response, content = self._request(url, extra_headers=headers)
  File "/usr/lib/pymodules/python2.6/lazr/restfulclient/_browser.py", line 306, in _request
    raise HTTPError(response, content)
lazr.restfulclient.errors.HTTPError: HTTP Error 404: Not Found
Response headers:

With some code inserted to log about the failure, it seems the importer is trying to read
<https://api.launchpad.net/~ubuntu-branches/ubuntu/oneiric/cloud-utils/oneiric-201108110905> and the problem is that it's missing the api version component (eg '/beta')

and I think the specific problem is that file_mp is making up a url by string manipulation, and it's omitting the /1.0 component.

... and that was basically fallout from http://bazaar.launchpad.net/+branch/launchpadlib/revision/82 which took that component out of the predefined urls.

@poolie: Got a fix that unblocked semantik, I'm going to try the others

I had to test on jubany as this required access to lp itself and the package importer access rights...

The only fallout is now for libraw but for a different error:

lazr.restfulclient.errors.HTTPError: HTTP Error 400: Bad Request
Response headers:
---
connection: close
content-length: 177
content-type: text/plain
date: Fri, 02 Sep 2011 22:56:06 GMT
server: zope.server.http (HTTP)
status: 400
vary: Accept,Accept-Encoding
via: 1.1 api.launchpad.net
x-lazr-notifications: []
x-powered-by: Zope (www.zope.org), Python (www.python.org)
---
Response body:
---
There is already a branch merge proposal registered for branch lp:~ubuntu-branches/ubuntu/natty/libraw/natty-201103250146 to land on lp:ubuntu/natty/libraw that is still active.

Not sure if that should be considered part of this bug...

@poolie: Do you remember testing something with libraw ? Could this fallout be seen as a fallout from some error that happened *after* the mp was filed (2011-03-25 !!!) or should we just silently catch new attempts to file an mp more than noce ?