bzr pull does not remember urllib links

Short answer: uninstall pycurl.

Longer answer:

bzr < 2.5 urllib implementation does not verify certificates (including
self-signed ones).

http://pad.lv/651161 (part of 2.5b6) implements certificate verification
http://pad.lv/920455 (part of 2.5b6) provide sane defaults for CAs

So starting with bzr-2.5b6 you'll be able to use two options to control the
certificate verification:

- ssl.cert_reqs:

  Whether to require a certificate from the remote side. (default:required)

  Possible values:
   * none: Certificates ignored
   * required: Certificates required and validated

- ssl.ca_certs:

  Path to certification authority certificates to trust.

  This should be a valid path to a bundle containing all root Certificate
  Authorities used to verify an https server certificate.

  Use ssl.cert_reqs=none to disable certificate verification.

So to allow a connection to a self-signed server to succeed you can use:

  bzr pull -Ossl.cert_reqs=None

  You may want to define an alias for that if you need both secure and
  unsecure https access.

or set ``ssl.ca_certs`` to the path of your server certificate.

http://pad.lv/924220, once fixed patches welcome ;), will allow setting
these options on a per-host basis.

I'll mark this bug as a dupe of 924220, please comment if you find it inappropriate.

Another alternative would be for you to override the remembered pull location with:

bzr config parent_location=https+urllib://...

Shouldn't explicitely pulling from a remote https+urllib://... branch just use urllib instead of ignoring the '+urllib' and revert to using pycurl?

I have the following use case:

 1. I create a remote branch in a shared repository with: bzr push https+urllib://server/remote-branch
 2. Another developer branches this remote branch performs changes and pushed these changes back to https+urllib://server/remote-branch
 3. I try to pull the changes from the remote branch to my local branch with: bzr pull https+urllib://server/remote-branch
 4. Result: I get the obvious self-signed certificate error from pycurl

The question is: the above workflow seems to be perfectly valid: why is bzr reverting to use 'pycurl' in my 'bzr pull' command if I am telling it explicitly to use 'urllib'? It seems to me that this is a different bug than #924220.

Thanks in advance for any feedback regarding this.

Hmm, weird, can you retry the pull with -Dhttp and attach the relevant part of the ~/.bzr.log file ?

I can't figure out why the +urllib is dropped here and the log may help.

Please find attached the requested log (disclaimer: I replaced the actual hostname with 'localhost' for privacy reasons which shouldn't have altered anything).

Please find attached the requested log (disclaimer: I replaced the actual hostname with 'localhost' for privacy reasons which shouldn't have altered anything).

Soory for the delay (vacations), as the log shows, there is a redirection involved:

0.232 > GET /bzr/sandbox/trunk
0.232 > Host: localhost

0.313 < HTTP/1.1 301 Moved Permanently
0.313 < Date: Fri, 24 Feb 2012 09:09:57 GMT
< Server: Apache/2.2.20 (Ubuntu)
< location: https://localhost/bzr/sandbox/trunk/changes

Depending on why this redirection is in place you may want to use the final location (/bzr/sandbox/trunk/changes) instead of (/bzr/sandbox/trunk) to avoid the issue.

But I don't clearly understand where this redirection is coming from, what kind of server are you using ? Is there a bzr smart server involved ?

Hi Vincent, no worries. :)

Yes, there is a bzr smart server involved (and loggerhead is also running on the same webserver), so I assume the redirection is done by the smart server.

Still it doesn't seem to be correct that bzr is changing from using urllib to using curl just because the remote smart server is responding with a redirect. The expected behaviour for me would be to consistently use the same http handing library throughout a single bzr transaction. What are your thoughts on this?

Ha, ok (smart server and loggerhead).

Yes, it's a bug that indirections switch transport implementations

[19797] 2012-02-24 09:09:55.508 INFO: https://localhost/bzr/sandbox/trunk is permanently redirected to https://localhost/bzr/sandbox/trunk/changes

doesn't change host nor protocol so it should keep the the same implementation.

The weird thing is that I was pretty sure we had tests to guard against regression there...

Which bzr version are you using ? And can you retry again with older or newer ones (this is handled on the client side so no need to upgrade/downgrade the server).

Sorry, just seen you're using 2.4.1, still worth trying 2.5.0 and trunk if you can

Confirmed the 'switch from urllib to curl with smart server' behaviour with the latest trunk revision (6488).