_bad_oauth_token crashes on python3 (str vs bytes)

Bug #1471894 reported by Zygmunt Krynicki
36
This bug affects 5 people
Affects Status Importance Assigned to Milestone
launchpadlib
Fix Released
Undecided
Unassigned
python-launchpadlib (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
Medium
Dimitri John Ledkov

Bug Description

[Impact]

 * Unable to renew/relogin with a new oauth token, using python3 scripts
 * python2 APIs work (because python2 does not care about str vs bytes)

[Test Case]

 * Attempt login_with() using python3 and an expired/invalid existing token
 * Relogin should be successful, without crashing scripts

[Regression Potential]

 * No change in behaviour on python2 (which is what used by reverse dependencies)
 * this is a backport to fix python3 behaviour, for all the newly ported maintainance scripts in python3 that prefer to run on "stable" systems

[Other Info]

 * Well tested in later releases, and with users running from trunk.
 * I believe this still is not published in the cheeseshop.

[Original bug report]

Hi

Running something as simple as

>>> from launchpadlib.launchpad import Launchpad
>>> launchpad = Launchpad.login_with("app", "production")

Results in a traceback such as this one:

Traceback (most recent call last):
  File "./foo.py", line 31, in <module>
    lb = launchpad.projects[project]
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/resource.py", line 1001, in __getitem__
    shim_resource._ensure_representation()
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/resource.py", line 382, in _ensure_representation
    representation = self._root._browser.get(self._wadl_resource)
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/_browser.py", line 436, in get
    response, content = self._request(url, extra_headers=headers)
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/_browser.py", line 387, in _request
    str(url), method=method, body=data, headers=headers)
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/_browser.py", line 357, in _request_and_retry
    url, method=method, body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1291, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 132, in _request
    return self.retry_on_bad_token(response, content, *args)
  File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 139, in retry_on_bad_token
    if (self._bad_oauth_token(response, content)
  File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 125, in _bad_oauth_token
    (content.startswith("Expired token")
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

This doesn't happen on python2. I will follow up with a patch shortly.

Related branches

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

I've added a quick patch that I cooked for this issue. Have a look to say if I'm on the right track. This builds cleanly for me in the ppa:zyga/launchpadlib-backports in a trusty sbuild chroot.

Colin Watson (cjwatson)
Changed in launchpadlib:
status: New → Fix Committed
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Could we get this fix released and SRUed to previous releases as well? Thank you!

Revision history for this message
Robert Bruce Park (robru) wrote :

I doubt it will be SRU'd because python 3 is not officially supported and didn't exist before xenial anyway.

But I'm OK with that because I'll be upgrading the train to xenial asap after it is released for other reasons anyway, so as long as this fix gets into xenial I'm happy.

Revision history for this message
Colin Watson (cjwatson) wrote :

1.10.4 (2016-07-12)
===================
- Fix _bad_oauth_token crash on Python 3. [bug=1471894]
- Time out make_end_user_authorize_token after 15 minutes.
- Ignore PendingDeprecationWarning from lazr.restfulclient. [bug=1473577]
- Ask forgiveness rather than permission when creating cache directories.
- Fix browser token authorization on OS X. [bug=1516080]

Changed in launchpadlib:
status: Fix Committed → Fix Released
Changed in python-launchpadlib (Ubuntu):
status: New → Fix Released
Changed in python-launchpadlib (Ubuntu Xenial):
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Dimitri John Ledkov (xnox)
milestone: none → ubuntu-16.04.2
Changed in python-launchpadlib (Ubuntu Xenial):
status: Triaged → In Progress
Revision history for this message
Brian Murray (brian-murray) wrote : Missing SRU information

Thanks for uploading the fix for this bug report to -proposed. However, when reviewing the package in -proposed and the details of this bug report I noticed that the bug description is missing information required for the SRU process. You can find full details at http://wiki.ubuntu.com/StableReleaseUpdates#Procedure but essentially this bug is missing some of the following: a statement of impact, a test case and details regarding the regression potential. Thanks in advance!

description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Zygmunt, or anyone else affected,

Accepted python-launchpadlib into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-launchpadlib/1.10.3-3ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python-launchpadlib (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Tiago Stürmer Daitx (tdaitx) wrote :

Yes, I can confirm that the package in -proposed fixes that.

Now:
$ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> from launchpadlib.launchpad import Launchpad
>>> def no_credential():
... print("Can't proceed without Launchpad credential.")
... sys.exit()
...
>>> launchpad = Launchpad.login_with('Test', 'production', credential_save_failed=no_credential, version='devel')
>>> me = launchpad.me
>>> me.getPPAByName(name="openjdk")
The authorization page:
 (https://launchpad.net/+authorize-token?oauth_token=NotMyRealTokenOfCourse&allow_permission=DESKTOP_INTEGRATION)
should be opening in your browser. Use your browser to authorize
this program to access Launchpad on your behalf.
Waiting to hear from Launchpad about your decision...
Created new window in existing browser session.
<archive at https://api.launchpad.net/devel/~tdaitx/+archive/ubuntu/openjdk>

Before:
$ python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> from launchpadlib.launchpad import Launchpad
>>> def no_credential():
... print("Can't proceed without Launchpad credential.")
... sys.exit()
...
>>> launchpad = Launchpad.login_with('Test', 'production', credential_save_failed=no_credential, version='devel')
>>> me = launchpad.me
>>> me.getPPAByName(name="openjdk")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/resource.py", line 609, in __call__
    extra_headers=extra_headers)
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/_browser.py", line 387, in _request
    str(url), method=method, body=data, headers=headers)
  File "/usr/lib/python3/dist-packages/lazr/restfulclient/_browser.py", line 357, in _request_and_retry
    url, method=method, body=body, headers=headers)
  File "/usr/lib/python3/dist-packages/httplib2/__init__.py", line 1312, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 132, in _request
    return self.retry_on_bad_token(response, content, *args)
  File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 139, in retry_on_bad_token
    if (self._bad_oauth_token(response, content)
  File "/usr/lib/python3/dist-packages/launchpadlib/launchpad.py", line 125, in _bad_oauth_token
    (content.startswith(b"Expired token")
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

tags: added: verification-done
removed: verification-needed
Revision history for this message
Tiago Stürmer Daitx (tdaitx) wrote :

And the exact version tested:

$ apt-cache policy python3-launchpadlib
python3-launchpadlib:
  Installed: 1.10.3-3ubuntu0.1
  Candidate: 1.10.3-3ubuntu0.1
  Version table:
 *** 1.10.3-3ubuntu0.1 400
        400 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages
        400 http://archive.ubuntu.com/ubuntu xenial-proposed/main i386 Packages
        100 /var/lib/dpkg/status
     1.10.3-3 500
        500 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages
        500 http://archive.ubuntu.com/ubuntu xenial/main i386 Packages

Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of the Stable Release Update for python-launchpadlib has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-launchpadlib - 1.10.3-3ubuntu0.1

---------------
python-launchpadlib (1.10.3-3ubuntu0.1) xenial; urgency=medium

  * Cherrypick python3 crash fixes:
    - _bad_oauth_token (LP: #1471894)
    - AccessToken.from_string() (LP: #1471927)

 -- Dimitri John Ledkov <email address hidden> Thu, 22 Dec 2016 15:05:54 +0000

Changed in python-launchpadlib (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.