update-manager stopped loading update descriptions / changelog

Status changed to 'Confirmed' because the bug affects multiple users.

Software Updater fail to load the list of changes.
Version 1:16.04.16.
Ubuntu 16.04 LTS with all updates.

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3/dist-packages/UpdateManager/Core/MyCache.py", line 320, in get_news_and_changelog
    self.get_changelog(name)
  File "/usr/lib/python3/dist-packages/UpdateManager/Core/MyCache.py", line 376, in get_changelog
    changelog = self._get_changelog_or_news(name, "changelog")
  File "/usr/lib/python3/dist-packages/UpdateManager/Core/MyCache.py", line 245, in _get_changelog_or_news
    "https locations with username/password are not"
UpdateManager.Core.MyCache.HttpsChangelogsUnsupportedError: https locations with username/password are notsupported to fetch changelogs

I thing there is a bug in MyCache.py :

`if res.scheme == "https" and res.username != "":`

The value of `res.username` is `None` when it is not set, not the empty string.

Code from UpdateManager/Core/MyCache.py :

```
    def _get_changelog_or_news(self, name, fname, strict_versioning=False,
                               changelogs_uri=None):
        " helper that fetches the file in question "
    ...
        # https uris are not supported when they contain a username/password
        # because the urllib2 https implementation will not check certificates
        # and so its possible to do a man-in-the-middle attack to steal the
        # credentials
        res = urlsplit(uri)
        if res.scheme == "https" and res.username != "":
            raise HttpsChangelogsUnsupportedError(
                "https locations with username/password are not"
                "supported to fetch changelogs")
```

Confirmed, it's a regression from the recent Xenial SRU, changelog fetching is not working on a normal connection without proxy in use

This has been fixed since Ubuntu 16.10.

update-manager (1:16.10.3) yakkety; urgency=medium

  [Nicolas Delvaux]
  * Attempt to retrieve Changelogs from PPA sources (LP: #253119)
  * Correctly detect the usage of a username in changelog URIs

 -- Brian Murray <email address hidden> Wed, 27 Jul 2016 11:14:53 -0700

Could somebody add a test case to this bug report? That is an important part of the Stable Release Update process and would help a lot.

Test:
Ubuntu 16.04 LTS.
Something to update in Software Updater from archive ubuntu (NOT from a PPA).
Open a terminal to launch update-manager
In the window "Software Updater", click "Technical description" to open the "Changes" panel.
Click a line in "Details of updates" panel.
In the "Changes" panel we see "Downloading list of changes..." for ever.
In the terminal we see a exception : UpdateManager.Core.MyCache.HttpsChangelogsUnsupportedError: https locations with username/password are notsupported to fetch changelogs.

This is because of this line in MyCache.py
    if res.scheme == "https" and res.username != "":
In this case, res.username is None, and None is not equal to "".
if we change the code like this:
    if res.scheme == "https" and res.username:

and re-test, then... tout rentre dans l'ordre.

Hello Mingun, or anyone else affected,

Accepted update-manager into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/update-manager/1:16.04.17 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted update-manager (1:16.04.17) for xenial have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-release-upgrader/1:16.04.27 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#update-manager

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

The package update-manager/xenial-proposed 1:16.04.17 fixes the bug for me.

The package update-manager/xenial-proposed 1:16.04.17 fixes the bug for me as well.

enabled -proposed and pulled in those files via synaptic, leaving the other packages as is. ran update-manager and changelogs show properly again on those packages.

The verification of the Stable Release Update for update-manager has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package update-manager - 1:16.04.17

---------------
update-manager (1:16.04.17) xenial; urgency=medium

  * Correctly detect the usage of a username in changelog URIs. (LP: #1849004)

 -- Brian Murray <email address hidden> Mon, 11 Nov 2019 18:54:28 -0800