Dist Upgrade from 13.04 to 13.10 stuck during flashplugin-installer

When I was upgrading from Ubuntu 13.04 (raring) to Ubuntu 13.10 (saucy), the installation was stuck for several hours here:

    flashplugin-installer: downloading http://archive.canonical.com/pool/partner/a/adobe-flashplugin/adobe-flashplugin_11.2.202.310.orig.tar.gz

Eventually, due to a system reboot (caused by baby pounding on the keyboard), I had to recover from the half-finished upgrade process manually.

It might be an isolated incident, but I am not alone, as seen by this user's experience here:

    "13.04 to 13.10 upgrade stalled at flashplugin-installer", posted 2013-10-19:
    http://askubuntu.com/questions/361591/13-04-to-13-10-upgrade-stalled-at-flashplugin-installer

After I got my system back in order (e.g. sudo dpkg --configure -a; purging obsolete/unneeded packages; removing ~/.config/gtk-3.0 to get the correct set of system tray icons; adding input methods, etc.), I tried "sudo apt-get --reinstall install flashplugin-installer" again:

    Processing triggers for update-notifier-common ...
    flashplugin-installer: downloading http://archive.canonical.com/pool/partner/a/adobe-flashplugin/adobe-flashplugin_11.2.202.310.orig.tar.gz

And it got stuck again, with or without proxy.

I originally suspected that it may have something to do with GFW of China, but then a normal "wget http://archive.canonical.com/pool/partner/a/adobe-flashplugin/adobe-flashplugin_11.2.202.310.orig.tar.gz", albeit somewhat slow, fetched the file just fine, both with direct connection and with proxy.

Digging deeper, it dawned on me that Ubuntu's flashplugin-installer differs from Debian's flashplugin-nonfree, mainly, the Ubuntu-specific "update-notifier" and its package-data-downloader Python script is used instead of wget for downloading external package data.

So, I finally found "/usr/share/package-data-downloads/flashplugin-installer",
and in "/usr/lib/update-notifier/package-data-downloader", these relevant lines:

                        print "%s: downloading %s" % (relfile, files[i])
                        dest_file = urllib.urlretrieve(files[i])[0]
                        output = subprocess.check_output(["sha256sum", dest_file])

Failed to discover the location of the tempfile that urllib.urlretrieve() writes to, I fell asleep.

Today, I decided to file this bug report before my interests die down and my memory fails me. :-)

As I was typing this, I tried to install flashplugin-installer again, and I discovered that urllib.urlretrieve() writes to /tmp/tmpXXXXXX.gz (e.g. /tmp/tmpXbszB4.gz) by default, and was able to see the file growing in size.

True to Murphy's Law, despite failing to install flashplugin-installer for two days, now I see it installs successfully, as I am finishing this bug report! :-p

Nevertheless, I think this bug is worth looking into and taking care of. There are at least two such confirmed incidents, and it is such a downer when the whole distribution upgrade process got stuck by something as "innocuous" as the Adobe Flash Plugin. :-p

A few points to look at:

 1. Are Python's urllib.urlretrieve() and friends able to provide a download progress bar of some sort so that the end-user isn't stuck wondering whether the download stalled, or just that it is slow?

 2. From my cable TV 6 Mbps Internet connection in Beijing, depending on the time of the day, the download speed http://archive.canonical.com/pool/partner/a/adobe-flashplugin/adobe-flashplugin_11.2.202.310.orig.tar.gz may get as low as 20KB/s or even 10KB/s, which translates to 10 to 20 minutes of download for this 13MB tarball. Even without connection timeout or whatnot, the end-user cannot help but wonder what is happening. Debian's older flashplugin-nonfree package with wget's progress bar has the definite advantage here.

 3. Does urllib.urlretrieve() and the rest of /usr/lib/update-notifier/package-data-downloader handle all kinds of exceptions like network error, DNS timeout, download interruption, etc. gracefully? Or are there some corner cases where "stalling" would indeed happen?

Many thanks for looking into this!

Cheers,
Anthony Fok

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: flashplugin-installer 11.2.202.310ubuntu1
ProcVersionSignature: Ubuntu 3.11.0-12.19-generic 3.11.3
Uname: Linux 3.11.0-12-generic x86_64
ApportVersion: 2.12.5-0ubuntu2
Architecture: amd64
Date: Tue Oct 22 15:17:50 2013
InstallationDate: Installed on 2012-12-11 (314 days ago)
InstallationMedia: Ubuntu 12.10 "Quantal Quetzal" - Release amd64 (20121017.5)
MarkForUpload: True
SourcePackage: flashplugin-nonfree
UpgradeStatus: Upgraded to saucy on 2013-10-21 (1 days ago)