update-manager inconsistent with download size

I see this in up to date 9.10 on i386. I am attaching a shot of the window showing the first inconsistency.

This bug is also found in Maverick 64 bit.

this bug is also found on maverick 32 bit..i attach a screenshot of that..

here is the attach..

Another one for update-manager 1:0.142.20 on maverick

I have chased that bug a bit further. Today few small updates wait for downloading, and I executed apt-get update.

Current version of update-manager 1:0.142.22 on maverick 2.6.35-27-generic

user@desktop:~$ python
Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from apt import Cache
>>> a=0
>>> for pkg in Cache():
... if pkg.is_upgradable:
... print "%s: %s" % (pkg.name, pkg.candidate.size)
... a=a+pkg.candidate.size
...
krb5-multidev: 103014
libgssapi-krb5-2: 120252
libgssrpc4: 75052
libk5crypto3: 96398
libkadm5clnt-mit7: 58928
libkadm5srv-mit7: 71640
libkdb5-4: 59080
libkrb5-3: 348606
libkrb5-dev: 36444
libkrb5support0: 42672
libtiff4: 136668
tzdata: 682952
tzdata-java: 145636
>>> a
1977342
>>>

update-manager shows these packages with their size:
krb5-multidev: 100kB
libgssapi-krb5-2: 117 kB
libgssrpc4: 73 kB
libk5crypto3: 94 kB
libkadm5clnt-mit7: 57 kB
libkadm5srv-mit7: 69 kB
libkdb5-4: 57 kB
libkrb5-3: 340 kB
libkrb5-dev: 35 kB
libkrb5support0: 41 kB
libtiff4: 133 kB
tzdata: 666 kB
tzdata-java: 142 kB
That fits to the sizes shown in python

BUT: The total size to download in update-manager shows 1.0 MB
This is definitely wrong when I compare that to the total size in Python of 1977342 B (=~1,9 MB)

If I copy the definition of humanize_size from Update-Manager and call
humanize_size(1977342) what do I see?
'1.0 MB'

So there is definitely a bug in the humanize_size function.
Instead of
        return locale.format(_("%.1f MB"), bytes / 1024 / 1024)
it has to be
        return locale.format(_("%.1f MB"), float(bytes) / 1024 / 1024)
(Without the float command python used integer division and cuts all remainders off!)

I am not sure if that will cure all differences in the display of file sizes, but I hope it is at least a first step into the right direction.

I found this buggy definition for humanize_size in the following files
/usr/lib/python2.6/dist-packages/UpdateManager/Core/utils.py
/usr/lib/python2.6/dist-packages/DistUpgrade/utils.py
As far as I can see all versions up to natty contain the same.

I have done some more searching of the sources and that is what I can conclude:

There is no consistency in use of conversion programs in different parts of upgrade-manager
There is use of "humanize_size" defined in updatemanager and of "apt_pkg.SizeToStr" defined in apt.

humanize_size contains a bug as described above, that for integers (and size in Bytes usually is integer) it cuts off the fraction of MBs.

humanize_size works with multiples of 1024 and switches from KB to MB above 1023 KB
apt_pkg.SizeToStr works with multiples of 1000 and switches from KB to MB above 9999 KB
The different multiplication factors lead to the difference of about 4.8% for big files that can also be calculated in some of the screenshots attached above.

Also the display of the numbers varies: in a "decimal point is comma"-speaking country like Austria some figures are displayed with a '.' as decimal separator, and some others with a ',' (see attachment in comment #6).

I definitely see the need for changing something to create consistency.

Almost all programs to download and update packages display the download size in 'decimal kilo' (e.g. synaptic, apt-get, gedbi-gtk, downloading window of update-manager). Most of these packages use the size_to_str function of apt/apt-pkg/strutl.cc. This function converts a value into a suitable smaller value with the SI multiplier character added (just 'K' or 'M' added), and does not add a 'B' for Bytes)

Only in update-manager the list of packages and the total size are expressed in 'binary kilo' size, using the function humanize_size defined in update-manager/UpdateManager/Core/utils.py. As far as I can see this program does not take into account the different decimal separator used in different languages. This Python script adds the 'B' for Bytes in the value returned.
Furthermore the humanize_size has an error that for KB values the remainder is cut off due to integer division (instead of using float division with rounding).

My proposal now is to get rid of humanize_size and use size_to_str in all parts of update-manager.
Just replacing the calls to humanize_size(value) in update-manager/UpdateManager/GTKProgress.py and update-manager/UpdateManager/GTKProgress.py by apt_pkg.size_to_str(value) is not enough because of the differences in adding 'B' for 'Bytes' respective its local language equivalent.

quick and dirty patch to convert humanize_size to decimal kilos (as size_to_str does), also caring that MB decimals are not always cut off to zero. - not sure if the decimal separator is show according to locale or always '.'

Patch is against natty version 0.146.5

similar patch against maverick version 0.142.22

At least the conversion "float(bytes)" is definitely missing in all Ubuntu sources (but not in Debian!) and should be added a.s.a.p.; the 1000 or 1024 as factor for kilos can be discussed.

The float conversion is added in http://bazaar.launchpad.net/~ubuntu-core-dev/update-manager/main/revision/2125.2.1, released in Oneiric. Can this bug be set to Fix released? Can anyone check whether it's reproducible in Oneiric?
---
Ubuntu Bug Squad volunteer triager
http://wiki.ubuntu.com/BugSquad

The missing float conversion had the effect of cutting off the decimal fraction of the size and has meanwhile been fixed.

There, however is a second part to this bug:
When doing an update with update-manager some numbers displayed on screen come from update-manager and some from apt.
And now apt is displaying the numbers with decimal kilos (K=1000), and update-manager is displaying the numbers with binary kilos (K=1024). Even if the float problem is solved, this still gives a difference in display of about 4.8% for Megabyte numbers. (see some more details in my comment #9 above and some of the attached pictures).

Before this bug is set to 'Fix Released' I would like to raise the question if it would make sense to consistently use one and the same factor for kilo/mega representation throughout all package management programs. This could be done either by changing humanize_size to decimal as well, or changing apt/apt-pkg/strutl.cc to binary, or even by totally eliminating humanize_size and using also the apt subroutines in update-manager.

As far as I can see there is even another weakness in humanize_size (at least in older versions, I have not checked the most recent ones yet): the localization of the decimal separator is not correct and always displayed as '.' even in 'decimal point is comma' local settings as with German language.

Thanks Manfred, you've got a really good point there, and you've found the root of the problem. Setting to Triaged, and updating the description of the bug according to your highly detailed comment.
---
Ubuntu Bug Squad volunteer triager
http://wiki.ubuntu.com/BugSquad

This bug was fixed in the package update-manager - 1:0.152.7

---------------
update-manager (1:0.152.7) oneiric; urgency=low

  [ Michael Vogt ]
  * merged lp:~mterry/update-manager/813778 to fix crash in
    initCache() LP: #813778. Many thanks to Michael Terry
  * add jenkins slave setup (config, upstart job), similar to the
    server-isotesting (disabled by default)
  * DistUpgrade/DistUpgrade.ui:
    - add minimal size for the details expander (and let glade
      reindent/reformat the entire file along the way)
  * merged lp:~evfool/update-manager/stringfixes

  [ Robert Roth ]
  * Updated translator comment to follow the Ubuntu Units policy
  * Display sizes according to the Ubuntu Units Policy (LP: #410310)
  * Fix ambiguous text explaining updates to running release (LP: #461780)
  * Added label to distinguish candidate version from installed version
    (LP: #537942)
  * Rename Check all/Uncheck all to Select/Deselect all
 -- Michael Vogt <email address hidden> Mon, 25 Jul 2011 18:44:45 +0200