When unconstrained, udm sometimes downloads files to wrong location

Bug #1341685 reported by Barry Warsaw
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical System Image
Fix Released
High
Michael Frey
ubuntu-download-manager (Ubuntu)
Fix Released
Critical
Loïc Minier

Bug Description

I'm working on making the s-i tests more robust and I've run into an odd situation, where udm sometimes stores its downloaded files using an incorrect path, and not the one that s-i (which is unconstrained) requests. This only appears to happen when the full s-i test suite is run. When run in isolation, the test succeeds.

Here's a log of the createDownloadGroup call I'm making:

createDownloadGroup: [Record(url='https://localhost:8943/gpg/blacklist.tar.xz', destination='/tmp/tmpe7mo8qm9/ubuntu/cache/keyring.tar.xz', checksum=''), Record(url='https://localhost:8943/gpg/blacklist.tar.xz.asc', destination='/tmp/tmpe7mo8qm9/ubuntu/cache/keyring.tar.xz.asc', checksum='')]

url is the source url, destination is the local path to save the file in. This call produces incorrect destination paths in the 'finished' signal:

FINISHED: dbus.Array([dbus.String('/home/barry/.local/share/ubuntu-download-manager//usr/lib/telepathy/mission-control-5/Downloads/blacklist (20).tar.xz'), dbus.String('/home/barry/.local/share/ubuntu-download-manager//usr/lib/telepathy/mission-control-5/Downloads/blacklist (20).tar.xz.asc')], signature=dbus.Signature('s'))

Notice the very strange directory udm chooses which is definitely not the path requested, and in fact doesn't exist. This matches the observed behavior that after a succesful group download completion (i.e. no errors occur), I assert that the requested destination path exists, and in the failing test it does not.

I suspect some state is not getting reset in udm.

Tags: ota-1 rtm14

Related branches

Revision history for this message
Barry Warsaw (barry) wrote :
Download full text (8.2 KiB)

Now, I'm absolutely positive I never give a local file path with 'mission-control' in it, but look at this:

@limelight[~:630]% ls -lR ~/.local/share/ubuntu-download-manager/usr/
/home/barry/.local/share/ubuntu-download-manager/usr/:
total 8
drwxr-xr-x 3 barry barry 4096 Jul 11 18:09 bin/
drwxr-xr-x 3 barry barry 4096 Jun 9 13:00 lib/

/home/barry/.local/share/ubuntu-download-manager/usr/bin:
total 4
drwxr-xr-x 3 barry barry 4096 Jul 11 18:09 mediascanner-service-2.0/

/home/barry/.local/share/ubuntu-download-manager/usr/bin/mediascanner-service-2.0:
total 4
drwxr-xr-x 2 barry barry 4096 Jul 11 18:09 Downloads/

/home/barry/.local/share/ubuntu-download-manager/usr/bin/mediascanner-service-2.0/Downloads:
total 24
-rw-r--r-- 1 barry barry 1388 Jul 11 18:09 blacklist.tar.xz
-rw-r--r-- 1 barry barry 819 Jul 11 18:09 blacklist.tar.xz.asc

/home/barry/.local/share/ubuntu-download-manager/usr/lib:
total 4
drwxr-xr-x 4 barry barry 4096 Jul 11 09:22 telepathy/

/home/barry/.local/share/ubuntu-download-manager/usr/lib/telepathy:
total 8
drwxr-xr-x 3 barry barry 4096 Jun 9 13:00 mission-control-5/
drwxr-xr-x 3 barry barry 4096 Jul 11 09:22 telepathy-ofono/

/home/barry/.local/share/ubuntu-download-manager/usr/lib/telepathy/mission-control-5:
total 12
drwxr-xr-x 2 barry barry 12288 Jul 14 14:38 Downloads/

/home/barry/.local/share/ubuntu-download-manager/usr/lib/telepathy/mission-control-5/Downloads:
total 816
-rw-r--r-- 1 barry barry 1388 Jul 11 14:39 blacklist (10).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 11 14:39 blacklist (10).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 11 15:20 blacklist (11).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 11 15:20 blacklist (11).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 11 18:29 blacklist (12).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 11 18:29 blacklist (12).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 14 10:05 blacklist (13).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 14 10:05 blacklist (13).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 14 10:26 blacklist (14).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 14 10:26 blacklist (14).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 14 10:42 blacklist (15).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 14 10:42 blacklist (15).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 14 10:42 blacklist (16).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 14 10:42 blacklist (16).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 14 10:58 blacklist (17).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 14 10:58 blacklist (17).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 14 11:28 blacklist (18).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 14 11:28 blacklist (18).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 14 11:48 blacklist (19).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 14 11:48 blacklist (19).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jun 9 14:44 blacklist (1).tar.xz
-rw-r--r-- 1 barry barry 819 Jun 9 14:44 blacklist (1).tar.xz.asc
-rw-r--r-- 1 barry barry 1384 Jul 14 12:08 blacklist (20).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 14 12:08 blacklist (20).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jul 14 14:38 blacklist (21).tar.xz
-rw-r--r-- 1 barry barry 819 Jul 14 14:38 blacklist (21).tar.xz.asc
-rw-r--r-- 1 barry barry 1388 Jun 1...

Read more...

Revision history for this message
Barry Warsaw (barry) wrote :

I wonder if it's some messed up calculation as to whether we're unconstrained or not?

Revision history for this message
Barry Warsaw (barry) wrote :

Running the full test suite on two different machines (laptop and desktop), I see different test failures with the same symptoms. The local paths are messed up in the described way, but there's no overlap between the two machine's test failures. At least on the desktop it's always the same test that fails.

Revision history for this message
Barry Warsaw (barry) wrote :

One more data point for the day. Indeed, while the set of tests that are failing with this symptom differs between my desktop and laptop, they are completely reproducible on each machine. The same one test fails on desktop every time, while it's the same three tests that always fail on laptop.

Revision history for this message
Barry Warsaw (barry) wrote :

It always seems to be /usr/lib/telepathy/mission-control-5 even on different systems. Why?!

Revision history for this message
Barry Warsaw (barry) wrote :

I deleted ~/.local/share/ubuntu-download-manager and re-ran the tests, and so far haven't gotten any failures. However, now I see that there are more directories under ~/.local/share/ubuntu-download-manager such as usr/lib/telepathy/{mission-control-5,telepathy-ofono}/

Maybe there's a leakage in paths from one download (or group download) to another. E.g. after telepathy runs it pollutes the download paths for other clients? Just more conjecture.

Revision history for this message
Barry Warsaw (barry) wrote :

Hi Manuel - have you had time to look into this? It's still causing intermittent test failures in system-image.

Revision history for this message
Barry Warsaw (barry) wrote :

My assertion is getting triggered again.

Changed in ubuntu-download-manager:
importance: Undecided → Critical
Revision history for this message
Barry Warsaw (barry) wrote :

I've beefed up the output when the assertion fails. here's what I see:

AssertionError: dbus.Boolean(False) is not true : Missing destination files: ['/tmp/tmp89brss02/ubuntu/cache/keyring.tar.xz', '/tmp/tmp89brss02/ubuntu/cache/keyring.tar.xz.asc']
local_paths: [dbus.String('/home/barry/.local/share/ubuntu-download-manager//usr/lib/telepathy/telepathy-ofono/Downloads/blacklist (20).tar.xz'), dbus.String('/home/barry/.local/share/ubuntu-download-manager//usr/lib/telepathy/telepathy-ofono/Downloads/blacklist (20).tar.xz.asc')]

Is there any explanation for why those bogus local_paths are being returned in the `finished` signal? Note that the missing destination file paths are what's being given to udm.

Revision history for this message
Barry Warsaw (barry) wrote :

Even the fact that /usr/lib/telepathy/telepathy-ofono even appears is crazy! That's a binary, not a directory so can't possibly have Downloads/ in it.

Revision history for this message
Loïc Minier (lool) wrote :

Not sure why, can't attach my branch to the bug; it's at lp:~lool/ubuntu-download-manager/dbus-proxy-requires-dbus

Revision history for this message
Loïc Minier (lool) wrote :

Testsuite now passes on above branch; would like to get a build of it.

Changed in ubuntu-download-manager:
assignee: nobody → Loïc Minier (lool)
status: New → In Progress
Changed in ubuntu-download-manager (Ubuntu):
importance: Undecided → Critical
status: New → In Progress
Changed in ubuntu-download-manager (Ubuntu RTM):
importance: Undecided → Critical
status: New → In Progress
Revision history for this message
Olli Ries (ories) wrote :

please fix after RTM via OTA

tags: added: ota-1 rtm14
Revision history for this message
Barry Warsaw (barry) wrote : Re: [Bug 1341685] Re: When unconstrained, udm sometimes downloads files to wrong location

On Oct 28, 2014, at 03:49 PM, Olli Ries wrote:

>please fix after RTM via OTA

Shouldn't this be fixed for RTM?

Revision history for this message
Manuel de la Peña (mandel) wrote :

Any updates on when we can land this? Can I propose it for vivid?

Revision history for this message
Barry Warsaw (barry) wrote :

On Nov 17, 2014, at 10:36 PM, Manuel de la Peña wrote:

>Any updates on when we can land this? Can I propose it for vivid?

Are there any blocks on vivid landings?

Revision history for this message
Victor Tuson Palau (vtuson) wrote :

there are no blocks on vivid landings

Olli Ries (ories)
Changed in canonical-devices-system-image:
assignee: nobody → Canonical Devices Products (canonical-devices-products-team)
importance: Undecided → High
milestone: none → r1
status: New → Confirmed
Changed in ubuntu-download-manager (Ubuntu):
status: In Progress → Fix Released
assignee: nobody → Loïc Minier (lool)
no longer affects: ubuntu-download-manager
Revision history for this message
Pat McGowan (pat-mcgowan) wrote :

this could have landed any time this year

Changed in ubuntu-download-manager (Ubuntu RTM):
assignee: nobody → Manuel de la Peña (mandel)
Changed in canonical-devices-system-image:
milestone: ww51-2014 → ww05-2015
Changed in canonical-devices-system-image:
status: Confirmed → In Progress
milestone: ww05-2015 → ww07-2015
Changed in canonical-devices-system-image:
assignee: Canonical Devices Products (canonical-devices-products-team) → Michael Frey (mfrey)
milestone: ww07-2015 → ww09-2015
Changed in canonical-devices-system-image:
milestone: ww09-2015 → ww11-2015
status: In Progress → Fix Released
no longer affects: ubuntu-download-manager (Ubuntu RTM)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.