Meta Queue takes ages: 28 minutes for 1457 objects

Logs...

With libgnome being uploaded.

rtg@buzz:~/.local/share$ date # connect syncdaemon
Wed Mar 3 21:08:13 EET 2010
rtg@buzz:~/.local/share$ date # upload finished
Wed Mar 3 22:06:41 EET 2010

So it took 1h2m for upload to complete...

I think that he's seeing is a local rescan on no actual server queries. Can you confirm?

I can confirm this bug. I added Ubuntu One to Xubuntu 9.10 and it worked great. I added a couple of folders to see it work. It seemed very fast. I then added it to my second computer. Apparently, it doesn't work so fast any more. It took well over an hour to get it to sync the existing folders and files. We are only talking a couple of hundred items at this point. Thanks to the original reporter, I was able to find out it is working, even if slow.

    find ~/Ubuntu\ One | wc -l

gives 283.

Hi Roman, John,

John, you'r right this isn't server rescan.

Roman, it's taking 28 min to full sync, not to query the server.

I'll change the title to reflect the real issue.

Thanks!

This also seems to be along the lines of one of the issues which I have been seeing with Ubuntu One Syncing 1000's of items.
Just looking at the comments I can't belive that it has been marked as a "Low" Importance, when this seems to be a Major/High bug which for certain users makes it almost un-useable for quite a few people who would relie on this as there file backup system.

Im still trying to sync my last 800 items upto Ubuntu One with very slow & painfull luck.
With Dropbox this is not an issue.

And why has the taf of u1-lucid been removed & u1-makerik addes, when surley it should be a target for Lucid final as it seems to be a big useability bug.

I am using Ubuntu One to test it out and have a 50GB account, with 11GB data. Mostly photos at say 1MB each that is around 11000 files (?). I have not used U1 for a few months now because it did not seem to be getting very far for my purposes. I got very frustrated, but I have a great feeling of support for Ubuntu and its aims.

Now starting to use it again - thinking testing before ubuntu 10.04 is released would be quite useful.

However the 50GB account does not seem to me to be very useful even now, if 11GB of typical files causes my service to grind to a (almost) a halt. I am seeing download (sync) speeds of less than 100MB per 24 hours.

I urge that this bug be fixed asap please?

Hi,
Why this bug is marked with low importance? And why is it so hard to fix this? I was thinking about upgrading to 50GB but Dropbox upgrade seems much more economical in terms of time to update...
Could Someone please fix this....

more information:
overnight, say 7 hours, my machine running 10.04 beta with updates has downloaded 4GB of my stuff to the previously empty machine. This is using a seriously useful chunk of my bandwidth and is looking good at this stage.

However, I have another machine which is 9.10, and this is not syncing at all, only a MB here and there, which is hopeless for the sort of data I hope to be using (11GB current tests)

please could some more attention be given to this issue, it is are making this commercial product completely unusable in real world situations. I know as I have been trying to use it for 3 months now and it just doesn't work as expected. Considering canceling my account :(

I too am having upload woes. I signed up for the free UbuntuOne service today, marked one of my folders to synchronize with One (it contains 6 files, all less than 1Mb in size) and... almost nothing happened. I can see the files are marked as "uploading" on the One web interface, but even after an hour there is no progress. The web interface reports that 0 bytes have been uploaded.

This is currently unusable.

To follow up: it has now finished uploading 1 file.

I have the same problem syncing 24gb of pictures/vids which contains just under 12,000 files in it. Its making it impossible to upload my data and rendering the whole service useless for me. Can this please be upgraded from low priority as the software is fundamentally broken.

I have a paid up 50G account for the moment, have been using it for the last 2 months and can only say, as a business user of Ubuntu, how disappointing this sync service is. One should not have to leave one's computer on overnight just to sync 25 new files weighing in at less than 1Mb each. Nor should one have the unpleasant surprise of rebooting one's computer after shutdown to find the folder one had previously created in UbuntuOne marked as "u1conflict". I get better performance out of ADrive, and that regularly kicks me off during "massive" (several 10s of megs) FTP uploads. Looks like I too will be cancelling my subscription and checking out DropBox instead. Shame, but honestly, if a company wants to be serious about offering cloud storage, then it has to provide a system that works reliably, quickly and conveniently. With UbuntuOne, we have something that is convenient, but convenience is nothing if the system is not reliable or quick (obviously dependent on bandwidth too).

Alex

Hey Alex,

I too paid for the account and have experienced very similar problems as you in the past. But I have to say recently there has been a massive improvement in stability, speed and features. I'm currently syncing 15GB and increasing it everyday. Dropping files in the folder take about 5 mins to index and then upload at 1MB/s. Waaay better than the 30KB/s I used to get.

But I have to agree that as a commercial product this should have been running that way from the start. But I have stuck with it, and actively helped them with bug fixing, and figured since Canonical has given so much to me for free till now, whats a couple of dollars per month as a donation. That is ofcourse just my opinion, you need to do what best for you.

Dropbox is cool, but I still have some bugs with dragging and dropping files, which causes nautilus to crash. Apparently this bug was fixed, but my PC at home still suffers. And dropbox does not have the notes, bookmarks, contacts integration and with all the new features coming soon (app config integration etc) ubuntuone will end up being amazing.

So it's up to you. Ensure you running the latest ubuntu and ubuntuone and try again. It seems to be working well for me. But definitely keep coming back and trying out till it works as this is one awesome service worth watching.

Thanks to the ubuntuone team for fixing my bugs :), keep up the good work

I am closing this bug report and marking it as fix released due to the following:
1. The database layer got pgbouncer that keeps the number of active (memory consuming) connections to the minimum possible. The worst times for the clients were around Lucid Lynx release.
2. The server rescan was completely rewritten and it takes now around 2 seconds to complete.
3. All calls to Query() were removed, now get delta takes < 2 seconds and one delta brings many changes.

I'm using Ubuntu 10.10 and have just started trying to use the Ubuntu One service and although the status of this bug is 'fixed', it seems clear to me that it is not. I've been trying to sync a folder with 11,000 files, in total just under 1GB and transferring the meta data alone has taken in excess of 5 hours which really seems unacceptable. Maybe this bug should be re-opened?

Jamie

On 03/22/2011 06:53 PM, James Hanlon wrote:
> I'm using Ubuntu 10.10 and have just started trying to use the Ubuntu
> One service and although the status of this bug is 'fixed', it seems
> clear to me that it is not. I've been trying to sync a folder with
> 11,000 files, in total just under 1GB and transferring the meta data
> alone has taken in excess of 5 hours which really seems unacceptable.
> Maybe this bug should be re-opened?
>
> Jamie
>

The processing of thousands of new files at once is a problem we are aware of and is being handled in lp:733390
Recent changes in the nightlies has improved performance of this, but we are far from getting acceptable speeds for this
process.

Unfortunately, we were not able to get a fix for this yet as we would need to make considerable design changes to
client's logic. However, it is on our list of 'important' things to address as soon as we can.