Performance regression uploading images to glance in juno

Bug #1378388 reported by James Page
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Glance
New
Undecided
Unassigned
glance (Ubuntu)
Invalid
Undecided
Unassigned
python-glance-store (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

Testing: 1:2014.2~rc1-0ubuntu1

Uploads of standard ubuntu images to glance, backed by ceph, are 10x slower than on icehouse on the same infrastructure. With icehouse i saw around 200MBps, with juno around 20Mbps.

James Page (james-page)
summary: - Performance regression uploading images to glance
+ Performance regression uploading images to glance in juno
Revision history for this message
Stuart McLaren (stuart-mclaren) wrote :

Are you using SSL (https)? Judging on the numbers alone I'd be suspicious that SSL compression has been enabled -- but it's just a guess.

Revision history for this message
James Page (james-page) wrote :

The rbd connection on the backend appears to be OK:

ubuntu@juju-t-machine-6:~$ time sudo rbd --id glance --pool glance export 42f31416-261c-46c6-823d-5b38722ed800 localdisk
Exporting image: 100% complete...done.

real 0m30.209s
user 0m4.982s
sys 0m3.226s

ubuntu@juju-t-machine-6:~$ time sudo rbd --id glance --pool glance import localdisk 42f31416-261c-46c6-823d-5b38722ed800-clone
Importing image: 100% complete...done.

real 0m4.122s
user 0m0.164s
sys 0m0.354s

The image is 249MB in size.

Revision history for this message
James Page (james-page) wrote :

I think the problem is that glance is using a tiny calculated chunk size (from python-glance-store):

            chunk = self.conf.glance_store.rbd_store_chunk_size
            self.chunk_size = chunk * (1024 ^ 2)

this should be (from original glance rbd driver):

           1024 ** 2

Resulting in alot of tiny chucked writes instead of the default 8MB writes.

Changed in glance (Ubuntu):
assignee: nobody → James Page (james-page)
Revision history for this message
James Page (james-page) wrote :

I see this is fixed already in glance_store; hopefully there will be another release soon, in the meantime I'll cherry-pick the commit that fixes this issue.

Changed in glance (Ubuntu):
assignee: James Page (james-page) → nobody
status: New → Invalid
Revision history for this message
James Page (james-page) wrote :

This was fixed under https://bugs.launchpad.net/glance/+bug/1370247 in glance_store.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-glance-store - 0.1.8-1ubuntu1

---------------
python-glance-store (0.1.8-1ubuntu1) utopic; urgency=medium

  * d/p/fix-rbd-chunk-size.patch: Cherry pick fix from upstream VCS to
    correctly calculate RBD chunk size, resolving performance regression
    with rbd backend (LP: #1378388).
 -- James Page <email address hidden> Tue, 07 Oct 2014 16:08:15 +0100

Changed in python-glance-store (Ubuntu):
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.