tripleo

tripleo devtest overcloud image creation takes too long

Bug #1295731 reported by stephen mulcahy
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Expired
High
Unassigned

Bug Description

Discussions with Robert Collins indicate that I should expect overcloud image creation (with the default devtest configuration and settings) on Ubuntu 13.10 on reasonable hardware to complete in about 15 minutes. I am running repeated builds on a dedicated system with the following configuration:

Processor: 2 x Intel(R) Xeon(R) CPU X5650 @ 2.67GHz
Memory: 96G
Disk: 2TB
OS: Ubuntu 13.10

and seeing consistent builds times between 19 and 22 minutes.

devtest is configured to use a local pypi mirror, a local apt mirror and a local squid proxy.

See original description
stephen mulcahy (stephen-mulcahy)
description: updated
Revision history for this message
Jon-Paul Sullivan (jonpaul-sullivan) wrote :

Looking at http://goodsquishy.com/downloads/tripleo-jobs.html - specifically the check-tripleo-overcloud-precise timings - and they appear to match the timings seen herein.

Specifically taking http://logs.openstack.org/52/76952/9/check-tripleo/check-tripleo-overcloud-precise/ed953b6/console.html gives the following:

2014-03-20 18:42:27.471 | overcloud-control.qcow2 build started
2014-03-20 19:00:00.580 | overcloud-control.qcow2 created

2014-03-20 19:00:28.678 | overcloud-compute.qcow2 build started
2014-03-20 19:10:54.045 | overcloud-compute.qcow2 created

So 18 and 10 minutes for the control and compute images, respectively. This is approximately 28 minutes total, and so significantly slower (6 minutes, ~28%) than the figures described above.

Revision history for this message
Robert Collins (lifeless) wrote :

As a data point, please try adding --offline to your image building command lines (e.g. put it in ...COMMON_ELEMENTS...). This affects cache invalidation / liveness checks (specifically turns off checking for up to date images, and turns off having pyp.python.org as a fallback index).

Robert Collins (lifeless)
Changed in tripleo:
status: New → Triaged
importance: Undecided → High
Revision history for this message
stephen mulcahy (stephen-mulcahy) wrote :

This is a devtest --trash-my-machine -c run with --offline set

[2014-03-24 15:06:46] Total runtime: 2410 s
[2014-03-24 15:06:46] ramdisk : 0 s
[2014-03-24 15:06:46] seed : 255 s
[2014-03-24 15:06:46] undercloud : 617 s
[2014-03-24 15:06:46] overcloud : 1530 s
[2014-03-24 15:06:46] DIB_COMMON_ELEMENTS=stackuser pypi --offline use-ephemeral

Revision history for this message
Ghe Rivero (ghe.rivero) wrote :

As a quick test, just building both overcloud images with wheels, recudes the build time from 1045.5sec to 732.1

Revision history for this message
Derek Higgins (derekh) wrote :

There are various timing's being compared here,
   Roberts expectations
   stephens obsevations
   timings from a single CI run (built in a VM)
   and a timing with wheels

I can't see anyway to compare all of these, so marking as incomplete, Obviously I'd love to speed things up but
If we can't identify exactly what part of the image build process needs to be improved I'd like to close this

Changed in tripleo:
status: Triaged → Incomplete
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for tripleo because there has been no activity for 60 days.]

Changed in tripleo:
status: Incomplete → Expired
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.