Svn import of ruby taking 2.7G RSS

And it OOMed in the end! Yowee.

Luckily the fix is clear: incremental imports, which bzr-svn has recently been modified to support. Adapting Launchpad to take advantage should be easy, and will be done this cycle.

Fixed in stable r10495 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/10495>

seen a mem greedy run on strawberry:

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
importd 15009 4.9 71.3 2405640 1495932 pts/0 Rs+ 18:22 14:37 \_ /usr/bin/python2.5 /srv/importd.staging.launchpad.net/staging/launchpad/scripts/code-import-worker.py 14023 bzr-svn http://svn.ruby-lang.org/repos/ruby/trunk

~ 1.5Gb RSS.

revno 9109 of db-stable

The incremental import feature works, but the ruby import still takes too much memory. I'll have to talk to Jelmer about that.

Reopening this, as the incremental import feature doesn't seem to have helped.

Hi: And yet another OOM incident today, with svn/gcc:

importd@pear:/etc/nagios$ ps faux 17002
importd 17002 13.1 56.6 5075528 3462972 pts/1 Ds+ Mar25 282:04 \_ /usr/bin/python2.5 /srv/importd.launchpad.net/production/launchpad-rev-9093/scripts/code-import-worker.py 321303 bzr-svn svn://gcc.gnu.org/svn/gcc/trunk

We ended up just killing the job before it OOM'ed the box (the new importd host, pear, which has LOTS of RAM & CPU ) .

So this definitely still needs to be addressed.

I'm not sure if this will work, but it would be worth trying to get a
meliae dump along the lines of
http://jam-bazaar.blogspot.com/2009/11/memory-debugging-with-meliae.html

but that probably requires running it with a terminal attached.

Perhaps we need a signal that will tell bzr to dump its memory usage
and then exit (or continue.) Or a debug option that always does this.

--
Martin <http://launchpad.net/~mbp/>

Had another one with russkaya and the ruby import.
https://pastebin.canonical.com/30187/

was at 4Gb RSS before being killed.

We're now fully incremental, this should be a thing of ze past.