viewing the file listing consumes a lot memory

This seems to be causing problems in production, so it's definitely worth looking at.

Confirming, and marking private as someone could use this to DOS the codebrowse.lp.net machine.

Clicking the files listing for a single (admittedly pretty large) branch on codebrowse.launchpad.net eventually failed with a MemoryError:

INF [20070528-23:44:10.501] cherrypy.msg: HTTP: Traceback (most recent call last):
  File "/srv/codebrowse.launchpad.net/turbogears/lib/python2.4/site-packages/CherryPy-2.2.1-py2.4.egg/cherrypy/_cphttptools.py", line 105, in _run
    self.main()
  File "/srv/codebrowse.launchpad.net/turbogears/lib/python2.4/site-packages/CherryPy-2.2.1-py2.4.egg/cherrypy/_cphttptools.py", line 254, in main
    body = page_handler(*virtual_path, **self.params)
  File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/util.py", line 344, in _f
    out = f(*a, **kw)
  File "<string>", line 3, in default
  File "/srv/codebrowse.launchpad.net/turbogears/lib/python2.4/site-packages/TurboGears-1.0.1-py2.4.egg/turbogears/controllers.py", line 334, in expose
    output = database.run_with_transaction(
  File "<string>", line 5, in run_with_transaction
  File "/srv/codebrowse.launchpad.net/turbogears/lib/python2.4/site-packages/TurboGears-1.0.1-py2.4.egg/turbogears/database.py", line 302, in so_rwt
    retval = func(*args, **kw)
  File "<string>", line 5, in _expose
  File "/srv/codebrowse.launchpad.net/turbogears/lib/python2.4/site-packages/TurboGears-1.0.1-py2.4.egg/turbogears/controllers.py", line 351, in <lambda>
    mapping, fragment, args, kw)))
  File "/srv/codebrowse.launchpad.net/turbogears/lib/python2.4/site-packages/TurboGears-1.0.1-py2.4.egg/turbogears/controllers.py", line 378, in _execute_fun
c
    output = errorhandling.try_call(func, *args, **kw)
  File "/srv/codebrowse.launchpad.net/turbogears/lib/python2.4/site-packages/TurboGears-1.0.1-py2.4.egg/turbogears/errorhandling.py", line 73, in try_call
    return func(self, *args, **kw)
  File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/controllers/inventory_ui.py", line 92, in default
    vals = {
  File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/util.py", line 334, in locked
    return unbound(self, *args, **kw)
  File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/history.py", line 860, in get_filelist
    change_dict = dict([(c.revid, c) for c in self.get_changes(list(fetch_set))])
  File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/util.py", line 334, in locked
    return unbound(self, *args, **kw)
  File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/history.py", line 607, in get_changes
    changes = self._change_cache.get_changes(revid_list, get_diffs)
  File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/util.py", line 334, in locked
    return unbound(self, *args, **kw)
  File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/changecache.py", line 107, in get_changes
    changes = self.history.get_changes_uncached(fetch_list, get_diffs)
  File "/srv/codebrowse.launchpad.net/launchpad-loggerhead/loggerhead/util.py", line 334, in locked
    return unbound(self, *args, **kw)
  File "/srv/codebrowse.l...

Fix now in production. The file listing code was completely silly in at least three or four separate ways.