Import fails with memory error on files larger than 25MB

Bug #141411 reported by Todd Charron
2
Affects Status Importance Assigned to Milestone
CVS to Bazaar importer
Invalid
Medium
Unassigned

Bug Description

While using bzr-cvsps-import to import a CVS repository I received the following error when trying to import a CVS v file that was over 25MB.

bzr: ERROR: exceptions.MemoryError:

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 718, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 679, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 375, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.4/site-packages/bzrlib/plugins/cvsps/__init__.py", line 86, in run
    importer.process()
  File "/home/vantage/bzr-cvsps-import/cvsps/importer.py", line 1125, in process
  File "/home/vantage/bzr-cvsps-import/cvsps/importer.py", line 1068, in _process_patchsets
  File "/home/vantage/bzr-cvsps-import/cvsps/importer.py", line 649, in handle_patchset
  File "/home/vantage/bzr-cvsps-import/cvsps/importer.py", line 872, in _extract_changes
  File "/home/vantage/bzr-cvsps-import/cvsps/importer.py", line 375, in commit
  File "/usr/lib/python2.4/site-packages/bzrlib/repository.py", line 2057, in record_entry_contents
    ie.snapshot(self._new_revision_id, path, previous_entries, tree, self)
  File "/usr/lib/python2.4/site-packages/bzrlib/inventory.py", line 438, in snapshot
    work_tree, commit_builder)
  File "/usr/lib/python2.4/site-packages/bzrlib/inventory.py", line 453, in _snapshot_into_revision
    self._snapshot_text(previous_entries, work_tree, commit_builder)
  File "/usr/lib/python2.4/site-packages/bzrlib/inventory.py", line 722, in _snapshot_text
    self.text_sha1, self.text_size = commit_builder.modified_file_text(
  File "/usr/lib/python2.4/site-packages/bzrlib/repository.py", line 2104, in modified_file_text
    self._add_text_to_weave(file_id, new_lines, file_parents.keys())
  File "/usr/lib/python2.4/site-packages/bzrlib/repository.py", line 2120, in _add_text_to_weave
    versionedfile.add_lines(self._new_revision_id, parents, new_lines)
  File "/usr/lib/python2.4/site-packages/bzrlib/versionedfile.py", line 147, in add_lines
    return self._add_lines(version_id, parents, lines, parent_texts)
  File "/usr/lib/python2.4/site-packages/bzrlib/knit.py", line 687, in _add_lines
    return self._add(version_id, lines[:], parents, self.delta, parent_texts)
  File "/usr/lib/python2.4/site-packages/bzrlib/knit.py", line 752, in _add
    delta, self.factory.annotated)
  File "/usr/lib/python2.4/site-packages/bzrlib/knit.py", line 608, in _merge_annotations
    for i, j, n in seq.get_matching_blocks():
  File "/usr/lib/python2.4/site-packages/bzrlib/patiencediff.py", line 244, in get_matching_blocks
    len(self.a), len(self.b), matches, 10)
  File "/usr/lib/python2.4/site-packages/bzrlib/patiencediff.py", line 135, in recurse_matches
    for apos, bpos in unique_lcs(a[alo:ahi], b[blo:bhi]):
  File "/usr/lib/python2.4/site-packages/bzrlib/patiencediff.py", line 53, in unique_lcs
    index[line]= i
MemoryError

Tags: memory
Revision history for this message
Numérigraphe (numerigraphe) wrote :

FYI I just used cvsps-import on CVS modules containing big binary files (>200Mo) and did not face this issue.
I used bzr 2.1.1 from the official PPA, cvsps-import checkout from the trunk on an up-to-date Ubuntu Hardy 8.04.

Jelmer Vernooij (jelmer)
tags: added: memory
Changed in bzr-cvsps-import:
status: New → In Progress
status: In Progress → Triaged
importance: Undecided → Medium
Revision history for this message
John A Meinel (jameinel) wrote :

Note that this traceback was clearly during importing into a knit format repository, with annotations, etc. I'm tempted to think that this is just an artifact of our old repository format. 25MB seems very small, even for that, but I don't think this is a bug in cvsps-import.

Changed in bzr-cvsps-import:
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.