MemoryError on diff/commit due to corrupted dirstate

Hi,

Thanks for your bug report.

I am interested in what is causing the problem that you are
seeing. From the traceback I would guess that it is using a
lot of memory to read the dirstate (the on disk representation
of the state of the working tree).

Is this an open source project? If so can we have a copy of
the code to try and reproduce this. If not could we just have
the dirstate? It lists the filenames, but should not give any
more information than that. Failing that just the size
of the dirstate file would a start (ls -l .bzr/checkout/dirstate)

Is there any improvement if you branch the project to a different
location (on the same machine) and then make the changes?

Thanks,

James

On Mon, 2008-01-28 at 22:28 +0000, James Westby wrote:
> Hi,
>
> Thanks for your bug report.
>
> I am interested in what is causing the problem that you are
> seeing. From the traceback I would guess that it is using a
> lot of memory to read the dirstate (the on disk representation
> of the state of the working tree).

Actually, I suspect its a corrupt dirstate; using the non C version of
the dirstate parser will likely work better and give a better message.

-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

On Tue, 2008-01-29 at 11:35 +0000, Robert Collins wrote:
> On Mon, 2008-01-28 at 22:28 +0000, James Westby wrote:
> > Hi,
> >
> > Thanks for your bug report.
> >
> > I am interested in what is causing the problem that you are
> > seeing. From the traceback I would guess that it is using a
> > lot of memory to read the dirstate (the on disk representation
> > of the state of the working tree).
>
> Actually, I suspect its a corrupt dirstate; using the non C version of
> the dirstate parser will likely work better and give a better message.

Michael, in order to test this, could you please either delete or rename
/usr/lib/python2.5/site-packages/bzrlib/_dirstate_helpers_c*

This will mean that the python versions are used instead, hopefully
giving us some more information.

Everyone else, Is it worth adding an environment variable to override
the C versions and use the python versions for testing this sort of
thing in future?

Thanks,

James

I just commented on #153749 about this but I get the same error. It started for me when my computer locked up in the middle of a "bzr status" command. When I rebooted I started getting this error for most bzr commands.

Attached is my .bzr/checkout/dirstate

Here is the message I get using the non C version of the dirstate helper:

------------------
bzr: ERROR: exceptions.AssertionError:

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 806, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 762, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 492, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.5/site-packages/bzrlib/commands.py", line 772, in ignore_pipe
    result = func(*args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/builtins.py", line 189, in run
    to_file=self.outf, short=short, versioned=versioned)
  File "/usr/lib/python2.5/site-packages/bzrlib/status.py", line 114, in show_tree_status
    want_unversioned=want_unversioned)
  File "/usr/lib/python2.5/site-packages/bzrlib/tree.py", line 93, in changes_from
    want_unversioned=want_unversioned,
  File "/usr/lib/python2.5/site-packages/bzrlib/decorators.py", line 127, in read_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.5/site-packages/bzrlib/tree.py", line 707, in compare
    want_unversioned=want_unversioned)
  File "/usr/lib/python2.5/site-packages/bzrlib/delta.py", line 217, in _compare_trees
    want_unversioned=want_unversioned):
  File "/usr/lib/python2.5/site-packages/bzrlib/workingtree_4.py", line 1793, in _iter_changes
    state._read_dirblocks_if_needed()
  File "/usr/lib/python2.5/site-packages/bzrlib/dirstate.py", line 1877, in _read_dirblocks_if_needed
    _read_dirblocks(self)
  File "/usr/lib/python2.5/site-packages/bzrlib/_dirstate_helpers_py.py", line 203, in _read_dirblocks_py
    assert trailing == ''
AssertionError

bzr 1.1.0 on python 2.5.1.final.0 (linux2)
arguments: ['/usr/bin/bzr', 'status']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  bzrtools /usr/lib/python2.5/site-packages/bzrlib/plugins/bzrtools [1.1.0]
  gtk /usr/lib/python2.5/site-packages/bzrlib/plugins/gtk [0.93.0]
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [unknown]
  multiparent /usr/lib/python2.5/site-packages/bzrlib/plugins/multiparent.pyc [unknown]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.
-----------------------------------------

When trying to branch this same directory I get the following error which is the same as my previous comment except for the last line about the lock:

bzr: ERROR: exceptions.AssertionError:

(...same as previous comment...)

  File "/usr/lib/python2.5/site-packages/bzrlib/_dirstate_helpers_py.py", line 203, in _read_dirblocks_py
    assert trailing == ''
AssertionError

bzr 1.1.0 on python 2.5.1.final.0 (linux2)
arguments: ['/usr/bin/bzr', 'branch', 'trunk', 'trunk2']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  bzrtools /usr/lib/python2.5/site-packages/bzrlib/plugins/bzrtools [1.1.0]
  gtk /usr/lib/python2.5/site-packages/bzrlib/plugins/gtk [0.93.0]
  launchpad /usr/lib/python2.5/site-packages/bzrlib/plugins/launchpad [unknown]
  multiparent /usr/lib/python2.5/site-packages/bzrlib/plugins/multiparent.pyc [unknown]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.
/usr/lib/python2.5/site-packages/bzrlib/lock.py:79: UserWarning: lock on <open file u'/home/brett/devel/bauble/trunk/.bzr/checkout/dirstate', mode 'rb' at 0x8786188> not released
  warn("lock on %r not released" % self.f)

Is there a quick fix for this? At the moment I can't commit to my tree and so my work is stalled.

On Wed, 2008-02-06 at 17:03 +0000, brettatoms wrote:
> Is there a quick fix for this? At the moment I can't commit to my tree
> and so my work is stalled.
>

My best suggestion is to branch to somewhere else on your system and
then copy over the files that have changed.

I am going to look in the problem tonight, so I may be able to give
you a fix for your dirstate file.

Thanks,

James

It sounds like your dirstate file itself is corrupted, so you need to generate a new working tree and start there. There are a couple of things you could try, but step one is definitely

Backup the current work (a simple cp -a, or tarball would be sufficient)

The way I would do it is to create a new branch with:

cd ..
bzr branch project new_project

then copy over your changes

rsync --exclude .bzr project/ new_project

You may need to redo some stuff like renames, add, etc. But it should let you get back to work.

On Wed, 2008-02-06 at 17:03 +0000, brettatoms wrote:
> Is there a quick fix for this? At the moment I can't commit to my tree
> and so my work is stalled.
>

Sorry, just seen your other message saying that you can't branch
from this branch, so my suggestion will not work.

If you have an ssh server on that machine then branching over ssh
may work. Alternatively bzr serve and using bzr:// should accomplish the
same.

Thanks,

James

Hi,

The last line of your dirstate reads

^@web/images/screenshots/0.6^@bauble-sqlite-conn-dialog-0.6.0.png^@webimagesscreenshots-20070625115140-f77u2cc78wp550ne-730^@f^@df91fbb396872aea3bfc1ad609e73f277260d755^@28051^@y^@AABtk0egWKxHoFitAAADBgYIr7IAAIHt^@f^@df91fbb396872aea3bfc1ad609e73f277260d755^@28051^@y^@brettatoms@g

which is truncated.

Is there any chance that you had disk space issues, or that you suffered a power failure
or disk errors around the time that this problem occured?

Thanks,

James

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John A Meinel wrote:
> It sounds like your dirstate file itself is corrupted, so you need to
> generate a new working tree and start there. There are a couple of
> things you could try, but step one is definitely
>
> Backup the current work (a simple cp -a, or tarball would be sufficient)
>
> The way I would do it is to create a new branch with:
>
> cd ..
> bzr branch project new_project

Unfortunately, branch now opens working trees for the file copy
acceleration. So branch will break, too. I think push would work.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHqffy0F+nu1YWqI0RAgWwAJ998tVvy7mF3J2g2dr8eVo48aGmNgCfZN4k
Wq7tqE6D6SXckArz5TukJSE=
=hsD8
-----END PGP SIGNATURE-----

On Wed, Feb 06, 2008 at 06:09:55PM -0000 I heard the voice of
Aaron Bentley, and lo! it spake thus:
>
> Unfortunately, branch now opens working trees for the file copy
> acceleration. So branch will break, too. I think push would work.

You could probably also rm (or mv) .bzr/checkout/ and then branch (and
maybe copy back the .bzr/checkout/ from the new branch).

It would be nice to have a "bzr blat-out-pristine-dirstate" command
to streamline recovery from this sort of thing...

--
Matthew Fuller (MF4839) | <email address hidden>
Systems/Network Administrator | http://www.over-yonder.net/~fullermd/
           On the Internet, nobody can hear you scream.

here from bug #195231

I have a patch pending for at least giving better error reporting rather than getting MemoryError or a segfault.

Hmmm! Disk space issues! I hadn't thought of that!