AssertionError: get_next() called when there are no chars left in dirstate

My guess from the symptom is that the dirstate file is simply truncated. If so bzr should report an error (but not crash, obviously).

Possibly related to bug 328674, or at least the comments in that bug. In 2.0.0 bzr is more careful with the dirstate file, and tends to notice problems that much earlier versions often wouldn't.

Can you attach the .bzr/checkout/dirstate file from the working tree that causes the crash?

I'm not sure which branch is causing the problem. Looking at the output before the exception, it appears to be processing the base repository, which is shared and has no working tree. Also, those few branches under the _expired sub-path have no working tree also. I could attach all the dirstate files from all branches, but that will be about 20mb worth.

You can probably find which one it is by doing "bzr st" in each working tree until you find one that fails with a traceback like the one you got from "bzr check".

Quite right. The 2drender_performance branch is the problem. Interestingly it still has a checkout folder, it's as if something went wrong with it's remove-tree operation. There are no files in that branch, yet it appears to believe it still has a working tree. If I try a bzr remove-tree now, it gives me a similar exception. How do I recover from this, manually remove the checkout directory?

Yes. Unless you're using some form of checkout (which you aren't),
removing .bzr/checkout is safe.

(Well, if you actually had a working tree, uncommitted changes would get
confused, but you don't.)

I have a branch with uncommitted changes. Any recommendations on how to recover?

This happen a second time, I believe, under the same circumstances. I've been naughty. I'm doing wt operations through a gvfs sftp mount. Here is the .bzr.log entry where the dirstate corrupted:

Wed 2010-06-09 09:06:23 +0200
0.045 bazaar version: 2.2b2
0.045 bzr arguments: [u'diff', u'--using', u'kompare', u'-r', u'-2', u'tomcat/shared/classes/alfresco-global.properties']
0.059 looking for plugins in /home/garyvdm/.bazaar/plugins
0.111 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.126 encoding stdout as sys.stdout encoding 'UTF-8'
0.317 opening working tree '/home/garyvdm/.gvfs/sftp for root on ***.***.***.***/opt/Alfresco'
25.675 Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 907, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1111, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 685, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 700, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 172, in _do_with_cleanups
    cleanup(*c_args, **c_kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/workingtree_4.py", line 1142, in unlock
    self._dirstate.save()
  File "/usr/lib/python2.6/dist-packages/bzrlib/dirstate.py", line 2306, in save
    self._state_file.truncate()
IOError: [Errno 95] Operation not supported

25.676 Transferred: 0kB (0.0kB/s r:0kB w:0kB)
25.676 return code 3

Well, looks like we don't work with a working tree on GVFS - not
surprising really.

dirstate2 will help with this; dirstate-in-trunk really wants OS locks
(which that gvfs mount won't be doing in reality).

-Rob

Having the same problem on Ubuntu, attached error log.

Bazaar (bzr) 2.0.2
  Python interpreter: /usr/bin/python 2.5.2
  Python standard library: /usr/lib/python2.5
  Platform: Linux-2.6.24-19-server-i686-with-debian-lenny-sid
  bzrlib: /usr/lib/python2.5/site-packages/bzrlib

Just tested in Maverick too. Having the same issues. bzr st, bzr ci, nothing is working.

Bazaar (bzr) 2.2.0
  Python interpreter: /usr/bin/python 2.6.6
  Python standard library: /usr/lib/python2.6
  Platform: Linux-2.6.35-22-generic-i686-with-Ubuntu-10.10-maverick
  bzrlib: /usr/lib/python2.6/dist-packages/bzrlib

Just tested bzr trunk and I am getting the same issue too. Any idea on steps to fix this?

Bazaar (bzr) 2.3.0dev3
  Python interpreter: /usr/bin/python 2.6.6
  Python standard library: /usr/lib/python2.6
  Platform: Linux-2.6.35-22-generic-i686-with-Ubuntu-10.10-maverick
  bzrlib: /usr/local/lib/python2.6/dist-packages/bzrlib
  Bazaar configuration: /home/andrew/.bazaar
  Bazaar log file: /home/andrew/.bzr.log

Attached my log file generated from 2.3.0dev3 (trunk)

After reading the comments here and checking out my own copy it's definitely a problem with the dirstate being truncated in my checked out copy.

All our work is done locally so in my case it isn't because we're using gvfs. I've attached a copy of my broken dirstate and the last couple of logs from everyone that accessed the repository that I checked out. Not sure if any of this info helps but better then no info at all. :)

Hi Andrew,

Thanks for the data.

If the dirstate file is truncated on disk, just upgrading bzr isn't
going to fix it. Probably what you need to do is to make a fresh
checkout of that branch.

As far as fixing this, what would be useful is to know what lead up to
this file being truncated, and if you can reproduce it in a fresh
checkout.

--
Martin

I was able to fix the problem by checking out a new copy at the revision my repo was at and copying over the dirstate file.

As for reproducing the problem that's what I'm hoping to do however I am not sure how.

Here are some observations I've made though. My dirstate file has no data after a certain folder. In this folder there are files that seem to be generated by OSX (they have the "filename.ext.~1~" format) which have been set up to be ignored by bzr. Possible issue there?

Apart from that I can't see any problems with the actual commits that have happened or anything else.

Andrew Fenn wrote:
[...]
> Here are some observations I've made though. My dirstate file has no
> data after a certain folder. In this folder there are files that seem to
> be generated by OSX (they have the "filename.ext.~1~" format) which have
> been set up to be ignored by bzr. Possible issue there?

What size is the truncated file? I'm wondering if it's been cut off at some
round number that would indicate the OS failed to fully write out the contents
at some point. (Perhaps just before a system crash?)

I have the size of dirstate at 1896724

I attached my dirstate file above but here is the direct link..
https://bugs.launchpad.net/bzr/+bug/450047/+attachment/1702517/+files/bzr-bug-info.zip

When I checked out a new copy at the same revision the dirstate is 1700789 bits. I've attached that too in case it is useful.