TypeError: 'NoneType' object is unsubscriptable error in _dirstate_helpers_c.ProcessEntryC._process_entry when running status

Looking at the code, it looks like your "dirstate" file is corrupt. Specifically, it seems to have found a record, but not the containing directory. (eg, finding path/to/file, but not finding path/to).

A fix to at least give a better error would be:
=== modified file 'bzrlib/_dirstate_helpers_c.pyx'
--- bzrlib/_dirstate_helpers_c.pyx 2008-10-04 17:01:00 +0000
+++ bzrlib/_dirstate_helpers_c.pyx 2009-02-12 18:54:26 +0000
@@ -1256,8 +1256,14 @@
             path = self.pathjoin(entry[0][0], entry[0][1])
             # parent id is the entry for the path in the target tree
             # TODO: these are the same for an entire directory: cache em.
- parent_id = self.state._get_entry(self.target_index,
- path_utf8=entry[0][0])[0][2]
+ parent_entry = self.state._get_entry(self.target_index,
+ path_utf8=entry[0][0])
+ if parent_entry is None:
+ raise errors.DirstateCorrupt(self.state,
+ "We could not find the parent entry in index %d"
+ " for the entry: %s"
+ % (self.target_index, entry[0]))
+ parent_id = parent_entry[0][2]
             if parent_id == entry[0][2]:
                 parent_id = None
             if path_info is not None:

(sorry that lp generally destroys whitespace)

I think I've run into the same thing. I've been using bzr 1.3.1 on RHEL5 and decided to try out a newer version. On my unchanged branch running bzr1.11, bzr stat gives the exact stack above. The latest version from source control (and strangely, also bzr 1.9 and bzr 1.7) give the stack below. bzr 1.6 and earlier correctly do not print anything.

If I make a new clean branch/checkout of the code I don't get the error either. Some cursory examination similar to the patch above reveals that it's tripping up on the "site" directory that was removed via "bzr split" some time back. I've failed to make a simple test case unfortunately.

I've attached my entire branch from where it can be reproduced. The stack on the latest code is:

bzr: ERROR: exceptions.TypeError: 'NoneType' object is unsubscriptable

Traceback (most recent call last):
  File "/home/geoff/code/bzr.dev/bzrlib/commands.py", line 715, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/home/geoff/code/bzr.dev/bzrlib/commands.py", line 910, in run_bzr
    ret = run(*run_argv)
  File "/home/geoff/code/bzr.dev/bzrlib/commands.py", line 547, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/home/geoff/code/bzr.dev/bzrlib/commands.py", line 925, in ignore_pipe
    result = func(*args, **kwargs)
  File "/home/geoff/code/bzr.dev/bzrlib/builtins.py", line 298, in run
    show_pending=(not no_pending), verbose=verbose)
  File "/home/geoff/code/bzr.dev/bzrlib/status.py", line 118, in show_tree_status
    want_unversioned=want_unversioned)
  File "/home/geoff/code/bzr.dev/bzrlib/tree.py", line 95, in changes_from
    want_unversioned=want_unversioned,
  File "/home/geoff/code/bzr.dev/bzrlib/decorators.py", line 138, in read_locked
    result = unbound(self, *args, **kwargs)
  File "/home/geoff/code/bzr.dev/bzrlib/tree.py", line 847, in compare
    want_unversioned=want_unversioned)
  File "/home/geoff/code/bzr.dev/bzrlib/delta.py", line 230, in _compare_trees
    want_unversioned=want_unversioned):
  File "/home/geoff/code/bzr.dev/bzrlib/dirstate.py", line 3428, in iter_changes
    result = _process_entry(current_entry, current_path_info)
  File "/home/geoff/code/bzr.dev/bzrlib/dirstate.py", line 3119, in _process_entry
    path_utf8=entry[0][0])[0][2]
TypeError: 'NoneType' object is unsubscriptable

bzr 1.14dev on python 2.5.2 (linux2)
arguments: ['/home/geoff/code/bzr.dev//bzr', 'st']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
  cvsps_import /home/geoff/.bazaar/plugins/cvsps_import [unknown]
  launchpad /home/geoff/code/bzr.dev/bzrlib/plugins/launchpad [unknown]
  netrc_credential_store /home/geoff/code/bzr.dev/bzrlib/plugins/netrc_credential_store [unknown]
  scmproj /usr/lib/python2.5/site-packages/bzrlib/plugins/scmproj [0.4.5.dev.1]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.

Geoff, Jean-Francis: I'm very sure that this bug has been fixed in bzr2.0.0 - we prohibit a wide range of invalid changes that higher layers may request of the dirstate, which prevents it getting into an inconsistent state. Yay preconditions.

Anyhow, I'd deeply appeciate it if you could try whatever you were doing with 2.0.0, and:
 - if it fails in the same way, this bug stays open and we fix it
 - if it fails differently (for example with a InconsistentDelta error) we close this bug and find/open a bug for the higher level code.

Cheers,
Rob

Jean-Francis says 13:16 < jfroy> haven't seen that backtrace again I think
13:17 < jfroy> In any case, I have no idea what I was doing at the time that triggered the
               problem.

Will wait for feedback from Geoff, or some time to pass, then close.

Well, my repository is uploaded, it's just a question of downloading, unzipping and calling "bzr stat" to reproduce it (or not). If you insist I can install bzr 2.0 but I thought this would be easy for you to test.

On Wed, 2009-09-30 at 07:45 +0000, Geoff Bache wrote:
> Well, my repository is uploaded, it's just a question of downloading,
> unzipping and calling "bzr stat" to reproduce it (or not). If you insist
> I can install bzr 2.0 but I thought this would be easy for you to test.

A corrupt dirstate stays corrupt - what we need is the *action that
produces* the corruption to be reattempted, to find out if we've fixed
the bug permitting the corruption. If downloading your repository would
let me check, I'd be delighted to do that :) - so if you know the
commands that triggered the problem, I can download the repo and try
them.

-Rob

I do not know the sequence of events that led to this. I think it was a back-compatibility issue. The attached repository is not corrupt according to bzr 1.3.1 (as far as I know I'm still using an updated version of it) but is corrupt according to bzr 1.6 and later.

What I did to trigger it was download a newer Bazaar, and I was surprised to find my repository no longer worked at all. The actual "root cause" could have happened months before for all I know.

On Wed, 2009-09-30 at 13:48 +0000, Geoff Bache wrote:
> I do not know the sequence of events that led to this. I think it was a
> back-compatibility issue. The attached repository is not corrupt
> according to bzr 1.3.1 (as far as I know I'm still using an updated
> version of it) but is corrupt according to bzr 1.6 and later.
>
> What I did to trigger it was download a newer Bazaar, and I was
> surprised to find my repository no longer worked at all. The actual
> "root cause" could have happened months before for all I know.

Ok, thanks. What this looks very strongly like is a dirstate that didn't
update properly: bzr got interrupted(e.g. killed, or system power
failure) mid-write. We're addressing that via the dirstate2 project to
not do in-place updates of the dirstate metadata file.

If its ok with you, I propose to close this bug, as I don't think there
is more for either of us to action on it.

-Rob

OK. Though that sounds like a surprising explanation to me given that the repository works just fine in Bazaar 1.3.1 and has never been updated by any other version.

On Wed, 2009-09-30 at 21:11 +0000, Geoff Bache wrote:
> OK. Though that sounds like a surprising explanation to me given that
> the repository works just fine in Bazaar 1.3.1 and has never been
> updated by any other version.

I will double check to be sure.

-Rob

I've checked Geoff, and its a classic corrupt dirstate case; its working in 1.3.1 because the parser isn't doing as much, you will suffer random corruption eventually when you do an operation that depends on the dirstate being correct.

To fix the dirstate:
bzr co --lightweight . temp
mv temp/.bzr/checkout/dirstate .bzr/checkout
rm -rf temp

(Make sure you have no changes before you do this).

bzr 1.18 fixes all the root causes of such corruption that we know of; if you can upgrade to that it would be a good idea.