Bazaar

Overlapping autopacks can fail with NoSuchFile on index file

Bug #507553 reported by Gareth White
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bazaar
Confirmed
Low
Unassigned

Bug Description

If two autopacks overlap at the right time, one of them will fail with a NoSuchFile error on one of the index files (e.g. .six, .tix). This affects both bzr 2.0.4dev and 2.1.0dev5 on Windows XP and OS X.

It's difficult (but not impossible) to reproduce in real-world usage. If you run the attached shell script you should see it once or twice, but there are a couple of other related bugs which you might get instead.

However, I can reproduce it every time by doing the following:
1. In_create_pack_from_packs() in bzrlib/repofmt/groupcompress_repo.py add the following just before the call to self._copy_signature_texts():
        import pdb
        pdb.set_trace()
2. Create a dummy repository:
    bzr init-repo testrepo
    bzr init testrepo/branch1
    bzr branch testrepo/branch1 testrepo/branch2
3. Commit until it breaks into the debugger (should be the 10th time):
    bzr commit -m "test" --unchanged testrepo/branch1
4. In a separate window, commit on the other branch:
    bzr commit -m "test" --unchanged testrepo/branch2
5. Both windows should now be in the debugger.
6. In the first window type "c" to continue. The commit should finish successfully.
7. In the second window type "c". The commit will fail with an error such as:
bzr: ERROR: No such file: u'/Users/garethw/Programming/Bazaar/autopack2/testrepo/.bzr/repository/indices/4656c4d1b2cd1e347cf2b94d8d210dd9.six':

If you move the breakpoint to the line before self._copy_text_texts() you get the error on a .tix file instead.

Here's the full traceback:
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/commands.py", line 853, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/commands.py", line 1055, in run_bzr
    ret = run(*run_argv)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/commands.py", line 661, in run_argv_aliases
    return self.run_direct(**all_cmd_args)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/commands.py", line 665, in run_direct
    return self._operation.run_simple(*args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/builtins.py", line 3120, in run
    exclude=safe_relpath_files(tree, exclude))
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/decorators.py", line 194, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/workingtree_4.py", line 197, in commit
    result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/decorators.py", line 194, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/mutabletree.py", line 225, in commit
    *args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/commit.py", line 257, in commit
    possible_master_transports=possible_master_transports)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/cleanup.py", line 118, in run
    self.cleanups, self.func, self, *args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/commit.py", line 402, in _commit
    self.rev_id = self.builder.commit(self.message)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repository.py", line 179, in commit
    self.repository.commit_write_group()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repository.py", line 1563, in commit_write_group
    result = self._commit_write_group()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repofmt/pack_repo.py", line 2266, in _commit_write_group
    hint = self._pack_collection._commit_write_group()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repofmt/pack_repo.py", line 2126, in _commit_write_group
    result = self.autopack()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repofmt/pack_repo.py", line 1476, in autopack
    return self._do_autopack()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repofmt/pack_repo.py", line 1516, in _do_autopack
    reload_func=self._restart_autopack)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repofmt/groupcompress_repo.py", line 695, in _execute_pack_operations
    result = packer.pack()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repofmt/pack_repo.py", line 761, in pack
    return self._create_pack_from_packs()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repofmt/groupcompress_repo.py", line 481, in _create_pack_from_packs
    self._copy_signature_texts()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/repofmt/groupcompress_repo.py", line 465, in _copy_signature_texts
    signature_keys = source_vf.keys()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/groupcompress.py", line 1808, in keys
    result.update(source.keys())
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/groupcompress.py", line 2023, in keys
    return [node[1] for node in self._graph_index.iter_all_entries()]
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/index.py", line 1302, in iter_all_entries
    self._reload_or_raise()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/index.py", line 1296, in iter_all_entries
    for node in index.iter_all_entries():
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/btree_index.py", line 973, in iter_all_entries
    if not self.key_count():
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/btree_index.py", line 1421, in key_count
    self._get_root_node()
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/btree_index.py", line 917, in _get_root_node
    self._get_internal_nodes([0])
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/btree_index.py", line 942, in _get_internal_nodes
    return self._get_nodes(self._internal_node_cache, node_indexes)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/btree_index.py", line 934, in _get_nodes
    found.update(self._get_and_cache_nodes(needed))
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/btree_index.py", line 711, in _get_and_cache_nodes
    for node_pos, node in self._read_nodes(sorted(nodes)):
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/btree_index.py", line 1526, in _read_nodes
    data_ranges = self._transport.readv(self._name, ranges)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/transport/__init__.py", line 658, in readv
    return self._readv(relpath, offsets)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/transport/__init__.py", line 670, in _readv
    fp = self.get(relpath)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/transport/local.py", line 167, in get
    self._translate_error(e, path)
  File "/Users/garethw/Programming/Bazaar/bzr-repo/bzr/bzrlib/transport/__init__.py", line 308, in _translate_error
    raise errors.NoSuchFile(path, extra=e)
NoSuchFile: No such file: u'/Users/garethw/Programming/Bazaar/autopack2/testrepo/.bzr/repository/indices/4656c4d1b2cd1e347cf2b94d8d210dd9.six': [Errno 2] No such file or directory: u'/Users/garethw/Programming/Bazaar/autopack2/testrepo/.bzr/repository/indices/4656c4d1b2cd1e347cf2b94d8d210dd9.six'

Revision history for this message
Gareth White (gwhite-deactivatedaccount) wrote :
Revision history for this message
Gareth White (gwhite-deactivatedaccount) wrote :

Note this bug is related to bug 495000 but I think the cause is slightly different.

Revision history for this message
John A Meinel (jameinel) wrote : Re: [Bug 507553] [NEW] Overlapping autopacks can fail with NoSuchFile on index file

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gareth White wrote:
> Public bug reported:
>
> If two autopacks overlap at the right time, one of them will fail with a
> NoSuchFile error on one of the index files (e.g. .six, .tix). This
> affects both bzr 2.0.4dev and 2.1.0dev5 on Windows XP and OS X.
>
> It's difficult (but not impossible) to reproduce in real-world usage. If
> you run the attached shell script you should see it once or twice, but
> there are a couple of other related bugs which you might get instead.
>
> However, I can reproduce it every time by doing the following:
> 1. In_create_pack_from_packs() in bzrlib/repofmt/groupcompress_repo.py add the following just before the call to self._copy_signature_texts():
> import pdb
> pdb.set_trace()
> 2. Create a dummy repository:
> bzr init-repo testrepo
> bzr init testrepo/branch1
> bzr branch testrepo/branch1 testrepo/branch2
> 3. Commit until it breaks into the debugger (should be the 10th time):
> bzr commit -m "test" --unchanged testrepo/branch1
> 4. In a separate window, commit on the other branch:
> bzr commit -m "test" --unchanged testrepo/branch2
> 5. Both windows should now be in the debugger.
> 6. In the first window type "c" to continue. The commit should finish successfully.
> 7. In the second window type "c". The commit will fail with an error such as:
> bzr: ERROR: No such file: u'/Users/garethw/Programming/Bazaar/autopack2/testrepo/.bzr/repository/indices/4656c4d1b2cd1e347cf2b94d8d210dd9.six':
>
> If you move the breakpoint to the line before self._copy_text_texts()
> you get the error on a .tix file instead.
>

Thanks for working out the exact steps to reproduce. I'm hoping to get
some time to look into this before 2.1.0rc1 final. (I suppose if you
submitted a patch, since I'm PatchPilot I would be forced to work on
it... :)

If it happened more often in practice, I would probably increase the
priority, but since it only really happens with an automated "how many
times can I commit concurrently" test, I'm keeping the priority a bit lower.

John
=:->

 status: confirmed
 importance: low
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAktPTlMACgkQJdeBCYSNAAON2ACgte1p6bSUzdf/oixp5f+mekrS
XuoAnA28P3XTiUAuJozXFy6D66imqVW7
=tvi1
-----END PGP SIGNATURE-----

Changed in bzr:
importance: Undecided → Low
status: New → Confirmed
Revision history for this message
Gareth White (gwhite-deactivatedaccount) wrote :

After some investigation I believe this is similar to bug 495000. I found a couple more places where reload_func isn't passed along - see the attached patch. The one in pack_repo.py doesn't seem to have any effect - presumbly this code path is for a different repository format?

I'm not sure how to write a test for this yet.

Jelmer Vernooij (jelmer)
tags: added: check-for-breezy
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.