Bazaar

UnicodeDecodeError in _inaccessible_normalized_filename

Bug #393038 reported by Reuben Firmin
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Bazaar
Confirmed
Medium
Unassigned
Bazaar Git Plugin
Fix Released
Medium
Jelmer Vernooij
Breezy
Fix Released
Medium
Unassigned
Breezy 3.0.0 "Pelican"

Bug Description

bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 25: ordinal not in range(128)

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/site-packages/bzrlib/commands.py", line 729, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/commands.py", line 924, in run_bzr
    ret = run(*run_argv)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/commands.py", line 560, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/builtins.py", line 1143, in run
    source_branch=br_from)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/bzrdir.py", line 1195, in sprout
    hardlink=hardlink)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/bzrdir.py", line 1599, in create_workingtree
    accelerator_tree=accelerator_tree, hardlink=hardlink)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/workingtree_4.py", line 1418, in initialize
    basis = accelerator_tree.revision_tree(revision_id)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/workingtree.py", line 2054, in revision_tree
    if revision_id == self.last_revision():
  File "/usr/local/lib/python2.6/site-packages/bzrlib/workingtree.py", line 1751, in last_revision
    return self._last_revision()
  File "/usr/local/lib/python2.6/site-packages/bzrlib/decorators.py", line 136, in read_locked
    self.lock_read()
  File "/usr/local/lib/python2.6/site-packages/bzrlib/workingtree.py", line 1768, in lock_read
    self._reset_data()
  File "/root/.bazaar/plugins/git/workingtree.py", line 161, in _reset_data
    self.repository._git.object_store)
  File "/root/.bazaar/plugins/git/inventory.py", line 283, in __init__
    ie = self.add_path(path, kind, file_id, self.add_parents(path))
  File "/usr/local/lib/python2.6/site-packages/bzrlib/inventory.py", line 1288, in add_path
    ie = make_entry(kind, parts[-1], parent_id, file_id)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/inventory.py", line 2004, in make_entry
    name = ensure_normalized_name(name)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/inventory.py", line 2023, in ensure_normalized_name
    norm_name, can_access = osutils.normalized_filename(name)
  File "/usr/local/lib/python2.6/site-packages/bzrlib/osutils.py", line 1185, in _inaccessible_normalized_filename
    normalized = unicodedata.normalize('NFC', unicode(path))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 25: ordinal not in range(128)

Tags: unicode

Related branches

lp://staging/bzr-git
Reuben Firmin (reubenf)
summary: bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode
byte 0xc3 in position 25: ordinal not in range(128)
- Traceback (most recent call last): File
- "/usr/local/lib/python2.6/site-packages/bzrlib/commands.py", line 729,
- in exception_to_return_code return the_callable(*args, **kwargs)
- File "/usr/local/lib/python2.6/site-packages/bzrlib/commands.py", line
- 924, in run_bzr ret = run(*run_argv) File
- "/usr/local/lib/python2.6/site-packages/bzrlib/commands.py", line 560,
- in run_argv_aliases return self.run(**all_cmd_args) File
- "/usr/local/lib/python2.6/site-packages/bzrlib/builtins.py", line 1143,
- in run source_branch=br_from) File "/usr/local/lib/python2.6/site-
- packages/bzrlib/bzrdir.py", line 1195, in sprout hardlink=hardlink)
- File "/usr/local/lib/python2.6/site-packages/bzrlib/bzrdir.py", line
- 1599, in create_workingtree accelerator_tree=accelerator_tree,
- hardlink=hardlink) File "/usr/local/lib/python2.6/site-
- packages/bzrlib/workingtree_4.py", line 1418, in initialize basis =
- accelerator_tree.revision_tree(revision_id) File
- "/usr/local/lib/python2.6/site-packages/bzrlib/workingtree.py", line
- 2054, in revision_tree if revision_id == self.last_revision():
- File "/usr/local/lib/python2.6/site-packages/bzrlib/workingtree.py",
- line 1751, in last_revision return self._last_revision() File
- "/usr/local/lib/python2.6/site-packages/bzrlib/decorators.py", line 136,
- in read_locked self.lock_read() File
- "/usr/local/lib/python2.6/site-packages/bzrlib/workingtree.py", line
- 1768, in lock_read self._reset_data() File
- "/root/.bazaar/plugins/git/workingtree.py", line 161, in _reset_data
- self.repository._git.object_store) File
- "/root/.bazaar/plugins/git/inventory.py", line 283, in __init__ ie =
- self.add_path(path, kind, file_id, self.add_parents(path)) File
- "/usr/local/lib/python2.6/site-packages/bzrlib/inventory.py", line 1288,
- in add_path ie = make_entry(kind, parts[-1], parent_id, file_id)
- File "/usr/local/lib/python2.6/site-packages/bzrlib/inventory.py", line
- 2004, in make_entry name = ensure_normalized_name(name) File
- "/usr/local/lib/python2.6/site-packages/bzrlib/inventory.py", line 2023,
- in ensure_normalized_name norm_name, can_access =
- osutils.normalized_filename(name) File "/usr/local/lib/python2.6/site-
- packages/bzrlib/osutils.py", line 1185, in
- _inaccessible_normalized_filename normalized =
- unicodedata.normalize('NFC', unicode(path)) UnicodeDecodeError: 'ascii'
- codec can't decode byte 0xc3 in position 25: ordinal not in range(128)
Martin Pool (mbp)
summary: - bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode
- byte 0xc3 in position 25: ordinal not in range(128)
+ UnicodeDecodeError in _inaccessible_normalized_filename
Changed in bzr:
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
Martin Packman (gz) wrote :

Can someone who has hit this give steps to reproduce? I am presuming this is related to bzr-git and non-ascii filenames, but trying some things out got bug 587074 rather than this one. I could fix the _inaccessible_normalized_filename function, but don't want to fiddle without knowing the root cause.

Jelmer Vernooij (jelmer)
tags: added: unicode
Revision history for this message
Martin Packman (gz) wrote :

It's possible that it's just the bzr-git plugin doing something wrong by passing byte strings to the bzrlib inventory code, but even if so, the bzrlib side should be much more robust.

Revision history for this message
Martin Packman (gz) wrote :

mkdir gitfsenc
cd gitfsenc
git init
python -c "file(u'\xe9','w').close()"
git add .
git commit -m "Initial version (filesystem encoding)"

Revision history for this message
Martin Packman (gz) wrote :

Missed the final step in that repo script, which was `bzr ls` though other things may also break.

The warning from bzrlib.plugins.git.inventory at the top of the log is interesting, and suggests that at the filepath handling there is part of the issue.

Jelmer Vernooij (jelmer)
Changed in bzr-git:
status: New → Fix Released
importance: Undecided → Medium
assignee: nobody → Jelmer Vernooij (jelmer)
Revision history for this message
Fabien Tassin (fta) wrote :

fix released? really?

in oneiric, etckeeper is broken because of bzr:

/etc/cron.daily/etckeeper:
bzr: failed to report crash using apport:
     AssertionError()
bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 34: ordinal not in range(128)

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 936, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 1136, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 697, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.7/dist-packages/bzrlib/commands.py", line 719, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/builtins.py", line 707, in run
    no_recurse, action=action, save=not dry_run)
  File "/usr/lib/python2.7/dist-packages/bzrlib/mutabletree.py", line 49, in tree_write_locked
    return unbound(self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/mutabletree.py", line 628, in smart_add
    for subf in sorted(os.listdir(abspath)):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc4 in position 34: ordinal not in range(128)

bzr 2.4b2 on python 2.7.2rc1 (Linux-2.6.39-3-generic-x86_64-with-
    Ubuntu-11.10-oneiric)
arguments: ['/usr/bin/bzr', 'add', '-q', '.']
plugins: bash_completion[2.4b2], builddeb[2.7.0dev], bzrtools[2.4.0],
    changelog_merge[2.4b2], etckeeper[unknown], launchpad[2.4b2],
    netrc_credential_store[2.4b2], news_merge[2.4b2], weave_fmt[unknown]
encoding: 'ANSI_X3.4-1968', fsenc: 'ANSI_X3.4-1968', lang: None

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.
etckeeper warning: bzr add failed

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

That's a different bug, which is not related to bzr-git. You seem to have a non-utf8 filename in your /etc directory.

Revision history for this message
Martin Packman (gz) wrote :

Fabien, you want bug 715547 not this one, lots of different problems can result in a UnicodeDecodeError so where the traceback comes from matters.

Jelmer Vernooij (jelmer)
tags: added: check-for-breezy
Jelmer Vernooij (jelmer)
tags: removed: check-for-breezy
Changed in brz:
status: New → Fix Released
importance: Undecided → Medium
milestone: none → 3.0.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.