Bazaar

Unicode exception occurs by "version-info"

Bug #518609 reported by Yasuyuki Furukawa
32
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Bazaar
Fix Released
Medium
Andrej A Antonov
Bazaar 2.3b2

Bug Description

When log comment includes the non ascii characters,
"bzr version-info" command crashs by UnicodeDecodeError exception.

$ bzr branch lp:bzr bzr.dev
$ cd bzr.dev
$ bzr version-info --include-history
bzr: ERROR: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 83219: ordinal not in range(128)

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/bzrlib/commands.py", line 853, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/bzrlib/commands.py", line 1055, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/site-packages/bzrlib/commands.py", line 661, in run_argv_aliases
    return self.run_direct(**all_cmd_args)
  File "/usr/lib/python2.6/site-packages/bzrlib/commands.py", line 665, in run_direct
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/bzrlib/cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/bzrlib/cmd_version_info.py", line 119, in run
    builder.generate(ui.ui_factory.make_output_stream())
  File "/usr/lib/python2.6/site-packages/bzrlib/version_info_formats/format_rio.py", line 80, in generate
    writer.write_stanza(info)
  File "/usr/lib/python2.6/site-packages/bzrlib/rio.py", line 49, in write_stanza
    stanza.write(self._to_file)
  File "/usr/lib/python2.6/site-packages/bzrlib/rio.py", line 215, in write
    to_file.writelines(self.to_lines())
  File "/usr/lib/python2.6/site-packages/bzrlib/ui/text.py", line 492, in writelines
    self.wrapped_stream.writelines(lines)
  File "/usr/lib/python2.6/codecs.py", line 359, in writelines
    self.write(''.join(list))
  File "/usr/lib/python2.6/codecs.py", line 351, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 83219: ordinal not in range(128)

bzr 2.1.0rc2 on python 2.6.4 (Linux-2.6.33-0.20.rc5.git0.fc13.i686-i686-with-fedora-13-Rawhide)
arguments: ['/usr/bin/bzr', 'version-info', '--include-history']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'ja_JP.utf8'
plugins:
  launchpad /usr/lib/python2.6/site-packages/bzrlib/plugins/launchpad [2.1.0rc2]
  netrc_credential_store /usr/lib/python2.6/site-packages/bzrlib/plugins/netrc_credential_store [2.1.0rc2]
  news_merge /usr/lib/python2.6/site-packages/bzrlib/plugins/news_merge [2.1.0rc2]

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.

Related branches

lp://staging/~parthm/bzr/518609-version-info-unicodedecodeerror
Rejected for merging into lp://staging/bzr
Robert Collins (community): Needs Fixing
Diff: 68 lines (+11/-4)
3 files modified
NEWS (+4/-0)
bzrlib/rio.py (+1/-1)
bzrlib/tests/test_version_info.py (+6/-3)
lp://staging/~polymorphm/bzr/BUG_FIX_518609__v1.c
Martin Packman (community): Approve
Diff: 48 lines (+21/-1)
3 files modified
NEWS (+4/-0)
bzrlib/cmd_version_info.py (+1/-1)
bzrlib/tests/blackbox/test_version_info.py (+16/-0)
Revision history for this message
Martin Pool (mbp) wrote :

version-info needs to declare that it's producing unicode output, etc.

Changed in bzr:
importance: Undecided → Medium
status: New → Confirmed
tags: added: easy unicode
Parth Malwankar (parthm)
Changed in bzr:
status: Confirmed → In Progress
assignee: nobody → Parth Malwankar (parthm)
Revision history for this message
Robert Collins (lifeless) wrote :

This isn't fixed by declaring version-info as outputting unicode: rio in non-utf8 encodings isn't parsable [today] unless the encoding was known. Better to define the file to write to (and perhaps accept - for folk that 'know what they are doing' and want it ond stdout).

Revision history for this message
Parth Malwankar (parthm) wrote :

Mailing list thread has further discussion on this.
http://thread.gmane.org/gmane.comp.version-control.bazaar-ng.general/67399

Parth Malwankar (parthm)
Changed in bzr:
assignee: Parth Malwankar (parthm) → nobody
status: In Progress → Confirmed
Vincent Ladeuil (vila)
tags: added: affect-emacs
Revision history for this message
Andrej A Antonov (polymorphm) wrote :

this bug is a bit like https://bugs.launchpad.net/bzr/+bug/518916 :-)

Revision history for this message
Andrej A Antonov (polymorphm) wrote :

^^^--- this (Message above) patch has not been thoroughly tested. it just yet -- as a probability version

Revision history for this message
Andrej A Antonov (polymorphm) wrote :

oops, I did not notice that the patch is already there [ from Parth Malwankar @ 2010-04-05 ] . i am sorry :-)

Revision history for this message
Andrej A Antonov (polymorphm) wrote :

as the "BUG_FIX_518609.patch", but fixed regression:

cases for unknown-encoding for output stream (example output is a file but not terminal)

Revision history for this message
Andrej A Antonov (polymorphm) wrote :
Revision history for this message
Andrej A Antonov (polymorphm) wrote :
Download full text (4.1 KiB)

it is more logical way

patch -- checked in Ubuntu (en_GB.UTF-8 with cyrillic symbals in Branch-History) and in Microsoft Vista (cp1251/CP866 with cyrillic symbals in Branch-History)

Windows terminal output is:

C:\Users\regular-user\Desktop\bzr.bzr\test-branch>python ..\current\bzr version-info --all
revision-id: <email address hidden>
date: 2010-09-21 04:23:05 +0400
build-date: 2010-09-21 18:10:39 +0400
revno: 3
branch-nick: test-branch
clean: True
revisions: id: <email address hidden>
        message: 'первый'
        date: 2010-09-20 14:22:05 +0400
        id: <email address hidden>
        message: 'second'
        date: 2010-09-20 14:33:05 +0400
        id: <email address hidden>
        message: УРА!

        date: 2010-09-21 04:23:05 +0400

file-revisions: path:
        revision: <email address hidden>
        path: файл.txt
        revision: <email address hidden>

bzr: warning: some compiled extensions could not be loaded; see <https://answers.launchpad.net/bzr/+faq/703>

C:\Users\regular-user\Desktop\bzr.bzr\test-branch>python ..\current\bzr version-info --all >file.txt
bzr: warning: some compiled extensions could not be loaded; see <https://answers.launchpad.net/bzr/+faq/703>

C:\Users\regular-user\Desktop\bzr.bzr\test-branch>type file.txt
revision-id: <email address hidden>
date: 2010-09-21 04:23:05 +0400
build-date: 2010-09-21 18:10:51 +0400
revno: 3
branch-nick: test-branch
clean: False
revisions: id: <email address hidden>
        message: 'первый'
        date: 2010-09-20 14:22:05 +0400
        id: <email address hidden>
        message: 'second'
        date: 2010-09-20 14:33:05 +0400
        id: <email address hidden>
        message: УРА!

        date: 2010-09-21 04:23:05 +0400

file-revisions: path:
        revision: <email address hidden>
        path: file.txt
        revision: unversioned
        path: файл.txt
        revision: <email address hidden>

C:\Users\regular-user\Desktop\bzr.bzr\test-branch>python ..\current\bzr version
Bazaar (bzr) 2.3.0dev2
  from bzr checkout C:/Users/regular-user/Desktop/bzr.bzr/current
    revision: 5437
    revid: <email address hidden>
    branch nick: BUG_FIX_518609__v1.b
  Python interpreter: c:\dev\Python27\python.exe 2.7.0
  Python standard library: c:\dev\Python27\lib
  Platform: Windows-Vista-6.0.6002-SP2
  bzrlib: C:\Users\regular-user\Desktop\bzr.bzr\current\bzrlib
  Bazaar configuration: C:\Users\regular-user\AppData\Roaming\bazaar\2.0
  Bazaar log file: C:\Users\regular-user\Documents\.bzr.log

Copyright 2005-2010 Canonical Ltd.
http://bazaar.canonical.com/

bzr comes with ABSOLUTELY NO WARRANTY. bzr is free software, and
you may use, modify and redistribute it under the terms of the GNU
General Public License version 2 or later.

Bazaar is part of the GNU Project to produce a free ope...

Read more...

Revision history for this message
Andrej A Antonov (polymorphm) wrote :

typo in BUG_FIX_518609__v1.b.patch (see lp:~polymorphm/bzr/BUG_FIX_518609__v1.b revision 5438 )

Revision history for this message
Eli Zaretskii (eliz) wrote : Re: [Bug 518609] Re: Unicode exception occurs by "version-info"

> Date: Tue, 21 Sep 2010 14:21:45 -0000
> From: Andrej A Antonov <email address hidden>
>
> it is more logical way
>
> patch -- checked in Ubuntu (en_GB.UTF-8 with cyrillic symbals in Branch-
> History) and in Microsoft Vista (cp1251/CP866 with cyrillic symbals in
> Branch-History)
>
>
> Windows terminal output is:
>
>
> C:\Users\regular-user\Desktop\bzr.bzr\test-branch>python ..\current\bzr version-info --all
> revision-id: <email address hidden>
> date: 2010-09-21 04:23:05 +0400
> build-date: 2010-09-21 18:10:39 +0400
> revno: 3
> branch-nick: test-branch
> clean: True
> revisions: id: <email address hidden>
> message: 'первый'
> date: 2010-09-20 14:22:05 +0400
> id: <email address hidden>
> message: 'second'
> date: 2010-09-20 14:33:05 +0400
> id: <email address hidden>
> message: УРА!
>
> date: 2010-09-21 04:23:05 +0400

Thanks.

How do I know which version of Bazaar will include this bugfix?

Revision history for this message
Martin Packman (gz) wrote :

> How do I know which version of Bazaar will include this bugfix?

Same way as always, when a branch lands, the corresponding bug will get marked fixed and the milestone will be set to the first version it will appear in. As a subscriber to this bug, you'll get mail when that happens.

There are complications here however, see the mailing list thread Parth linked above and the current mp from Andrej for more details.

Revision history for this message
Andrej A Antonov (polymorphm) wrote :

yes, the solution (this branch) -- quite not correctly.

I read a subject-discussion 150 times, and finally only on 151 times see what the problem :-) :-) :-)

Revision history for this message
Andrej A Antonov (polymorphm) wrote :

(Last message - the wrong thread .. :))

Alexander Belchenko (bialix)
Changed in bzr:
status: Confirmed → Fix Released
assignee: nobody → Andrej A Antonov (polymorphm)
milestone: none → 2.3b2
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.