Breezy

fast-export using existing marks is broken

Bug #1890174 reported by Ben Hutchings
26
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Breezy
Triaged
High
Jelmer Vernooij

Bug Description

Running 'brz fast-export' with the --export-marks (or --marks) option results in an internal error traceback:

$ mkdir brz-test
$ cd brz-test/
$ brz init
Created a standalone tree (format: 2a)
$ touch foo
$ brz add foo
adding foo
$ brz commit -m foo
bzr: warning: The commit message is a file name: "foo".
(use --file "foo" to take commit message from that file)
Committing to: /home/bwh/tmp/brz-test/
added foo
Committed revision 1.
$ brz fast-export --git-branch=trunk . ../brz-test-fastexport --export-marks=../brz-test-marks
16:47:58 Calculating the revisions to include ...
16:47:58 Starting export of 1 revisions ...
brz: ERROR: AttributeError: 'int' object has no attribute 'lstrip'

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/breezy/commands.py", line 1022, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/breezy/commands.py", line 1208, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python3/dist-packages/breezy/commands.py", line 762, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python3/dist-packages/breezy/commands.py", line 787, in run
    return class_run(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/breezy/plugins/fastimport/cmds.py", line 507, in run
    return exporter.run()
  File "/usr/lib/python3/dist-packages/breezy/plugins/fastimport/exporter.py", line 282, in run
    self._save_marks()
  File "/usr/lib/python3/dist-packages/breezy/plugins/fastimport/exporter.py", line 330, in _save_marks
    marks_file.export_marks(self.export_marks_file, revision_ids)
  File "/usr/lib/python3/dist-packages/breezy/plugins/fastimport/marks_file.py", line 81, in export_marks
    f.write(':%s %s\n' % (mark.lstrip(b':').decode('utf-8'),
AttributeError: 'int' object has no attribute 'lstrip'

brz 3.1.0 on python 3.8.5 (Linux-4.19.0-9-amd64-x86_64-with-glibc2.29)
arguments: ['/usr/bin/brz', 'fast-export', '--git-branch=trunk', '--export-
    marks=../brz-test-marks', '.', './brz-test-fastexport']
plugins: bash_completion[3.1.0], changelog_merge[3.1.0],
    commitfromnews[3.1.0], cvs[3.1.0], darcs[3.1.0], email[3.1.0],
    fastimport[3.1.0], flake8[unknown], fossil[3.1.0], github[3.1.0],
    gitlab[3.1.0], hg[3.1.0], launchpad[3.1.0], mtn[3.1.0],
    netrc_credential_store[3.1.0], news_merge[3.1.0], po_merge[3.1.0],
    propose[3.1.0], quilt[unknown], repodebug[3.1.0], rewrite[unknown],
    stats[3.1.0], svn[3.1.0], upload[3.1.0], weave_fmt[3.1.0],
    zsh_completion[3.1.0]
encoding: 'ascii', fsenc: 'utf-8', lang: 'C'

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/brz/+filebug
    including this traceback and a description of the problem.

A different error message is shown when using the --marks option and a previously generated marks file:

    13:47:46 Calculating the revisions to include ...
    13:47:46 Starting export of 446 revisions ...
    brz: ERROR: KeyError: b'null:'

    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/breezy/commands.py", line 1022, in exception_to_return_code
        return the_callable(*args, **kwargs)
      File "/usr/lib/python3.8/site-packages/breezy/commands.py", line 1208, in run_bzr
        ret = run(*run_argv)
      File "/usr/lib/python3.8/site-packages/breezy/commands.py", line 762, in run_argv_aliases
        return self.run(**all_cmd_args)
      File "/usr/lib/python3.8/site-packages/breezy/commands.py", line 787, in run
        return class_run(*args, **kwargs)
      File "/usr/lib/python3.8/site-packages/breezy/plugins/fastimport/cmds.py", line 507, in run
        return exporter.run()
      File "/usr/lib/python3.8/site-packages/breezy/plugins/fastimport/exporter.py", line 277, in run
        self.emit_commits(interesting)
      File "/usr/lib/python3.8/site-packages/breezy/plugins/fastimport/exporter.py", line 266, in emit_commits
        self.emit_commit(revobj, self.ref, trees[parent], trees[revid])
    KeyError: b'null:'

    brz 3.1.0 on python 3.8.3 (Linux-4.19.0-9-cloud-amd64-x86_64-with-glibc2.2.5)
    arguments: ['/usr/bin/brz', 'fast-export', '--git-branch=trunk', '/srv/lorry-
        working-area/delta_pytz/bzr/trunk', '/srv/lorry-working-
        area/delta_pytz/fast-exporttrunk', '--marks=/srv/lorry-working-
        area/delta_pytz/git/marks.bzr']
    plugins: bash_completion[3.1.0], changelog_merge[3.1.0],
        commitfromnews[3.1.0], cvs[3.1.0], darcs[3.1.0], email[3.1.0],
        fastimport[3.1.0], flake8[unknown], fossil[3.1.0], github[3.1.0],
        gitlab[3.1.0], hg[3.1.0], launchpad[3.1.0], mtn[3.1.0],
        netrc_credential_store[3.1.0], news_merge[3.1.0], po_merge[3.1.0],
        propose[3.1.0], quilt[unknown], repodebug[3.1.0], rewrite[unknown],
        stats[3.1.0], svn[3.1.0], upload[3.1.0], weave_fmt[3.1.0],
        zsh_completion[3.1.0]
    encoding: 'utf-8', fsenc: 'utf-8', lang: 'C.UTF-8'

    *** Bazaar has encountered an internal error. This probably indicates a
        bug in Bazaar. You can help us fix it by filing a bug report at
            https://bugs.launchpad.net/brz/+filebug
        including this traceback and a description of the problem.

Revision history for this message
Ben Hutchings (ben-hutchings-y) wrote :

I realise now that I've been testing two different versions.

The first failure is seen in 3.1.0. I believe it was fixed by https://code.launchpad.net/~jelmer/brz/marks-export/+merge/386706 but that fix is not present in any release yet.

The second failure is seen in r7580. I'll re-test with the top of the brz-3.1 branch.

I can close this and open a new report for the second failure if that's helpful.

Revision history for this message
Ben Hutchings (ben-hutchings-y) wrote :

The second failure is seen in r7616. I'll try to come up with a simple reproducer.

Revision history for this message
Ben Hutchings (ben-hutchings-y) wrote :

Reproducer for the second failure (KeyError: b'null:'):

# use Bazaar for the initial checkout, though it might work with Breezy too
bzr checkout lp:libmemcached
# use Breezy for the export, with attached marks file
brz fast-export --git-branch=trunk libmemcached - --marks libmemcached-marks >/dev/null

Jelmer Vernooij (jelmer)
Changed in brz:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Jelmer Vernooij (jelmer)
Ben Hutchings (ben-hutchings-y)
summary: - fast-export marks options are broken
+ fast-export using existing marks is broken
Revision history for this message
Sebastian Geiger (lanoxx) wrote :

I tried to migrate a bazaar repository to git using bzr-git-mass-convert (https://code.launchpad.net/~ubuntu-core-dev/+git/bzr-git-mass-convert/+ref/master) today and ran into this issue. I tried both on Ubuntu 20.04 and with a virtual machine using 20.10 (groovy beta). I am not sure how to continue now.

Revision history for this message
Stdedos (stdedos) wrote :

@Sebastian, are you sure the process did not end? :/

The output is indeed disturbing, but I did manage to extract a small repo to git alright (I only cared about the commits, so I got them)

With that said, fixing the bugs I think is always a good idea - since the software has to be used,

Revision history for this message
void-spark (void-spark) wrote :

Ran into this again today on debian bookworm. Seems it can't find a key which comes from a constant breezy.revision.NULL_REVISION. The actual trees map is empty. Bit of logging/debugging:

            for revid in chunk:
                revobj = history[revid]
                if len(revobj.parent_ids) == 0:
                    self.note("PARENT A")
                    parent = breezy.revision.NULL_REVISION
                else:
                    self.note("PARENT B")
                    parent = revobj.parent_ids[0]
                self.note("REVOBJ")
                self.note(revobj)
                self.note("SELF")
                self.note(self.ref)
                self.note("parent")
                self.note(parent)
                self.note("trees")
                self.note(trees)
                self.emit_commit(revobj, self.ref, trees[parent], trees[revid])

16:34:35 Calculating the revisions to include ...
16:34:35 Starting export of 1355 revisions ...
16:34:35 PARENT A
16:34:35 REVOBJ
16:34:35 <Revision id <email address hidden>'>
16:34:35 SELF
16:34:35 b'refs/heads/toykeeper'
16:34:35 parent
16:34:35 b'null:'
16:34:35 trees
16:34:35 {}
brz: ERROR: KeyError: b'null:'

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/breezy/commands.py", line 1036, in exception_to_return_code
    return the_callable(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/breezy/commands.py", line 1228, in run_bzr
    ret = run(*run_argv)
          ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/breezy/commands.py", line 779, in run_argv_aliases
    return self.run(**all_cmd_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/breezy/commands.py", line 804, in run
    return class_run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/breezy/plugins/fastimport/cmds.py", line 505, in run
    return exporter.run()
           ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/breezy/plugins/fastimport/exporter.py", line 277, in run
    self.emit_commits(interesting)
  File "/usr/lib/python3/dist-packages/breezy/plugins/fastimport/exporter.py", line 266, in emit_commits
    self.emit_commit(revobj, self.ref, trees[parent], trees[revid])
                                       ~~~~~^^^^^^^^
KeyError: b'null:'

I used to get around it by using a very old version on debian buster, but for some reason buster takes forever to build it's python packages or something now..

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.