git: AssertionError: Invalid sha for <Commit ...>: ...

Bug #1313861 reported by Michael Lustfield
100
This bug affects 18 people
Affects Status Importance Assigned to Milestone
Bazaar Git Plugin
Triaged
Undecided
Unassigned
Breezy
Fix Released
High
Jelmer Vernooij
Launchpad code imports
Confirmed
Undecided
Unassigned

Bug Description

I have an import from Debian git that is now failing. https://code.launchpad.net/~nginx/nginx/debian
The log: http://launchpadlibrarian.net/174083895/nginx-nginx-debian.log

I've done what I know how to do to verify that this repository is accurate. Maybe there's something wrong in the git repo, but I can't find anything. I'm able to clone the repo fine and it had been working fine.

William Grant (wgrant)
affects: launchpad → bzr-git
Revision history for this message
anatoly techtonik (techtonik) wrote :
Revision history for this message
anatoly techtonik (techtonik) wrote :

I switched my imports from git->bzr to git->git.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

This is fixed with brz-git.

Changed in brz-git:
status: New → Fix Released
assignee: nobody → Jelmer Vernooij (jelmer)
Revision history for this message
Pavel (spvkgn) wrote :

Same here https://code.launchpad.net/~mpv-team/mpv/master/
Last successful import was on 2018-03-16.

Jelmer Vernooij (jelmer)
Changed in brz-git:
milestone: none → integration
Revision history for this message
Petr Kubánek (petr-kubanek) wrote :
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

This is fixed in brz-git, Launchpad runs bzr-git at the moment.

Changed in launchpad:
status: New → Confirmed
Revision history for this message
Colin Watson (cjwatson) wrote :

We have internal agreement on moving our backend over to Breezy, but I was kind of hoping for a non-alpha release first, ideally. Is that planned soon?

Revision history for this message
Jelmer Vernooij (jelmer) wrote : Re: [Bug 1313861] Re: AssertionError: Invalid sha for <Commit ...>: ...

On Tue, Apr 10, 2018 at 09:49:39AM -0000, Colin Watson wrote:
> We have internal agreement on moving our backend over to Breezy, but I
> was kind of hoping for a non-alpha release first, ideally. Is that
> planned soon?
Do you have a bug tracking the migration that I can update on our
status?

I wouldn't recommend a migration just yet, since we're still making
largeish changes to the core APIs - i.e. to support git, nested trees and
colocated branches.

Hopefully we can do some sort of alpha release when the core APIs have
stabilized.

Cheers,

Jelmer

--
Jelmer Vernooij <email address hidden>
PGP Key: https://www.jelmer.uk/D729A457.asc

Jelmer Vernooij (jelmer)
affects: brz-git → brz
Changed in brz:
milestone: integration → none
milestone: none → 3.0.0
summary: - AssertionError: Invalid sha for <Commit ...>: ...
+ git: AssertionError: Invalid sha for <Commit ...>: ...
Revision history for this message
Seafile Ltd (seafile) wrote :
Revision history for this message
Rob Norris (rw-norris) wrote :

Also happening to me:

https://code.launchpad.net/~rw-norris/viking/trunk

http://launchpadlibrarian.net/370842251/rw-norris-viking-trunk.log

Both claimed failing commits in my log[1] & for seafile[1] are of a "Merge pull request XXX ..." type nature performed in Github.com.

[1] https://github.com/viking-gps/viking/commit/206165de5fe6812da5d509121a6f399a79a78021
[2] https://github.com/haiwen/seafile/commit/b128c7b574cbfe3c1adcc2f5ccf012f5caa07ae8

Hope that helps any investigations.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

FWIW it's clear what the problem is here, and there is a fix available for this issue in Breezy.
(I've pushed a manual import of the viking branch to lp:~jelmer/viking/brz)

In order for this issue to be fixed in Launchpad, it either needs to:

* migrate to Breezy; this is currently blocked on Breezy not having a stable API - yet (see #7)
* somebody needs to fix the issue in bzr-git and then update Launchpad to use a new bzr-git

Revision history for this message
Seafile Ltd (seafile) wrote :

Hello guys, is there any update on this? It's blocking us from releasing new versions of seafile client on all ubuntu versions since the sync from github can't work due to this bug.

We tried to push to the bazaar branch manually, but it seems one can't push to an imported branch[1], and we don't want to setup a new branch and recipes unless there is really no alternatives.

[1] http://linux.derkeiler.com/Mailing-Lists/Ubuntu/2014-09/msg00446.html

Revision history for this message
Colin Watson (cjwatson) wrote :

Jelmer, could you point to what commits fixed this in brz-git? We could backport it to Launchpad's bzr-git branch, which might be the most expedient approach for now.

Revision history for this message
Jelmer Vernooij (jelmer) wrote : Re: [Bug 1313861] Re: git: AssertionError: Invalid sha for <Commit ...>: ...

On Tue, Aug 21, 2018 at 11:01:19AM -0000, Colin Watson wrote:
> Jelmer, could you point to what commits fixed this in brz-git? We could
> backport it to Launchpad's bzr-git branch, which might be the most
> expedient approach for now.

Note that the title of this bug is a symptom that can be caused by any
kind of data corruption bug. bzr-git is trying to create a Git object
from some Bazaar data and comes up with content that doesn't match
the SHA1 checksum that it expected.

It's possible that the various repositories mentioned in comments
for this bug are because different issues; I've only looked at the
repository in the initial description. That said..

It looks like for seafile, the repository is corrupted, since bzr-git
can't recreate the original git objects from it.

This can be seen by running "bzr git-objects".

$ bzr branch lp:~seafile-team/seafile/lpad
$ cd lpad
$ bzr git-objects

This means that that branch will need to be nuked and then reimported
once the bug is fixed.

The bug that introduces the corruption is still present in lp:bzr-git
- a fresh clone with lp:bzr and lp:bzr-git still fails the "bzr git-objects" test
mentioned above. For Breezy this is not the case.

Unfortunately brz-git has changed significantly since lp:bzr-git, and
we can't just isolate a smaller number of commits to backport. :(

blodeuwedd:~/src/breezy% bzr diff -rbranch:lp:bzr-git.. breezy/git | diffstat -s
 63 files changed, 8706 insertions(+), 4458 deletions(-)

--
Jelmer Vernooij <email address hidden>
PGP Key: https://www.jelmer.uk/D729A457.asc

Jelmer Vernooij (jelmer)
Changed in bzr-git:
status: New → Triaged
Changed in brz:
importance: Undecided → High
Revision history for this message
Nicholas Zatkovich (nickz-roshub) wrote :

Any update on this? Any timeline for this bug to be fixed in the version of bzr-git used by launchpad?

Revision history for this message
Watteel (piwi3910) wrote :

any updates?

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

There are two paths towards fixing this:

1) fixing bzr-git or backporting the fix from breezy to bzr-git. I don't believe there is anybody working on this.

2) Switching Launchpad over to using Breezy rather than Bazaar, since Breezy has already fixed this bug. I believe this is the eventual plan, but it's blocked on several other things (including lack of svn support in Breezy).

tags: added: code-import git-support import
Colin Watson (cjwatson)
tags: added: git lp-code
removed: git-support import
Revision history for this message
i30817 (i30817) wrote :

I have encountered this problem on lp:munt while auto-importing it from its github page

http://launchpadlibrarian.net/462662122/munt-munt-trunk.log

but it occurs (with a different object, thus sha1) even with the bzr branch/bzr git-objects test with a local branch of only the commits already there.

This is pretty annoying since to change to bzr git i have to recreate all of the recipes and the repositories they use *and* the local repositories equivalents, because i'd have to change '# bzr-builder format ...' to '# git-builder format ...'

Please fix this!

Revision history for this message
Cameron White (cameronwhite91) wrote :

I've also been encountering this. https://code.launchpad.net/~pinta-maintainers/pinta/trunk hasn't been updating for several months, preventing new translation strings from being imported.

Are there any updates on this?

Revision history for this message
FloSoft (flosoft) wrote :

We have the same problem here:

https://code.launchpad.net/~s25rttrteam/s25rttr/s25rttr-languages

blocking us from importing language changes.

Can't you simply add support for having git repositories for language/translation support?

Revision history for this message
Colin Watson (cjwatson) wrote :

@flosoft I started on that a while ago: the launchpad-buildd side has the necessary support as of August, but we still need to do the corresponding things to Launchpad itself. This is a bit harder than you might think due to some data model differences between bzr and git hosting in Launchpad (basically, series aren't really involved in git hosting, but they're quite central to how translation hosting works, and we need to work out how to bridge that gap). Hopefully in 2021 though!

Colin Watson (cjwatson)
affects: launchpad → lp-codeimport
Revision history for this message
Alkis Georgopoulos (alkisg) wrote :

Same here with https://code.launchpad.net/~epoptes/epoptes/epoptes
Until it's fixed, we'll probably need to manually upload the translations.pot periodically...

Revision history for this message
Olly Betts (ojwb) wrote :

It seems this wasn't fixed in 2021 as I'm still seeing it at https://code.launchpad.net/~ojwb/survex/master and after deleting and recreating the bzr mirror it imports OK and then seems fails to ever update after that:

 Last successful import was on 2022-02-21.
[Failure] Import started 14 hours ago on juju-1e3bde-prod-lp-code-import-15 and finished 14 hours ago taking 5 seconds — see the log
[Failure] Import started on 2022-02-23 on juju-1e3bde-prod-lp-code-import-14 and finished on 2022-02-23 taking 5 seconds — see the log
[Failure] Import started on 2022-02-22 on juju-1e3bde-prod-lp-code-import-14 and finished on 2022-02-22 taking 5 seconds — see the log
[Failure] Import started on 2022-02-22 on juju-1e3bde-prod-lp-code-import-12 and finished on 2022-02-22 taking 5 seconds — see the log
[Failure] Import started on 2022-02-21 on juju-1e3bde-prod-lp-code-import-14 and finished on 2022-02-21 taking 5 seconds — see the log
[Success] Import started on 2022-02-21 on juju-1e3bde-prod-lp-code-import-12 and finished on 2022-02-21 taking 2 minutes — see the log

So at least for my case, it doesn't seem that pre-existing corruption in the bzr repo is to blame, because the bzr repo is brand new.

I don't actually want a bzr mirror of my git repo as such, but that seems to be required by launchpad's translation feature since the bzr mirror got automatically created for me when I pointed to translations in my git repo. The only workaround there seems to be (repeatedly deleting and recreating the bzr mirror) isn't really practical, and if I'm going to do that I might as well just upload the translation files manually instead.

This hasn't been fixed in approaching 8 years now and I wonder if that reflects that bzr support isn't a priority any more and/or that fixing this is hard. Perhaps the translation feature could be changed to work directly from git repos more easily?

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Are the code importers still using Bazaar rather Breezy? If so, this is probably just a matter of moving over to Breezy.

Revision history for this message
Colin Watson (cjwatson) wrote :

I believe the remaining blocker to getting lp-codeimport moved to Breezy is finishing the cscvs port. cscvs is extremely old and crufty code, but there are still a few CVS-to-bzr imports on Launchpad and I'd rather not break them if I can avoid it; at present our architecture doesn't allow migrating these separately from other import types. I have most of a port, which isn't really very complicated in itself, but last time I tried I had extreme difficulty getting the test suite to pass so that I could verify that I hadn't broken anything obvious. I guess I'll give it another try at some point.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Fri, Feb 25, 2022 at 10:27:01PM -0000, Colin Watson wrote:
> I believe the remaining blocker to getting lp-codeimport moved to Breezy
> is finishing the cscvs port. cscvs is extremely old and crufty code,
> but there are still a few CVS-to-bzr imports on Launchpad and I'd rather
> not break them if I can avoid it; at present our architecture doesn't
> allow migrating these separately from other import types. I have most
> of a port, which isn't really very complicated in itself, but last time
> I tried I had extreme difficulty getting the test suite to pass so that
> I could verify that I hadn't broken anything obvious. I guess I'll give
> it another try at some point.

Is your port and/or the test output available somewhere public?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.