Premature end of data in tag file line 247651 with libxml2

Status changed to 'Confirmed' because the bug affects multiple users.

When will this be fixed? I'm currently using a single server to host both debs and rpms, and this bug triggers fairly frequently, causing me to have to take down my repo and regenerate it.

Thank you for your bug report. Do you have a testcase you could add to the bug? That would be make easier to test/find out the fix to backport

Aron, you seem to co-maintain libxml in Debian, do you have any idea about the issue?

Hi Sebastien,

The problem is I've found no consistent trigger for the bug. I have 225 packages in my repository, built out for two different branches and two oses (so 225x4). createrepo just randomly corrupts when executing 'createrepo --update .' when adding a new source/binary rpm pair. It is trivially solvable (rm -rf repodata .repodata; createrepo .), but having to briefly have a 100% repo outage is the problem.

I've not seen anything specific about the packages that would cause this to occur. It seems to be some type of race condition during the update process, because of it complaining about " I/O error : No such file or directory".

FWIW, I am currently experiencing the same error when running "createrepo" on a repo containing 20 RPMs. The host system is running Ubuntu 14.04.5 LTS, libxml2 version is 2.9.1+dfsg1-3ubuntu. For me, the error occurs every time, removing the repodata directories does not resolve this.

 % createrepo .
Spawning worker 0 with 20 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
I/O error : No such file or directory

(process:18005): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory.
This indicates a bug in someone's code. You must ensure an error is NULL before it's set.
The overwriting error message was: Parsing filelists.xml error: Premature end of data in tag package line 2263

(process:18005): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory.
This indicates a bug in someone's code. You must ensure an error is NULL before it's set.
The overwriting error message was: Parsing filelists.xml error: Premature end of data in tag filelists line 2

Traceback (most recent call last):
  File "/usr/share/createrepo/genpkgmetadata.py", line 308, in <module>
    main(sys.argv[1:])
  File "/usr/share/createrepo/genpkgmetadata.py", line 280, in main
    mdgen.doRepoMetadata()
  File "/usr/lib/python2.7/dist-packages/createrepo/__init__.py", line 999, in doRepoMetadata
    rp.getFilelists(complete_path, csum)
  File "/usr/lib/python2.7/dist-packages/sqlitecachec.py", line 54, in getFilelists
    self.repoid))
TypeError: Parsing filelists.xml error: Premature end of data in tag file line 2332

I have seen a similar issue using 'yum check-update' on Ubunty Trusty with libxml2=2.9.1+dfsg1-3ubuntu4.8 failing to parse the primary.xml.gz index.

$ yum check-update

custom-repo: [########################################################################### ] 6900/6903
(process:1482): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory.
This indicates a bug in someone's code. You must ensure an error is NULL before it's set.
The overwriting error message was: Parsing primary.xml error: attributes construct error

(process:1482): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory.
This indicates a bug in someone's code. You must ensure an error is NULL before it's set.
The overwriting error message was: Parsing primary.xml error: Couldn't find end of Start Tag size line 238347

(process:1482): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory.
This indicates a bug in someone's code. You must ensure an error is NULL before it's set.
The overwriting error message was: Parsing primary.xml error: Premature end of data in tag package line 238338

(process:1482): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory.
This indicates a bug in someone's code. You must ensure an error is NULL before it's set.
The overwriting error message was: Parsing primary.xml error: Premature end of data in tag metadata line 2

Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 288, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 140, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 436, in doCommands
    self._getTs(needTsRemove)
  File "/usr/lib/python2.7/dist-packages/yum/depsolve.py", line 101, in _getTs
    self._getTsInfo(remove_only)
  File "/usr/lib/python2.7/dist-packages/yum/depsolve.py", line 112, in _getTsInfo
    pkgSack = self.pkgSack
  File "/usr/lib/python2.7/dist-packages/yum/__init__.py", line 892, in <lambda>
    pkgSack = property(fget=lambda self: self._getSacks(),
  File "/usr/lib/python2.7/dist-packages/yum/__init__.py", line 673, in _getSacks
    self.repos.populateSack(which=repos)
  File "/usr/lib/python2.7/dist-packages/yum/repos.py", line 294, in populateSack
    sack.populate(repo, mdtype, callback, cacheonly)
  File "/usr/lib/python2.7/dist-packages/yum/yumRepo.py", line 190, in populate
    dobj = repo_cache_function(xml, csum)
  File "/usr/lib/python2.7/dist-packages/sqlitecachec.py", line 46, in getPrimary
    self.repoid))
TypeError: Parsing primary.xml error: Specification mandate value for attribute archive

I have seen this a few times, but never took the time to dig into it until recently. What I found was the size of the primary.xml.gz file is what seems to be causing the failure. If (size of file in bytes)/(8192 bytes) has a remainder of 1 byte this failure will ...