cache corruption on api.launchpad.net,1.0,-application,vnd.sun.wadl+xml,fc06437932a618a4b30d0d0417f9234c

Bug #1014621 reported by Vincent Ladeuil
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
launchpadlib
Triaged
High
Unassigned

Bug Description

While running tests on quantal for the udd package importer I encountered a series of tracebacks:

Traceback (most recent call last):
  File "/pkgimport/scripts/bin/import-package", line 5, in <module>
    sys.exit(main(sys.argv))
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 1172, in main
    only_before=options.only_before)
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 1072, in _import_package
    possible_transports=possible_transports)
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 952, in handle_collisions
    if clean_collision(importp, suite, db, temp_dir, download_dir, name, updates_branch):
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 886, in clean_collision
    return check_same(importp, db, revid, name, updates_branch, temp_dir, download_dir)
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 853, in check_same
    importp, db, revid, name, updates_branch, temp_dir, download_dir)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 132, in run
    self.cleanups, self.func, self, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/bzrlib/cleanup.py", line 166, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/pkgimport/scripts/udd/scripts/import_package.py", line 873, in _check_same
    pull_debian=False)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/import_dsc.py", line 1243, in import_package
    pull_debian=pull_debian)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/import_dsc.py", line 1154, in _import_normal_package
    timestamp=timestamp, file_ids_from=file_ids_from)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/import_dsc.py", line 1003, in import_debian
    get_commit_info_from_changelog(changelog, self.branch)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/util.py", line 532, in get_commit_info_from_changelog
    bugs = find_bugs_fixed(changes, branch, _lplib=_lplib)
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/util.py", line 458, in find_bugs_fixed
    deb_bugs = _lplib.debian_bugs_for_ubuntu_bug(match.group(0))
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/launchpad.py", line 84, in debian_bugs_for_ubuntu_bug
    lp = get_launchpad()
  File "/usr/lib/python2.7/dist-packages/bzrlib/plugins/builddeb/launchpad.py", line 43, in get_launchpad
    service_root=LPNET_SERVICE_ROOT)
  File "/usr/lib/python2.7/dist-packages/launchpadlib/launchpad.py", line 373, in login_anonymously
    version=version)
  File "/usr/lib/python2.7/dist-packages/launchpadlib/launchpad.py", line 198, in __init__
    credentials, service_root, cache, timeout, proxy_info, version)
  File "/usr/lib/python2.7/dist-packages/lazr/restfulclient/resource.py", line 460, in __init__
    self._wadl = self._browser.get_wadl_application(self._root_uri)
  File "/usr/lib/python2.7/dist-packages/lazr/restfulclient/_browser.py", line 300, in get_wadl_application
    return Application(str(url), content)
  File "/usr/lib/python2.7/dist-packages/wadllib/application.py", line 1071, in __init__
    self.doc = self._from_string(markup)
  File "/usr/lib/python2.7/dist-packages/wadllib/application.py", line 1108, in _from_string
    return self._from_stream(BytesIO(markup))
  File "/usr/lib/python2.7/dist-packages/wadllib/application.py", line 1095, in _from_stream
    for event, elem in ET.iterparse(stream, events):
  File "<string>", line 107, in next
cElementTree.ParseError: unclosed token: line 13343, column 6

Deleting .launchpadlib/api.launchpad.net/cache/api.launchpad.net,1.0,-application,vnd.sun.wadl+xml,fc06437932a618a4b30d0d0417f9234c addressed my issue.

I had a brief look at the file before deleting it and it appeared truncated.

This occurred at least three times so I expect it will happen again, if the
file itself is needed for analysis, I'll keep a copy next time.

I don't remember seeing that kind of failure on jubany (running lucid) nor
on precise.

Revision history for this message
Vincent Ladeuil (vila) wrote :

My understanding here is that two processes are doing the same query and stepping on each other toes. Updating the file itself should be atomic to avoid this (create a tmp file, rename to the existing one) or something.

Changed in launchpadlib:
status: New → Triaged
importance: Undecided → High
tags: added: api launchpadlib
Revision history for this message
Chris Johnston (cjohnston) wrote :

I'm seeing the same issue. It starts after receiving a timeout from LP, and since LP timeouts are increasing in frequency, I suspect this will start happening more often.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.