Incorrect DB-API wrapper exception type caching

Bug #1845702 reported by Colin Watson
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Storm
Fix Released
Critical
Colin Watson

Bug Description

I noticed this peculiar OOPS when pushing a Bazaar branch to Launchpad:

$ bzr push
Using saved push location: lp:~cjwatson/launchpad/remove-tickcount
Using default stacking branch /+branch-id/24637 at chroot-140673014279760:///~cjwatson/launchpad/
<Fault 380: 'An unexpected error has occurred while updating a Launchpad branch. Please report a Launchpad bug and quote: OOPS-470576daeaa7e2b13e70874860fb2adc.'>
"<Fault 380: 'An unexpected error has occurred while updating a Launchpad branch. Please report a Launchpad bug and quote: OOPS-470576daeaa7e2b13e70874860fb2adc.'>"
Created new stacked branch referring to /+branch-id/24637.

The OOPS says:

  Fault: <Fault -1: 'Unexpected Zope exception: LaunchpadTimeoutError: Statement: u"UPDATE Branch SET last_mirrored_id=%s, last_mirrored=CURRENT_TIMESTAMP AT TIME ZONE \'UTC\', stacked_on=%s WHERE Branch.id = %s"\nParameters:[<storm.sqlobject.AutoUnicodeVariable object at 0x7efdba357de8>, <storm.variables.IntVariable object at 0x7efdbae15230>, <storm.variables.IntVariable object at 0x7efdbae152a8>]\nOriginal error: QueryCanceledError(\'could not serialize access due to concurrent update\\n\',)'>

    Traceback (most recent call last):
  File "/srv/bazaar.launchpad.net/production/codehosting2-rev-18816/lib/lp/services/twistedsupport/__init__.py", line 127, in wrapped
    return func(*args, **kwargs)
  File "/srv/bazaar.launchpad.net/production/codehosting2-rev-18816/lib/lp/codehosting/vfs/branchfs.py", line 688, in got_path_info
    data['id'], **info)
  File "/srv/bazaar.launchpad.net/production/codehosting2-rev-18816/lib/lp/codehosting/vfs/branchfsclient.py", line 123, in branchChanged
    repository_string)
  File "/srv/bazaar.launchpad.net/production/codehosting2-rev-18816/lib/lp/services/twistedsupport/xmlrpc.py", line 52, in callRemote
    method_name, *args, **kwargs)
--- <exception caught here> ---
  File "/srv/bazaar.launchpad.net/production/codehosting2-rev-18816/env/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/srv/bazaar.launchpad.net/production/codehosting2-rev-18816/lib/lp/services/twistedsupport/xmlrpc.py", line 37, in callRemote
    return getattr(self._proxy, method_name)(*args, **kwargs)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1243, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1602, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.7/xmlrpclib.py", line 1283, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1316, in single_request
    return self.parse_response(response)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1493, in parse_response
    return u.close()
  File "/usr/lib/python2.7/xmlrpclib.py", line 800, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault -1: 'Unexpected Zope exception: LaunchpadTimeoutError: Statement: u"UPDATE Branch SET last_mirrored_id=%s, last_mirrored=CURRENT_TIMESTAMP AT TIME ZONE \'UTC\', stacked_on=%s WHERE Branch.id = %s"\nParameters:[<storm.sqlobject.AutoUnicodeVariable object at 0x7efdba357de8>, <storm.variables.IntVariable object at 0x7efdbae15230>, <storm.variables.IntVariable object at 0x7efdbae152a8>]\nOriginal error: QueryCanceledError(\'could not serialize access due to concurrent update\\n\',)'>

Initially I thought I'd just got unlucky with timing in a way I never had before, but then I looked more closely. psycopg2 turns the "could not serialize access due to concurrent update" error into TransactionRollbackError, not QueryCanceledError. Both of these are subclasses of OperationalError, and OperationalError is in storm.exceptions._wrapped_exception_types but neither of the subclasses is.

I think that QueryCanceledError happens to be seen first, causing Database._make_combined_exception_type to cache a wrapper for it, but it caches it using OperationalError as a key, so when Storm later encounters TransactionRollbackError it returns a QueryCanceledError instead. That isn't good, and we need to fix the caching.

Related branches

Colin Watson (cjwatson)
Changed in storm:
status: New → In Progress
importance: Undecided → Critical
assignee: nobody → Colin Watson (cjwatson)
Colin Watson (cjwatson)
Changed in storm:
status: In Progress → Fix Committed
Colin Watson (cjwatson)
Changed in storm:
milestone: none → 0.22
Revision history for this message
Colin Watson (cjwatson) wrote :

Fixed in Storm 0.22.

Changed in storm:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.