KARL3

(Eurasia Prod) mailin: database conflict error

Bug #614368 reported by Jim B. Glenn
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
KARL3
Fix Released
Low
Chris Rossi
KARL3 m45

Bug Description

Reported via error_monitor

ENTRY
Fri Aug 6 01:31:06 2010
Error after 3 tries
Traceback (most recent call last):
  File "/opt/karl/eurasia/3.12-1/src/karl/karl/scripting.py", line 66, in run_daemon
    func()
  File "/opt/karl/eurasia/3.12-1/src/karl/karl/scripts/mailin.py", line 202, in run
    transaction.commit()
  File "/opt/karl/eurasia/3.12-1/eggs/ZODB3-3.8.5-py2.5-linux-x86_64.egg/transaction/_manager.py", line 93, in commit
    return self.get().commit()
  File "/opt/karl/eurasia/3.12-1/eggs/ZODB3-3.8.5-py2.5-linux-x86_64.egg/transaction/_transaction.py", line 325, in commit
    self._commitResources()
  File "/opt/karl/eurasia/3.12-1/eggs/ZODB3-3.8.5-py2.5-linux-x86_64.egg/transaction/_transaction.py", line 427, in _commitResources
    rm.tpc_vote(self)
  File "/opt/karl/eurasia/3.12-1/eggs/ZODB3-3.8.5-py2.5-linux-x86_64.egg/ZODB/Connection.py", line 748, in tpc_vote
    s = vote(transaction)
  File "/opt/karl/eurasia/3.12-1/eggs/ZODB3-3.8.5-py2.5-linux-x86_64.egg/ZEO/ClientStorage.py", line 1005, in tpc_vote
    return self._check_serials()
  File "/opt/karl/eurasia/3.12-1/eggs/ZODB3-3.8.5-py2.5-linux-x86_64.egg/ZEO/ClientStorage.py", line 843, in _check_serials
    raise s
ConflictError: database conflict error (oid 0x5b96, class BTrees.IOBTree.IOBTree, serial this txn started with 0x0388010b1af4c422 2010-08-06 05:31:06.317840, serial currently committed 0x0388010b1bf792bb 2010-08-06 05:31:06.554785)

Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Hmm, I thought we were handling conflict errors via retry on mailin. Chris, feel free to triage some of this out to M46.

Changed in karl3:
assignee: nobody → Chris Rossi (chris-archimedeanco)
importance: Undecided → Low
milestone: none → m45
Chris Rossi (chris-archimedeanco)
Changed in karl3:
status: New → In Progress
Revision history for this message
Chris Rossi (chris-archimedeanco) wrote :

> Hmm, I thought we were handling conflict errors via retry on mailin. Chris, feel free to triage some of this out to M46.
>
We are, but we only catch three consecutive errors before we raise, in case we've run into something actually blocking mailin. In this case, mailin does not seem to be blocked, currently, for mailin, so my best guess is we just got lucky and happened to hit three conflict errors in a row. The only action item I could think to take away from this might be to bump up the number of retries before we're notified.

Chris Rossi (chris-archimedeanco)
Changed in karl3:
status: In Progress → Fix Committed
Revision history for this message
Chris Rossi (chris-archimedeanco) wrote :

Converted retry functionality used when running console scripts as daemons to be time based rather than number of iterations based. By default, now, if a daemon script gets ConflictErrors, it will retry once a minute for 30 minutes. If unable to fulfill its duties for 30 minutes, it will raise an exception which will trigger all of the usual alarm bells.

Revision history for this message
JimPGlenn (jpglenn09) wrote :

fixed

Changed in karl3:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.