Database conflict error in mailin drops emails?

Bug #394355 reported by Chris Rossi
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
KARL3
Fix Released
Medium
Tres Seaver

Bug Description

I will post a recent log from mailin on OSI's production below. As far as I can tell from the log, 12 incoming messages were processed but then a database conflict error occurred at commit time. In the next log entry, no messages are processed. This suggests to me that the 12 in the previous transaction have been lost, probably because the transaction on the pending.db has already been committed before the transaction on the zodb.

We need to make sure that the mailin will retry incoming messages in the event that the zodb transaction cannot be committed.

Revision history for this message
Chris Rossi (chris-archimedeanco) wrote :

Tres, can you take a look at this? I'm making this a high priority because it means a possible loss of customer data.

Changed in karl3:
assignee: nobody → Tres Seaver (tseaver)
importance: Undecided → High
Revision history for this message
Chris Rossi (chris-archimedeanco) wrote :

Since I have the message ids of those 12 messages in the log and the emails are still present in the filesystem, I'm going to attempt to manually requeue them in the pending.db. Does that seem like a reasonable thing to do, Tres?

Revision history for this message
Tres Seaver (tseaver) wrote : Re: [Bug 394355] Re: Database conflict error in mailin drops emails?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Rossi wrote:
> Since I have the message ids of those 12 messages in the log and the
> emails are still present in the filesystem, I'm going to attempt to
> manually requeue them in the pending.db. Does that seem like a
> reasonable thing to do, Tres?

I would first check the blogs of the affected communities to ensure that
the messages didn't in fact get through.

If in fact those blog entries weren't created, re-adding the message IDs
to the sqlite pending queue would probably work. You might should apply
the following patch first, to ensure that changes to the pending table
get committed only if the ZODB transaction commits.

- --- osi/scripts/mailin.py (revision 3376)
+++ osi/scripts/mailin.py (working copy)
@@ -272,8 +272,8 @@
             print
         if not self.dry_run:
             import transaction
+ transaction.commit()
             self.pending.sql.commit()
- - transaction.commit()

 def main():
     MailinRunner(sys.argv)()

I will then look at using a mechanism which coordinates the two
transactions more cleanly.

Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKS9MK+gerLs4ltQ4RAno5AJ9Tc+rvrqdVxMEUMmLkHGfgi9gElACeK7n/
QMinMk02Gd2xisc+X1OLkF4=
=It6c
-----END PGP SIGNATURE-----

Revision history for this message
Chris Rossi (chris-archimedeanco) wrote :

I have gotten the 12 emails from the original error distributed to where
they need to go.

Thanks,
Chris

Revision history for this message
Chris Rossi (chris-archimedeanco) wrote :

Scheduling for this week.

Changed in karl3:
importance: High → Medium
milestone: none → m22
Tres Seaver (tseaver)
Changed in karl3:
status: New → In Progress
Revision history for this message
Tres Seaver (tseaver) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

 status fixcommitted

See revision 3483 in the karl trunk.

Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 <email address hidden>
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKVMIu+gerLs4ltQ4RArdjAJ4uYHW5IXq4yuIkpMlDxcR+m1DbVgCcDEi4
56L1tyYFufnfkhCecEhJHhM=
=nq7O
-----END PGP SIGNATURE-----

Changed in karl3:
status: In Progress → Fix Committed
Changed in karl3:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.