ValueError: Can't commit with opened blobs

Moving this up the list. It's happening almost every day.

Do we have any field reports from users about errors? If so, it would be helpful to collect their stories and see if there are any clues about how to make this happen.

My first inclination is that something is going wrong with attachments when creating a new blog entry, but testing that, it seems to work fine. This may be some kind of intermittent timing based issue that could be difficult to track down. I'll keep poking around and see if I can figure something out.

Unfortunately we don't have any field reports. Perhaps the first step is to deploy a fix that, when the exception is raised, saves the offender in var?

--Paul

On Nov 29, 2012, at 11:20 AM, Chris Rossi <email address hidden> wrote:

> Do we have any field reports from users about errors? If so, it would
> be helpful to collect their stories and see if there are any clues about
> how to make this happen.
>
> My first inclination is that something is going wrong with attachments
> when creating a new blog entry, but testing that, it seems to work fine.
> This may be some kind of intermittent timing based issue that could be
> difficult to track down. I'll keep poking around and see if I can
> figure something out.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1081282
>
> Title:
> ValueError: Can't commit with opened blobs
>
> Status in KARL3:
> New
>
> Bug description:
> From the error monitor:
>
> ValueError: Can't commit with opened blobs.
> Exception when processing https://karl.soros.org/communities/democratic-republic-of-congo/blog/add_blogentry.html
> Referer: https://karl.soros.org/communities/democratic-republic-of-congo/blog/add_blogentry.html
>
> Traceback (most recent call last):
> File "/srv/osfkarl/production/11/eggs/pyramid-1.2.1-py2.6.egg/pyramid/tweens.py", line 17, in excview_tween
> response = handler(request)
> File "/srv/osfkarl/production/11/eggs/pyramid_tm-0.5-py2.6.egg/pyramid_tm/__init__.py", line 107, in tm_tween
> return response
> File "/srv/osfkarl/production/11/eggs/pyramid_tm-0.5-py2.6.egg/pyramid_tm/__init__.py", line 73, in __exit__
> return self._retry_or_raise(*sys.exc_info())
> File "/srv/osfkarl/production/11/eggs/pyramid_tm-0.5-py2.6.egg/pyramid_tm/__init__.py", line 60, in _retry_or_raise
> reraise(t, v, tb) # otherwise reraise the exception
> File "/srv/osfkarl/production/11/eggs/pyramid_tm-0.5-py2.6.egg/pyramid_tm/__init__.py", line 69, in __exit__
> self.manager.commit()
> File "/srv/osfkarl/production/11/eggs/transaction-1.1.1-py2.6.egg/transaction/_manager.py", line 89, in commit
> return self.get().commit()
> File "/srv/osfkarl/production/11/eggs/transaction-1.1.1-py2.6.egg/transaction/_transaction.py", line 329, in commit
> self._commitResources()
> File "/srv/osfkarl/production/11/eggs/transaction-1.1.1-py2.6.egg/transaction/_transaction.py", line 443, in _commitResources
> rm.commit(self)
> File "/srv/osfkarl/production/11/eggs/ZODB3-3.10.5_perfmetrics-py2.6-linux-i686.egg/ZODB/Connection.py", line 567, in commit
> self._commit(transaction)
> File "/srv/osfkarl/production/11/eggs/ZODB3-3.10.5_perfmetrics-py2.6-linux-i686.egg/ZODB/Connection.py", line 623, in _commit
> self._store_objects(ObjectWriter(obj), transaction)
> File "/srv/osfkarl/production/11/eggs/ZODB3-3.10.5_perfmetrics-py2.6-linux-i686.egg/ZODB/Connection.py", line 668, in _store_objects
> raise ValueError("Can't commit with opened blobs.")
> ValueError: Can't commit with opened blobs.
>
> To manage notifications about this bug go to:
> https://b...

And I have reproduced it! Uploading multiple attachments at once seems to do it.

Well, this is a peculiar one that is going to prove difficult to test.

As near as I can tell the issue arises because in order to archive uploaded files in the Repozitory, we hand the repozitory the stream from the blobs opened for reading. We weren't specifically closing those files, as we tend not to do for files opened for reading. In this case, because of the flow of control, it was actually a little hard to close them. In most cases, those open file objects wind up getting garbage collected, which causes them to close in time. Under some circumstances, though, those objects still have a non-zero refcount at commit time and have not been garbage collected, leading to the exception above.

The circumstances that lead this condition are entirely deterministic and repeatable. Once I found a particular set of inputs that exposed the problem I was able to reproduce the problem dozens of time while sorting out how to fix it. Despite being deterministic, however, they also seem to be chaotic. Very similar seeming inputs can have different results. It's not clear why one set of inputs exposes the bug while another does not. In my local development environment I stumbled across a particular set of four image files, which when I included them as attachments on a new blog entry, exposed the problem. A set of four different image files, though, would work just fine. It had to be those four specific files. What's more, those same four files don't expose the problem on Karldev. Presumably there is some set of blog attachments that would expose the bug on Karldev but I haven't found them. So reproducing the problem in a testing environment and then proving it's been fixed will be tricky, perhaps beyond feasibility.

At an rate, the fix is on the branch 'crossi-1081282-open-blobs'.

I agree that testing will be hard. I'll put this on a candidate list for a next release (r3.104) and issue a pull request.