history-db should handle OperationalError: database is locked
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
loggerhead |
Triaged
|
Low
|
Unassigned | ||
loggerhead-breezy |
Triaged
|
Low
|
Unassigned |
Bug Description
If you have multiple instances of loggerhead serving the same content, and sharing the same cache files, it is entirely possible that both processes will want to write to the same cache file concurrently.
This is pretty easy to trigger if you use 'bzr serve --http' and configure "http_sql_dir" so you have a permanent shared (single) cache.
A single process currently excludes multiple writers by using a threading.Lock. We should probably just spin around the OperationalError, because that allows us to support multiple concurrent processes, rather than just multiple threads in a single process.
How I set up testing this:
http_sql_dir = /path/to/sql/cache
# Make sure the cache doesn't exist, so that serving a request takes a while to build it
rm /path/to/
cd $launchpad
bzr serve --http --port 8080 &
bzr serve --http --port 8081 &
wget http://
wget http://
The second request should fail and there should be a traceback about sqlite cache file locked.
Marking this critical, because it should really block landing history-db into trunk.
tags: | added: history-db-blocker |
affects: | bzr → loggerhead |
Changed in loggerhead-breezy: | |
status: | New → Triaged |
importance: | Undecided → Low |
I'm dropping this to high because its not a regression/ oops/timeout or escalated by a stakeholder. I do think its very important, just not drop-everything as critical is. And yes, I think its a blocker to merging historydb to trunk.