Not handling administrative shutdown under Oneiric

Bug #871596 reported by Jeroen T. Vermeulen
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
Critical
Gavin Panella
Storm
Fix Released
Critical
Gavin Panella

Bug Description

On Oneiric I've been unable to run many tests. For example, when running "./bin/test lp.buildmaster.tests.test_manager" I see Librarian failing repeatedly:

 psycopg2.OperationalError: terminating connection due to administrator command
 SSL connection has been closed unexpectedly

That's in the Librarian log. The test itself will then fail with something like:

UploadFailed: Server said: 500 Internal server error

I'm trying to paste the output, but gedit is having trouble with the file I dumped it into. And that reminds me: the Librarian log output on these failed test runs contains what look like blocks of null bytes, several kilobytes (but no particularly familiar number) at a time. The blocks seem to get longer with each successive test.

In "less," the chunks of librarian log look like: http://paste.ubuntu.com/705277/

I tried checking for running librarian instances and other twisted-based scripts with "ps -ef | grep twist" but none seem to be running. This is strange; I'm used to seeing at least a bunch of Librarian processes (both before and after I started having this problem).

Related branches

Revision history for this message
Jeroen T. Vermeulen (jtv) wrote :

I also tried starting a Librarian manually, with “bin/start_librarian.” That starts up happily, without any weirdness in the blog. I suppose things only start getting weird at the time of first use. It's not technically impossible though that those null bytes are just sitting in a buffer somewhere waiting to be flushed by the first error message.

Revision history for this message
Julian Edwards (julian-edwards) wrote :

I am seeing the same problem. However, it only happens when running more than one test - the second one fails with the upload error.

Revision history for this message
Raphaël Badin (rvb) wrote :

I can confirm this is Oneiric specific. Works fine on Natty.

Gavin Panella (allenap)
Changed in launchpad:
assignee: nobody → Gavin Panella (allenap)
Revision history for this message
Stuart Bishop (stub) wrote :

This is a Storm issue, as it is tasked with intercepting the various disconnection exceptions, resetting the stores appropriately and raising a DisconnectionError for the application to deal with.

Changed in launchpad:
status: Triaged → Invalid
summary: - Can't run tests involving Librarian
+ Not handling administrative shutdown under Oneiric
Gavin Panella (allenap)
Changed in storm:
status: New → In Progress
importance: Undecided → Critical
assignee: nobody → Gavin Panella (allenap)
Revision history for this message
Aaron Bentley (abentley) wrote :

I am getting this symptom on Natty, too.

Revision history for this message
William Grant (wgrant) wrote :

This is caused by a change in libpq5 8.4.9, which is now in Lucid. Parts of the DC (including buildbot) have been upgraded, so test failures are everywhere.

Gavin Panella (allenap)
Changed in storm:
status: In Progress → Fix Committed
Changed in launchpad:
status: Invalid → In Progress
Gavin Panella (allenap)
Changed in launchpad:
status: In Progress → Fix Released
William Grant (wgrant)
Changed in storm:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.