Twisted reactors not given chance to shut down properly in mod_wsgi
Bug #1350786 reported by
Gavin Panella
This bug affects 2 people
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| MAAS |
Fix Released
|
High
|
Gavin Panella | ||
Bug Description
The symptom is that stale records are being left behind in the `eventloops` table, pointing to defunct event-loops. These are cleared up once they're 5 minutes old, but in the meantime there are alarming log messages. MAAS may be operating correctly but it's hard to see between these log messages.
Two things we could do:
- Quieten the log messages.
- Investigate ways of getting mod_wsgi to gracefully shutdown Python processes (or discover if there's a different root cause).
Related branches
lp://staging/~allenap/maas/regiond
- Blake Rouse (community): Approve
- Christian Reis: Pending requested
-
Diff: 1892 lines (+753/-380)36 files modifiedHACKING.txt (+2/-2)
Makefile (+10/-8)
buildout.cfg (+1/-1)
contrib/maas-http.conf (+8/-28)
contrib/maas_local_settings.py (+1/-65)
contrib/wsgi.py (+0/-32)
etc/apache.saucy.conf (+12/-3)
etc/demo_maas_cluster.conf (+1/-1)
services/clusterd/run (+1/-1)
services/regiond/run (+4/-4)
services/reloader/run (+19/-15)
setup.py (+1/-2)
src/maas/__init__.py (+1/-20)
src/maas/demo.py (+0/-39)
src/maas/development.py (+1/-1)
src/maas/settings.py (+66/-35)
src/maasserver/clusterrpc/tests/test_utils.py (+2/-2)
src/maasserver/clusterrpc/utils.py (+2/-3)
src/maasserver/eventloop.py (+36/-43)
src/maasserver/middleware.py (+5/-18)
src/maasserver/plugin.py (+52/-0)
src/maasserver/start_up.py (+7/-14)
src/maasserver/testing/__init__.py (+4/-0)
src/maasserver/tests/test_eventloop.py (+38/-1)
src/maasserver/tests/test_plugin.py (+79/-0)
src/maasserver/tests/test_webapp.py (+176/-0)
src/maasserver/utils/tests/test_async.py (+0/-4)
src/maasserver/webapp.py (+124/-0)
src/maastesting/testcase.py (+4/-0)
src/provisioningserver/logger/__init__.py (+31/-0)
src/provisioningserver/plugin.py (+17/-18)
src/provisioningserver/tests/test_plugin.py (+5/-15)
src/provisioningserver/utils/tests/test_twisted.py (+12/-2)
src/provisioningserver/utils/twisted.py (+1/-1)
twisted/plugins/maasclusterd.py (+3/-2)
twisted/plugins/maasregiond.py (+27/-0)
lp://staging/~allenap/maas/packaging.regiond
- Andres Rodriguez (community): Approve
-
Diff: 297 lines (+80/-53)11 files modifieddebian/changelog (+31/-1)
debian/control (+1/-1)
debian/maas-cluster-controller.maas-clusterd.upstart (+2/-6)
debian/maas-region-controller-min.apport (+17/-14)
debian/maas-region-controller-min.install (+3/-3)
debian/maas-region-controller-min.logrotate (+0/-8)
debian/maas-region-controller-min.maas-regiond.upstart (+19/-0)
debian/maas-region-controller-min.postinst (+2/-3)
debian/maas-region-controller.postinst (+3/-3)
debian/patches/01-fix-database-settings.patch (+1/-14)
debian/rules (+1/-0)
lp://staging/~allenap/maas/rename-scripts-and-prepare-for-twistd-in-region
- Gavin Panella (community): Approve
- Graham Binns (community): Approve
-
Diff: 399 lines (+94/-100)5 files modifiedHACKING.txt (+2/-2)
Makefile (+42/-38)
buildout.cfg (+48/-58)
services/pserv/run (+1/-1)
src/apiclient/testing/django.py (+1/-1)
lp://staging/~allenap/maas/packaging.rename-scripts-and-prepare-for-twistd-in-region
- Newell Jensen (community): Approve
- Graham Binns (community): Approve
-
Diff: 10 lines (+1/-1)1 file modifieddebian/maas-cluster-controller.install (+1/-1)
| Changed in maas: | |
| assignee: | nobody → Gavin Panella (allenap) |
| Changed in maas: | |
| milestone: | none → 1.6.1 |
| Changed in maas: | |
| milestone: | 1.7.0 → next |
| Changed in maas: | |
| milestone: | next → 1.7.1 |
Revision history for this message
| Andres Rodriguez (andreserl) wrote | #1 |
| Changed in maas: | |
| status: | Triaged → In Progress |
| Changed in maas: | |
| milestone: | 1.7.1 → 1.7.2 |
| Changed in maas: | |
| status: | In Progress → Fix Committed |
| Changed in maas: | |
| milestone: | 1.7.2 → next |
| Changed in maas: | |
| status: | Fix Committed → Fix Released |
| milestone: | next → none |
To post a comment you must log in.
Ok, I just came across this one in OIL.
This is really frustrating error messaging that should not be present in the 1.7.0. This leads people to believe that something very wrong is happening, when it might not be the case, and this is critical enough to be released as part of 1.7.0.
2014-11-03 09:12:24+0000 [Uninitialized] Event-loop maas-trusty-