statusbot needs better logging

Bug #1240065 reported by Anita Kuno
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Core Infrastructure

Bug Description

Statusbot goes down and the current logging is inadequate to acertain why. This situation could stand some improvement.

Revision history for this message
Anita Kuno (anteaya) wrote :

<fungi> seems we've lost gerritbot and statusbot. giving them both a swift kick

<fungi> so looks like we lost openstackgerrit to a ping timeout today at 09:38:36 (but the process is still running on the server), and openstackstatus disapeared at 10:20:51 yesterday closing its connection (process is not running at all on the server)

<anteaya> utc time?
<fungi> yeah. i'm checking their logs before i (re)start them
<anteaya> k thanks
<anteaya> are their logs accessible publicly
<anteaya> would love to look

<fungi> no, they're not. but also nothing of interest in the gerritbot logs. dug around a good 10 minutes before the timeout all the way up through when it fell off the server and no indication in the logs that it ever knew anything out of the ordinary happened
<fungi> still maintained its connection for the gerrit event stream and was receiving events and thought it was announcing them

<fungi> and statusbot's logs (even the debug log) just stop at the point where the process died

<fungi> no error, no traceback, nothing

* openstackstatus (~<email address hidden>) has joined #openstack-infra

<anteaya> funny nothing showed up in the logs
<ttx> openstackstatus seems like the bot which should always be up and it's down half the time :)
<fungi> ttx: yes. perhaps we need an openstackstatusstatus bot to announce the status of openstackstatus
<anteaya> ha ha ha
<anteaya> does openstackstatus need better logging so that when it is down, we have a clearer picture as to why?

<fungi> or someone to leave it running under a debugger sometime and catch it when it dies. i suspect it wouldn't drag down performance much given its activity leve
<fungi> l
<fungi> anteaya: yes

Changed in openstack-ci:
assignee: nobody → Anita Kuno (anteaya)
Clark Boylan (cboylan)
Changed in openstack-ci:
status: New → Triaged
importance: Undecided → Medium
milestone: none → icehouse
Anita Kuno (anteaya)
Changed in openstack-ci:
assignee: Anita Kuno (anteaya) → nobody
Jeremy Stanley (fungi)
Changed in openstack-ci:
milestone: icehouse → kilo
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.