unexplained container restarts cause connectivity issues to rabbitmq
Bug #2016541 reported by
James Page
This bug affects 1 person
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| OpenStack Snap |
Triaged
|
High
|
Unassigned | ||
Bug Description
During operation of a long running sunbeam deployment (nearly a month now) I've noted spurious restarts of individual containers; generally the apps all deal with this fine (mysql, OVN and API services have all restarted in this way) but when the RabbitMQ container gets started, the clients panic and end up not interacting with the restarted broker once its back up and running.
a) we need to understand why the containers are getting restarted
b) something in eventlet/
| summary: |
- explained container restarts + explained container restarts cause connectivity issues to rabbitmq |
| Changed in snap-sunbeam: | |
| status: | New → Invalid |
| no longer affects: | snap-sunbeam |
| summary: |
- explained container restarts cause connectivity issues to rabbitmq + unexplained container restarts cause connectivity issues to rabbitmq |
| Changed in snap-openstack: | |
| status: | New → Triaged |
| importance: | Undecided → High |
Revision history for this message
| James Page (james-page) wrote | #1 |
Revision history for this message
| James Page (james-page) wrote | #2 |
To post a comment you must log in.
I managed to capture some K8S events during a set of container restarts; the impacted containers all had an unhealthy event related to the readyness probe failure.
Pebble fronts the probes for the services; seemed to not respond in a timely fashion which is why the container got restarted.
Note that this is not always the workload container - charm containers are also impacted.