Rabbitqm repeatedly stopped unexpectedly with code 0

Bug #2033917 reported by Bas de Bruijne

This bug report was marked for expiration 232 days ago. (find out why)

6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Snap
Incomplete
Medium
Unassigned

Bug Description

In test run https://solutions.qa.canonical.com/testruns/36d438ef-d53a-4801-80df-a85597114280/ (microstack 2023.1 candidate, jammym, singe-node), the test run fails because the rabbitmq unit is stuck waiting with status "Waiting for leader to create operator user". In the rabbitmq pod logs we see the following message repeated:

================
2023-08-31T18:15:53.538Z [pebble] Service "rabbitmq" stopped unexpectedly with code 0
2023-08-31T18:15:53.538Z [pebble] Service "rabbitmq" on-success action is "restart", waiting ~500ms before restart (backoff 1)
2023-08-31T18:15:54.079Z [pebble] Service "rabbitmq" starting: rabbitmq-server
2023-08-31T18:15:55.363Z [epmd] epmd: Thu Aug 31 18:15:55 2023: ** got ALIVE2_REQ
2023-08-31T18:15:55.363Z [epmd] epmd: Thu Aug 31 18:15:55 2023: registering 'epmd-starter-167649463:1693504589', port 34673
2023-08-31T18:15:55.363Z [epmd] epmd: Thu Aug 31 18:15:55 2023: type 77 proto 0 highvsn 6 lowvsn 5
2023-08-31T18:15:55.363Z [epmd] epmd: Thu Aug 31 18:15:55 2023: ** sent ALIVE2_RESP for "epmd-starter-167649463"
2023-08-31T18:15:55.376Z [epmd] epmd: Thu Aug 31 18:15:55 2023: unregistering 'epmd-starter-167649463:1693504589', port 34673
2023-08-31T18:15:55.415Z [epmd] epmd: Thu Aug 31 18:15:55 2023: ** got ALIVE2_REQ
2023-08-31T18:15:55.415Z [epmd] epmd: Thu Aug 31 18:15:55 2023: registering 'rabbit_prelaunch_1036:1693504979', port 25672
2023-08-31T18:15:55.415Z [epmd] epmd: Thu Aug 31 18:15:55 2023: type 77 proto 0 highvsn 6 lowvsn 5
2023-08-31T18:15:55.416Z [epmd] epmd: Thu Aug 31 18:15:55 2023: ** sent ALIVE2_RESP for "rabbit_prelaunch_1036"
2023-08-31T18:15:55.423Z [epmd] epmd: Thu Aug 31 18:15:55 2023: ** got NAMES_REQ
2023-08-31T18:15:55.424Z [epmd] epmd: Thu Aug 31 18:15:55 2023: ** sent NAMES_RESP
2023-08-31T18:15:55.424Z [epmd] epmd: Thu Aug 31 18:15:55 2023: unregistering 'rabbit_prelaunch_1036:1693504979', port 25672
2023-08-31T18:15:55.425Z [epmd] epmd: Thu Aug 31 18:15:55 2023: ** got ALIVE2_REQ
2023-08-31T18:15:55.425Z [epmd] epmd: Thu Aug 31 18:15:55 2023: reusing slot with same name 'rabbit'
2023-08-31T18:15:55.425Z [epmd] epmd: Thu Aug 31 18:15:55 2023: registering 'rabbit:1693504592', port 25672
2023-08-31T18:15:55.425Z [epmd] epmd: Thu Aug 31 18:15:55 2023: type 77 proto 0 highvsn 6 lowvsn 5
2023-08-31T18:15:55.425Z [epmd] epmd: Thu Aug 31 18:15:55 2023: ** sent ALIVE2_RESP for "rabbit"
2023-08-31T18:22:20.030Z [epmd] epmd: Thu Aug 31 18:22:20 2023: unregistering 'rabbit:1693504592', port 25672
2023-08-31T18:22:21.758Z [pebble] Service "rabbitmq" stopped unexpectedly with code 0
2023-08-31T18:22:21.758Z [pebble] Service "rabbitmq" on-success action is "restart", waiting ~500ms before restart (backoff 1)
2023-08-31T18:22:22.267Z [pebble] Service "rabbitmq" starting: rabbitmq-server
================

I haven't found the rabbitmq logs, so I can't dig much deeper.

Logs and configs can be found here: https://oil-jenkins.canonical.com/artifacts/36d438ef-d53a-4801-80df-a85597114280/index.html

Revision history for this message
Bas de Bruijne (basdbruijne) wrote :

This test run has the same, but during the rabbitmq installation: https://solutions.qa.canonical.com/testruns/13cf8cca-08a2-48cd-9c1f-0d4af3a8a1b3/

Revision history for this message
James Page (james-page) wrote :

RMQ is def not happy - I think its logs are still going to a disk logfile - I'll dig and have a look.

Without those a little tricky to triage.

Revision history for this message
James Page (james-page) wrote :

3.9/candidate has fixed logging so we should be able to see exactly why RMQ does not start.

Please re-test with this version and report back!

Changed in snap-openstack:
status: New → Incomplete
importance: Undecided → Medium
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.