[17.08+][ocata+] Unit has peers, but RabbitMQ not clustered
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack RabbitMQ Server Charm |
Confirmed
|
Low
|
Unassigned |
Bug Description
On a fresh 17.08 ocata deployment rabbitmq server used to clusterize just fine but I was able to get to a state where is does not.
All units "see each other" via relation data but a cluster is not formed. Update status sets an appropriate status message: "Unit has peers, but RabbitMQ not clustered"
http://
Looking at .unit-state.db for rabbitmq-server/2 which is a leader unit I do not see initial_
sqlite> select * from kv;
charm_revisions
env|{"CHARM_DIR": "/var/lib/
unit|"rabbitmq-
relid|null
# sqlite> select * from hooks;
# for rabbitmq-server/2
http://
ubuntu@
Cluster status of node 'rabbit@
[{nodes,
{running_
{cluster_
{partitions,[]}]
summary: |
- [17.08][ocata] Unit has peers, but RabbitMQ not clustered + [17.08+][ocata+] Unit has peers, but RabbitMQ not clustered |
tags: |
added: foundations-engine removed: cpe-foundations |
Changed in charm-rabbitmq-server: | |
status: | New → Confirmed |
Changed in charm-rabbitmq-server: | |
importance: | Undecided → Low |
After redeploying rabbitmq-server it clusterized just fine:
juju remove-application rabbitmq-server
cat ../rabbitmq.yaml partition- handling: pause_minority
rabbitmq-server:
source: cloud:xenial-ocata
min-cluster-size: 3
cluster-
juju deploy rabbitmq-server --config ../rabbitmq.yaml -n 3 --to lxd:0,lxd:1,lxd:12 --bind "space-oam amqp=space-oam cluster=space-oam"
juju s rabbitmq-server
Unit Workload Agent Machine Public address Ports Message
rabbitmq-server/3* active idle 0/lxd/9 10.30.21.164 5672/tcp Unit is ready and clustered
rabbitmq-server/4 active idle 1/lxd/9 10.30.21.104 5672/tcp Unit is ready and clustered
rabbitmq-server/5 active idle 12/lxd/9 10.30.21.119 5672/tcp Unit is ready and clustered
Machine State DNS Inst id Series AZ Message 12-lxd- 9 xenial z03 Container started
0 started 103.77.105.129 7nc46s xenial z01 Deployed
0/lxd/9 started 10.30.21.164 juju-ada6ad-0-lxd-9 xenial z01 Container started
1 started 103.77.105.130 hxgx7c xenial z02 Deployed
1/lxd/9 started 10.30.21.104 juju-ada6ad-1-lxd-9 xenial z02 Container started
12 started 103.77.105.132 prbbg4 xenial z03 Deployed
12/lxd/9 started 10.30.21.119 juju-ada6ad-
initial_ client_ update_ done is now set to true as it should be:
root@juju- ada6ad- 0-lxd-9: /var/lib/ juju/agents/ unit-rabbitmq- server- 3/charm# sqlite3 .unit-state.db
sqlite> select * from kv; |["150" ] MACHINE_ ID": "0/lxd/9", "NO_PROXY": "127.0. 0.1,::1, localhost" , "JUJU_CHARM_DIR": "/var/lib/ juju/agents/ unit-rabbitmq- server- 3/charm" , "JUJU_API_ ADDRESSES" : "10.30. 20.42:17070" , "JUJU_CONTEXT_ID": "rabbitmq- server/ 3-cluster- relation- changed- 359294288848534 2330", "PATH": "/var/lib/ juju/tools/ unit-rabbitmq- server- 3:/usr/ local/sbin: /usr/local/ bin:/usr/ sbin:/usr/ bin:/sbin: /bin", "JUJU_MODEL_UUID": "8580a94a- 5a38-44db- 8375-98e4a3ada6 ad", "JUJU_METER_INFO": "not set", "JUJU_AGENT_ SOCKET" : "@/var/ lib/juju/ agents/ unit-rabbitmq- server- 3/agent. socket" , "no_proxy": "127.0. 0.1,::1, localhost" , "JUJU_METER_ STATUS" : "AMBER", "DEBIAN_FRONTEND": "noninteractive", "JUJU_UNIT_NAME": "rabbitmq- server/ 3", "JUJU_RELATION": "cluster", "JUJU_AVAILABIL ITY_ZONE" : "z01", "JUJU_REMOTE_UNIT": "rabbitmq- server/ 4", "JUJU_PRINCIPAL _UNIT": "", "APT_LISTCHANGE S_FRONTEND" : "none", "CHARM_DIR": "/var/lib/ juju/agents/ unit-rabbitmq- server- 3/charm" , "JUJU_SLA": "unsupported", "JUJU_MODEL_NAME": "openstack", "JUJU_RELATION_ID": "cluster:228"} server/ 3" client_ update_ done|true
charm_revisions
env|{"JUJU_
unit|"rabbitmq-
relid|"cluster:228"
initial_