kolla-ansible

The extended control node is rabbitmq error reporting

Series ocata
Bug #1731142

Bug #1731142 reported by LiKui on 2017-11-09

This bug affects 5 people

	Status	Importance	Assigned to
kolla-ansible	Invalid	Undecided	Unassigned
Ocata	Confirmed	Low	Unassigned
Pike	Confirmed	Low	Unassigned
Queens	Confirmed	Low	Unassigned
Rocky	Fix Released	Low	Unassigned

Bug Description

Rabbitmq exception when extending a single control node to a 3 control node
On the extended two control nodes, the following is shown
#docker logs -f rabbitmq
.......
=INFO REPORT==== 8-Nov-2017::21:43:22 ===
Multiple different configurations with equal version numbers detected. Shutting down.

Revision history for this message

Huikai Ran (huikai.ran) wrote on 2018-01-22:

I have the same issue. with one rabbitmq, it works fine, after I deployed other two rabbitmq, the other two cannot startup, keeping restarting.

Revision history for this message

doubor (doubor) wrote on 2018-04-07:

I have the same issue.

Revision history for this message

Florian Faltermeier (florianfa) wrote on 2019-02-15:

Same issue on rocky:
=INFO REPORT==== 15-Feb-2019::14:21:27 ===
Multiple different configurations with equal version numbers detected. Shutting down.

Revision history for this message

Michal Nasiadka (mnasiadka) wrote on 2019-03-28:

Currently a workaround is to manually bump up the config version.
TODO: test on Stein (if new clusterer setup is fixing that)

Changed in kolla-ansible:
status:	New → Confirmed
importance:	Undecided → Low

Revision history for this message

Grzegorz Sikorski (gregors) wrote on 2020-03-17:

0001-rabbitmq-update-cluster-version-on-reconfigure.patch Edit (2.0 KiB, text/plain)

Please see attached patch file. This solves the issue for us.

Revision history for this message

Radosław Piliszek (yoctozepto) wrote on 2020-03-17:

Well, we can't really propose to raise the cluster version with each run.

I haven't seen this issue since Stein - could it be gone?

no longer affects:	kolla-ansible/stein
Changed in kolla-ansible:
status:	Confirmed → Invalid
importance:	Low → Undecided

Revision history for this message

Grzegorz Sikorski (gregors) wrote on 2020-03-18:

Possibly, we are using Rocky and Ocata, haven't yet tested with further version. Let me try to refactor this piece to be idempotent.

Revision history for this message

Grzegorz Sikorski (gregors) wrote on 2020-03-18:

0001-rabbitmq-update-cluster-version-on-reconfigure.patch Edit (2.9 KiB, text/plain)

Now bumping of the version only occurs if anything in the configs is modified.

Revision history for this message

Radosław Piliszek (yoctozepto) wrote on 2020-03-18:

Let's say I'm not entirely convinced you got it to work considering it has a typo: "cluser_version" in template. Otherwise looks pretty legit. Please propose to gerrit.

Revision history for this message

Grzegorz Sikorski (gregors) wrote on 2020-03-18:

#10

Sorry, the typo was corrected and tested afterwards. I must have done something stupid with git.

Revision history for this message

Radosław Piliszek (yoctozepto) wrote on 2020-03-18:

#11

All right, see https://docs.openstack.org/kolla/latest/contributor/CONTRIBUTING.html

Revision history for this message

Grzegorz Sikorski (gregors) wrote on 2020-03-18:

#12

OK, I'll push this to Gerrit as soon as some legal stuff pass here :D

Revision history for this message

Grzegorz Sikorski (gregors) wrote on 2020-03-18:

#13

https://review.opendev.org/713628

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-04-05: Fix merged to kolla-ansible (stable/rocky)

#14

Reviewed: https://review.opendev.org/713628
Committed: https://git.openstack.org/cgit/openstack/kolla-ansible/commit/?id=7408d499799e4d3a043f5e5ebf5f61c206bc0d2e
Submitter: Zuul
Branch: stable/rocky

commit 7408d499799e4d3a043f5e5ebf5f61c206bc0d2e
Author: Grzegorz Sikorski <email address hidden>
Date: Tue Mar 17 09:07:03 2020 +0100

rabbitmq: update cluster version on reconfigure

    This fixes https://bugs.launchpad.net/kolla-ansible/+bug/1731142. The
    solution is to bump a version number of the cluster configuration
    file (/etc/kolla/rabbitmq/rabbitmq-clusterer.config) every time
    RabbitMQ configuration files change.

Closes-bug: #1731142
Change-Id: I26cce92e52b2ba6b905658e11d794c65e361f545

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-05-04: Change abandoned on kolla-ansible (stable/ocata)

#15

Change abandoned by Grzegorz Sikorski (<email address hidden>) on branch: stable/ocata
Review: https://review.opendev.org/713878