Comment 6 for bug 1500204

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

From the txz. Units 0 and 1 finish installing and step back because the cluster is not yet complete. Last messages from both:

unit-rabbitmq-server-1[1068]: 2015-09-26 19:37:23 INFO unit.rabbitmq-server/1.juju-log server.go:254 cluster:1: cluster_joined: cookie not yet set.
unit-rabbitmq-server-0[969]: 2015-09-26 19:37:25 INFO unit.rabbitmq-server/0.juju-log server.go:254 cluster:1: cluster_joined: cookie not yet set.

Then unit 2 comes along, installs the packages and creates the same basic vhosts and users as the previous two units, but fails for no apparent reason:
(...)
unit-rabbitmq-server-2[1056]: 2015-09-26 19:39:02 INFO unit.rabbitmq-server/2.config-changed logger.go:40 Creating vhost "nagios-rabbitmq-server-2" ...
unit-rabbitmq-server-2[1056]: 2015-09-26 19:39:02 INFO unit.rabbitmq-server/2.config-changed logger.go:40 ...done.
unit-rabbitmq-server-2[1056]: 2015-09-26 19:39:02 INFO unit.rabbitmq-server/2.juju-log server.go:254 Created new vhost (nagios-rabbitmq-server-2).
unit-rabbitmq-server-2[1056]: 2015-09-26 19:39:10 INFO unit.rabbitmq-server/2.config-changed logger.go:40 Error: unable to connect to node 'rabbit@10-20-0-99': nodedown
...
unit-rabbitmq-server-2[1056]: 2015-09-26 19:39:10 INFO unit.rabbitmq-server/2.config-changed logger.go:40 subprocess.CalledProcessError: Command '['/usr/sbin/rabbitmqctl', 'list_users']' returned non-zero exit status 2

It's as if the rabbit app was shutdown mid-deploy, or the client somehow failed to reach the server even when it was up.

The rabbit logs around the same time, on rabbitmq-server/2, show a startup at 19:38:56, and confirm the successful creation of the nagios-rabbitmq-server-2 vhost:
=INFO REPORT==== 26-Sep-2015::19:38:56 ===
started TCP Listener on [::]:5672

=INFO REPORT==== 26-Sep-2015::19:38:56 ===
Server startup complete; 0 plugins started.

=INFO REPORT==== 26-Sep-2015::19:39:02 ===
Adding vhost 'nagios-rabbitmq-server-2'

That's where the rabbit log from unit 2 ends. Nothing indicates that the rabbit service died.