[10.0][BVT] Deployment process stops after failure of task rabbitmq on one of the nodes
Affects | Status | Importance | Assigned to | Milestone | ||
---|---|---|---|---|---|---|
Mirantis OpenStack | Status tracked in 10.0.x | |||||
10.0.x |
Fix Committed
|
High
|
MOS Oslo | |||
9.x |
Confirmed
|
Medium
|
MOS Oslo |
Bug Description
A recent run ( https:/
2016-09-23 00:26:05 DEBUG [16227] Cluster[]: Process node: Node[3]
2016-09-23 00:26:05 DEBUG [16227] Node[3]: Node 3: task rabbitmq, task status running
2016-09-23 00:26:05 ERROR [16227] Node 3(rabbitmq) status: error
2016-09-23 00:26:05 DEBUG [16227] Node 3 has failed to deploy. There is no more retries for puppet run.
2016-09-23 00:26:05 DEBUG [16227] {"nodes"
2016-09-23 00:26:05 DEBUG [16227] Task time summary: rabbitmq with status failed on node 3 took 00:12:54
after that deployment goes to error state:
2016-09-23 00:29:45 INFO [16227] Cluster[]: All nodes are finished. Failed tasks: Task[rabbitmq/3] Stopping the deployment process!
The task itself fails with:
/usr/lib/
/usr/bin/
2016-09-23 00:24:58 +0000 /Stage[
2016-09-23 00:26:03 +0000 /Stage[
2016-09-23 00:26:03 +0000 /Stage[
2016-09-23 00:26:05 +0000 Puppet (notice): Finished catalog run in 89.28 seconds
The most probable cause is that 'mnesia' and 'rabbit' applications were stopped inside Erlang VM (or didn't have time to complete their startup) when this command failed.