Ensure-availability doesn't work after stopping non-leader

Bug #1383453 reported by Jorge Niedbalski
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
Unassigned

Bug Description

[Environment]

ii juju 1.20.10-0ubuntu1~14.04.1~juju1 all next generation service orchestration system
ii juju-core 1.20.10-0ubuntu1~14.04.1~juju1 amd64 Juju is devops distilled - client

ubuntu@niedbalski-bastion:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty

$ juju bootstrap
$ juju ensure-availability ( 3 nodes )
$juju status

environment: niedbalski
machines:
  "0":
    agent-state: started
    agent-version: 1.20.10
    dns-name: 10.5.0.4
    instance-id: 5007ffe4-3f05-44d1-8dc5-4466adbda20a
    instance-state: ACTIVE
    series: trusty
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=20480M
    state-server-member-status: removing-vote
  "1":
    agent-state: started
    agent-version: 1.20.10
    dns-name: 10.5.0.5
    instance-id: c273df36-edd6-49ff-9f02-1919cbef989e
    instance-state: ACTIVE
    series: trusty
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=20480M
    state-server-member-status: has-vote
  "2":
    agent-state: started
    agent-version: 1.20.10
    dns-name: 10.5.0.6
    instance-id: 782258d6-6955-4565-aeab-8d042b7b9b1f
    instance-state: ACTIVE
    series: trusty
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=20480M
    state-server-member-status: has-vote

$ juju ssh 1
$ sudo service juju-machine-0 stop

Agent changes state to

machines:
  "0":
    agent-state: down
    agent-state-info: (started)
    agent-version: 1.20.10
    dns-name: 10.5.0.4
    instance-id: 5007ffe4-3f05-44d1-8dc5-4466adbda20a
    instance-state: ACTIVE
    series: trusty
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=20480M
    state-server-member-status: no-vote
  "1":
    agent-state: started
    agent-version: 1.20.10
    dns-name: 10.5.0.5
    instance-id: c273df36-edd6-49ff-9f02-1919cbef989e
    instance-state: ACTIVE
    series: trusty
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=20480M
    state-server-member-status: has-vote
  "2":
    agent-state: started
    agent-version: 1.20.10
    dns-name: 10.5.0.6
    instance-id: 782258d6-6955-4565-aeab-8d042b7b9b1f
    instance-state: ACTIVE
    series: trusty
    hardware: arch=amd64 cpu-cores=1 mem=2048M root-disk=20480M
    state-server-member-status: has-vote

Then i try to add a machine or run ensure-availaiblity again to remove the death node.

$juju add-machine

All the new machines keeps in 'pending' state forever

  "15":
    instance-id: pending
    series: trusty

$ juju ensure-availability

[Expected]

If the initial state 0 node is turned off ( jujud-machine-0 stop ) then juju add-machine/ensure-availability should works correctly.

Abel Deuring (adeuring)
Changed in juju-core:
status: New → Triaged
importance: Undecided → High
tags: added: ha
Curtis Hovey (sinzui)
tags: added: cts-cloud-review
removed: cts
tags: added: cts
Curtis Hovey (sinzui)
Changed in juju-core:
importance: High → Medium
summary: - Ensure-availability doesn't works after stop state server agent
+ Ensure-availability doesn't works after stopping non-leader
summary: - Ensure-availability doesn't works after stopping non-leader
+ Ensure-availability doesn't work after stopping non-leader
tags: added: sts
removed: cts
Curtis Hovey (sinzui)
tags: added: ci
Curtis Hovey (sinzui)
tags: added: landscape
Changed in juju-core:
milestone: none → 1.26.0
importance: Medium → High
David Britton (dpb)
tags: added: kanban-cross-team
tags: removed: kanban-cross-team
Changed in juju-core:
milestone: 1.26.0 → 2.0-beta5
tags: added: sts-needs-review
Revision history for this message
Cheryl Jennings (cherylj) wrote :

Jorge - do you have any logs for this?

Also, do the other state servers continue to function?

Changed in juju-core:
milestone: 2.0-beta5 → 2.0-beta4
Curtis Hovey (sinzui)
tags: added: jujuqa
Changed in juju-core:
milestone: 2.0-beta4 → 2.0.0
tags: removed: cts-cloud-review sts-needs-review
Changed in juju-core:
assignee: nobody → Richard Harding (rharding)
assignee: Richard Harding (rharding) → nobody
Revision history for this message
Anastasia (anastasia-macmood) wrote :

@Jorge,
Could you please confirm that you are still seeing it?

Changed in juju-core:
status: Triaged → Incomplete
affects: juju-core → juju
Changed in juju:
milestone: 2.0.0 → none
milestone: none → 2.0.0
Changed in juju:
milestone: 2.0.0 → none
tags: added: eda
Revision history for this message
Anastasia (anastasia-macmood) wrote :
Changed in juju:
status: Incomplete → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related blueprints

Remote bug watches

Bug watches keep track of this bug in other bug trackers.