Juju Charms Collection
nova-cloud-controller package

hacluster losing vip grp primitive following upgrade

Bug #1396246 reported by Edward Hope-Morley
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
charmhelpers (Juju Charms Collection)
Fix Released
High
Edward Hope-Morley
Juju Charms Collection 15.01
cinder (Juju Charms Collection)
Fix Released
High
Edward Hope-Morley
Juju Charms Collection 15.01
glance (Juju Charms Collection)
Fix Released
High
Edward Hope-Morley
Juju Charms Collection 15.01
keystone (Juju Charms Collection)
Fix Released
High
Edward Hope-Morley
Juju Charms Collection 15.01
neutron-api (Juju Charms Collection)
Fix Released
High
Edward Hope-Morley
Juju Charms Collection 15.01
nova-cloud-controller (Juju Charms Collection)
Fix Released
High
Edward Hope-Morley
Juju Charms Collection 15.01
openstack-dashboard (Juju Charms Collection)
Fix Released
High
Edward Hope-Morley
Juju Charms Collection 15.01
swift-proxy (Juju Charms Collection)
Fix Released
High
Edward Hope-Morley
Juju Charms Collection 15.01

Bug Description

When upgrading an existing deployment oh haclustered charms, the vip group primitive seems to get lost so the charms never find a leader (since they generally use the group primitive as a means of checking who is leader).

Tags: openstack cts

Related branches

lp://staging/~hopem/charm-helpers/retry-crm-resource-check-if-not-running
Jorge Niedbalski (community): Approve
OpenStack Charmers: Pending requested
James Page: Pending requested
Diff: 173 lines (+90/-17)
3 files modified
charmhelpers/contrib/hahelpers/cluster.py (+27/-12)
charmhelpers/core/decorators.py (+41/-0)
tests/contrib/hahelpers/test_cluster_utils.py (+22/-5)
lp://staging/~hopem/charms/trusty/cinder/charm-helpers-sync-1396246
Corey Bryant: Approve
Diff: 145 lines (+74/-16)
3 files modified
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
lp://staging/~hopem/charms/trusty/glance/charm-helpers-sync-1396246
Corey Bryant: Approve
Diff: 145 lines (+74/-16)
3 files modified
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
lp://staging/~hopem/charms/trusty/nova-cloud-controller/charm-helpers-sync-1396246
Corey Bryant: Approve
Diff: 196 lines (+117/-16)
4 files modified
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+43/-0)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
lp://staging/~hopem/charms/trusty/keystone/charm-helpers-sync-1396246
Corey Bryant: Approve
Diff: 196 lines (+117/-16)
4 files modified
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+43/-0)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
lp://staging/~hopem/charms/trusty/neutron-api/charm-helpers-sync-1396246
Corey Bryant: Approve
Diff: 196 lines (+117/-16)
4 files modified
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+43/-0)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
lp://staging/~hopem/charms/trusty/openstack-dashboard/charm-helpers-sync-1396246
Corey Bryant: Approve
Diff: 196 lines (+117/-16)
4 files modified
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+43/-0)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
lp://staging/~hopem/charms/trusty/swift-proxy/charm-helpers-sync-1396246
Corey Bryant: Approve
Diff: 196 lines (+117/-16)
4 files modified
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+43/-0)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
lp://staging/~hopem/charms/trusty/ceilometer/charm-helpers-sync-1396246
Corey Bryant: Approve
Diff: 196 lines (+117/-16)
4 files modified
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+43/-0)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
lp://staging/~hopem/charms/trusty/swift-storage/charm-helpers-sync-1396246
Corey Bryant: Approve
Diff: 196 lines (+117/-16)
4 files modified
hooks/charmhelpers/contrib/hahelpers/cluster.py (+26/-12)
hooks/charmhelpers/contrib/storage/linux/ceph.py (+43/-0)
hooks/charmhelpers/core/decorators.py (+41/-0)
hooks/charmhelpers/core/host.py (+7/-4)
Revision history for this message
Edward Hope-Morley (hopem) wrote :

I don;t have any further info right now so will have to see if I can reproduced this before confirming.

Revision history for this message
Edward Hope-Morley (hopem) wrote :

OK I think i see the problem here. The logic in the charms for is_elected_leader goes:

is clustered? and does grp resource exist?

If False then not leader. The problem here is that if you reboot the entire cluster there will be a short window where each unit will have:

is_clustered=True
grp_resource_exist=False

So a simple workaround here would be to allow for a re-query in the case where we are clustered but the grp resource does not yet exist since if all units fail to elect a leader the cluster will likely fail in all cases.

Changed in hacluster (Juju Charms Collection):
assignee: nobody → Edward Hope-Morley (hopem)
importance: Undecided → High
status: New → In Progress
Changed in charmhelpers (Juju Charms Collection):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Edward Hope-Morley (hopem)
Edward Hope-Morley (hopem)
Changed in charmhelpers (Juju Charms Collection):
status: In Progress → Fix Committed
affects: hacluster (Juju Charms Collection) → nova-cloud-controller (Juju Charms Collection)
affects: quantum-gateway (Juju Charms Collection) → cinder (Juju Charms Collection)
Changed in nova-cloud-controller (Juju Charms Collection):
status: In Progress → New
Changed in cinder (Juju Charms Collection):
importance: Undecided → High
Changed in keystone (Juju Charms Collection):
importance: Undecided → High
Changed in openstack-dashboard (Juju Charms Collection):
importance: Undecided → High
Changed in swift-proxy (Juju Charms Collection):
importance: Undecided → High
Changed in nova-cloud-controller (Juju Charms Collection):
assignee: Edward Hope-Morley (hopem) → nobody
affects: glance (Ubuntu) → glance (Juju Charms Collection)
Changed in glance (Juju Charms Collection):
importance: Undecided → High
Changed in neutron-api (Juju Charms Collection):
importance: Undecided → High
Edward Hope-Morley (hopem)
Changed in cinder (Juju Charms Collection):
status: New → In Progress
Changed in glance (Juju Charms Collection):
status: New → In Progress
Changed in keystone (Juju Charms Collection):
status: New → In Progress
Changed in neutron-api (Juju Charms Collection):
status: New → In Progress
Changed in nova-cloud-controller (Juju Charms Collection):
status: New → In Progress
Changed in openstack-dashboard (Juju Charms Collection):
status: New → In Progress
Changed in swift-proxy (Juju Charms Collection):
status: New → In Progress
Changed in cinder (Juju Charms Collection):
assignee: nobody → Edward Hope-Morley (hopem)
Changed in glance (Juju Charms Collection):
assignee: nobody → Edward Hope-Morley (hopem)
Changed in neutron-api (Juju Charms Collection):
assignee: nobody → Edward Hope-Morley (hopem)
Changed in openstack-dashboard (Juju Charms Collection):
assignee: nobody → Edward Hope-Morley (hopem)
Changed in swift-proxy (Juju Charms Collection):
assignee: nobody → Edward Hope-Morley (hopem)
Changed in nova-cloud-controller (Juju Charms Collection):
assignee: nobody → Edward Hope-Morley (hopem)
Changed in keystone (Juju Charms Collection):
assignee: nobody → Edward Hope-Morley (hopem)
Edward Hope-Morley (hopem)
Changed in cinder (Juju Charms Collection):
status: In Progress → Fix Committed
Changed in glance (Juju Charms Collection):
status: In Progress → Fix Committed
Changed in keystone (Juju Charms Collection):
status: In Progress → Fix Committed
Changed in neutron-api (Juju Charms Collection):
status: In Progress → Fix Committed
Changed in nova-cloud-controller (Juju Charms Collection):
status: In Progress → Fix Committed
Changed in openstack-dashboard (Juju Charms Collection):
status: In Progress → Fix Committed
Changed in swift-proxy (Juju Charms Collection):
status: In Progress → Fix Committed
James Page (james-page)
Changed in nova-cloud-controller (Juju Charms Collection):
milestone: none → 15.01
Changed in charmhelpers (Juju Charms Collection):
milestone: none → 15.01
Changed in swift-proxy (Juju Charms Collection):
milestone: none → 15.01
Changed in keystone (Juju Charms Collection):
milestone: none → 15.01
Changed in openstack-dashboard (Juju Charms Collection):
milestone: none → 15.01
Changed in cinder (Juju Charms Collection):
milestone: none → 15.01
Changed in glance (Juju Charms Collection):
milestone: none → 15.01
Changed in neutron-api (Juju Charms Collection):
milestone: none → 15.01
James Page (james-page)
Changed in nova-cloud-controller (Juju Charms Collection):
status: Fix Committed → Fix Released
Changed in charmhelpers (Juju Charms Collection):
status: Fix Committed → Fix Released
Changed in swift-proxy (Juju Charms Collection):
status: Fix Committed → Fix Released
Changed in keystone (Juju Charms Collection):
status: Fix Committed → Fix Released
Changed in openstack-dashboard (Juju Charms Collection):
status: Fix Committed → Fix Released
Changed in cinder (Juju Charms Collection):
status: Fix Committed → Fix Released
Changed in glance (Juju Charms Collection):
status: Fix Committed → Fix Released
Changed in neutron-api (Juju Charms Collection):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.