upgrading percona cluster from ~15.04 to 17.11 fails while mistakenly attempting to rebootstrap the cluster

Bug #1746651 reported by Paul Collins
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Percona Cluster Charm
Triaged
Low
Unassigned

Bug Description

Upgrading from percona-cluster ~15.04 (r52, revision-id: <email address hidden>) to 17.11 fails as follows:

2018-02-01 03:26:12 INFO upgrade-charm * Stopping MySQL (Percona XtraDB Cluster) mysqld
2018-02-01 03:26:12 INFO upgrade-charm ...done.
2018-02-01 03:26:12 INFO upgrade-charm * Bootstrapping Percona XtraDB Cluster database server mysqld
2018-02-01 03:28:02 INFO upgrade-charm * The server quit without updating PID file (/var/run/mysqld/mysqld.pid).
2018-02-01 03:28:02 INFO upgrade-charm ...fail!
2018-02-01 03:28:02 INFO upgrade-charm Traceback (most recent call last):
[...]
2018-02-01 03:28:02 INFO upgrade-charm File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/upgrade-charm", line 221, in render_config_restart_on_changed
2018-02-01 03:28:02 INFO upgrade-charm bootstrap_pxc()
[...]
2018-02-01 03:28:02 INFO upgrade-charm File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
2018-02-01 03:28:02 INFO upgrade-charm raise child_exception
2018-02-01 03:28:02 INFO upgrade-charm OSError: [Errno 2] No such file or directory

From reading the code and comparing two deployments, it seems that this is due to is_bootstrapped() mistakenly returning False because bootstrap-uuid is not set on the cluster relation:

def is_bootstrapped():
    [...]
        for unit in units:
            id = relation_get('bootstrap-uuid', unit=unit, rid=rid)
            if id:
                uuids.append(id)
    [...]
    if uuids:
        [...]
        return True
    return False

$ juju run --unit percona-cluster/0 'relation-get -r cluster:4 bootstrap-uuid percona-cluster/0'
$ juju run --unit percona-cluster-seventeen-eleven/0 'relation-get -r cluster:16 bootstrap-uuid percona-cluster-seventeen-eleven/0 '
aabb5dad-0702-11e8-9b4d-ee83bd330150
$ _

Paul Collins (pjdc)
summary: - upgrading percona cluster from ~15.04 to 17.11 fails while attempting to
- rebootstrap the cluster
+ upgrading percona cluster from ~15.04 to 17.11 fails while mistakenly
+ attempting to rebootstrap the cluster
Paul Collins (pjdc)
tags: added: canonical-is-ps45-1711-upgrade
James Page (james-page)
tags: added: upgrade
Pen Gale (pengale)
Changed in charm-percona-cluster:
importance: Undecided → Low
status: New → Triaged
Revision history for this message
Pen Gale (pengale) wrote :

This bug has been hanging on for a while without further activity, and the is_bootstrapped routine has been updated since the bug was filed. Marking this as Low, and I think the best plan is to set aside some time to verify that the issue no longer affects us, and close.

If this is affecting anybody actively upgrading charms, please leave a comment, and we'll bump the priority.

tags: added: charm-upgrade
removed: upgrade
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.