Comment 0 for bug 1940627

Revision history for this message
Radu Malica (radumalica) wrote : Masakari starting fail after upgrade to Wallaby

I am running openstack version Victoria on Focal Ubuntu deployed with Juju charms and i wanted to upgrade to Wallaby

Everything worked fine by the book until i ran the upgrade on Masakari app with the following commands:

juju config masakari action-managed-upgrade=True
juju config masakari openstack-origin=cloud:focal-wallaby

The Masakari stack is having 3 instances with HA cluster deployed. The HA cluster upgraded successfully

masakari/6 error idle 4/lxd/4 10.118.0.138 15868/tcp hook failed: "start"
  masakari-hacluster/3 active idle 10.118.0.138 Unit is ready and clustered
masakari/7 error idle 5/lxd/1 10.118.0.137 15868/tcp hook failed: "start"
  masakari-hacluster/5 active idle 10.118.0.137 Unit is ready and clustered
masakari/8* error idle 3/lxd/6 10.118.0.136 15868/tcp hook failed: "start"
  masakari-hacluster/4* active idle 10.118.0.136 Unit is ready and clustered

From the logs on each instances i can see this error:

unit-masakari-7: 07:24:57 ERROR unit.masakari/7.juju-log Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-masakari-7/charm/reactive/masakari_handlers.py", line 40, in render_config
    charm_class.upgrade_if_available(args)
  File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms_openstack/charm/core.py", line 1142, in upgrade_if_available
    if self.openstack_upgrade_available(self.release_pkg):
  File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms_openstack/charm/core.py", line 1109, in openstack_upgrade_available
    raise RuntimeError(
RuntimeError: In charms_openstack.charm.core.openstack_upgrade_available() cur_vers=None and avail_vers=2021.1, one of which is None. This usually implies that the openstack version is not present in the self.package_codenames or os_utils.PACKAGE_CODENAMES. Please re-visit and fix.

unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start Traceback (most recent call last):
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start File "/var/lib/juju/agents/unit-masakari-7/charm/hooks/start", line 22, in <module>
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start main()
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms/reactive/__init__.py", line 74, in main
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start bus.dispatch(restricted=restricted_mode)
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 390, in dispatch
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start _invoke(other_handlers)
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 359, in _invoke
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start handler.invoke()
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms/reactive/bus.py", line 181, in invoke
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start self._action(*args)
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start File "/var/lib/juju/agents/unit-masakari-7/charm/reactive/masakari_handlers.py", line 40, in render_config
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start charm_class.upgrade_if_available(args)
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms_openstack/charm/core.py", line 1142, in upgrade_if_available
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start if self.openstack_upgrade_available(self.release_pkg):
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start File "/var/lib/juju/agents/unit-masakari-7/.venv/lib/python3.8/site-packages/charms_openstack/charm/core.py", line 1109, in openstack_upgrade_available
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start raise RuntimeError(
unit-masakari-7: 07:24:57 WARNING unit.masakari/7.start RuntimeError: In charms_openstack.charm.core.openstack_upgrade_available() cur_vers=None and avail_vers=2021.1, one of which is None. This usually implies that the openstack version is not present in the self.package_codenames or os_utils.PACKAGE_CODENAMES. Please re-visit and fix.
unit-masakari-7: 07:24:58 ERROR juju.worker.uniter.operation hook "start" (via explicit, bespoke hook script) failed: exit status 1

It seems the charm is unable to get the openstack release from the OS, which is the same LXD containers with Focal as all other openstack components.

From the error above i am assuming that Masakari didn't even upgrade to Wallaby, since the script cannot find the cur_vers of the Openstack (which was Victoria and running properly)

Thanks