swift-storage charm action-managed-upgrade may upgrade other UCA packages for smooshed applications

Bug #1817387 reported by Drew Freiberger
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Swift Storage Charm
Triaged
Medium
Unassigned

Bug Description

I have an environment where I have the following on a metal:

ceph-osd
swift-storage
nova-compute

I started at xenial-pike and upgraded to xenial-queens on ceph-osd through the action-managed-upgrade process, however ran into no UCA archive change due to lp1778823.

I then moved on to swift-storage. I had to workaround a swift-storage bug lp#1817384 to "force" the upgrade.

When completed, I checked my juju model status and found nova-compute complaining about nova-compute service not running.

Connecting to the host, I get the log ERROR oslo_service.service RemoteError: Remote error: IncompatibleObjectVersion Version 1.4 of MigrationList is not supported in the nova-compute.log

nova-cloud-controller, neutron-api, etc are still running under pike versions, which is why they are crashing, as I'm still in group 2 of my upgrade per https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/app-upgrade-openstack.html#upgrade-order

This is forcing my hand into a very rushed upgrade since nova-compute is supposed to be group 4 upgraded.

I believe it's reasonable to expect that ceph-osd, nova-compute, and swift-storage could all be smooshed on the same metal and interactions of these three apps on the same unit doing openstack release upgrades would be very helpful.

Revision history for this message
Drew Freiberger (afreiberger) wrote :

Something I've noted, we've not run into this in the past as nova-compute has been in our personal "order of upgrade" to be upgraded before swift-storage.

Here's the classic Bootstack ordering we've used in the past. As you can see, nova-compute updating n-ovs and ceilometer-agent was well known in the past, so I believe nova-compute may have the same issue as just was witnessed with swift-storage when smooshed/co-located on the same metal.

    Ceph-osd
    Ceph-mon
    ceph-radosgw
    Keystone
    Glance
    Nova-cloud-controller
    Neutron-api
    Nova-compute
    Neutron-openvswitch (is automatically pulled in while nova-compute updates)
    Neutron-gateway
    Cinder
    Swift-proxy
    Swift-storage
    Openstack-dashboard
    Heat
    Ceilometer
    Gnocchi
    Ceilometer-agent (is pulled in during the nova-compute update)

Revision history for this message
James Page (james-page) wrote :

ceph-* is definitely selective in when an upgrade is triggered.

swift-* and nova-compute will perform dist-upgrades on the units so side-effect upgrades are unavoidable with the current upgrade approach which means that order for converged deploys needs to be re-considered.

Revision history for this message
James Page (james-page) wrote :

It might be possible to switch the swift packages to perform a selective upgrade, rather than a dist-upgrade but that will need validating; the swift project consumes less shared openstack libraries so being in a transitional state may be OK.

Changed in charm-swift-storage:
status: New → Triaged
importance: Undecided → Medium
milestone: none → 19.04
David Ames (thedac)
Changed in charm-swift-storage:
milestone: 19.04 → 19.07
David Ames (thedac)
Changed in charm-swift-storage:
milestone: 19.07 → 19.10
David Ames (thedac)
Changed in charm-swift-storage:
milestone: 19.10 → 20.01
tags: added: openstack-upgrade
James Page (james-page)
Changed in charm-swift-storage:
milestone: 20.01 → 20.05
David Ames (thedac)
Changed in charm-swift-storage:
milestone: 20.05 → 20.08
James Page (james-page)
Changed in charm-swift-storage:
milestone: 20.08 → none
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.