Fuel for OpenStack

Ceph is being updated during patching from 5.0 to 5.0.2

Bug #1363983 reported by Vladimir Kuklin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
Critical
Dmitry Pyzhov
Fuel for OpenStack 5.0.2

Bug Description

In 5.0 release we had ceph 0.67.8. In 5.0.1 and 5.0.2 we have 0.67.9 version. Because of this we may have the problem when ceph packages are updated during OpenStack patching.

There are two options:

1) we need to split environment versions into two:

  a) 5.0.2.0 with old ceph and ceph package version not specified in *-versions.yaml
  b) 5.0.2.1 with new ceph

2) we need to ensure that minor release update of ceph does not break anything

Dmitry Pyzhov (dpyzhov)
Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Dmitry Pyzhov (lux-place)
Revision history for this message
Roman Vyalov (r0mikiam) wrote :

rpm
ceph-0.67.9-16.g69a99e6.mira1.x86_64.rpm
ceph-deploy-1.2.7-0.mira.1.noarch.rpm
ceph-radosgw-0.67.9-16.g69a99e6.mira1.x86_64.rpm
ceph-test-0.67.9-16.g69a99e6.mira1.x86_64.rpm
python-ceph-0.67.9-16.g69a99e6.mira1.x86_64.rpm

deb
ceph-common_0.67.9-ubuntu3_amd64.deb
ceph-deploy_1.2.7-ubuntu1_all.deb
ceph-fs-common_0.67.9-ubuntu3_amd64.deb
ceph-fuse_0.67.9-ubuntu3_amd64.deb
ceph-mds_0.67.9-ubuntu3_amd64.deb
ceph_0.67.9-ubuntu3_amd64.deb
python-ceph_0.67.9-ubuntu3_amd64.deb

Revision history for this message
Roman Vyalov (r0mikiam) wrote :

DEB:
ceph-common-dbg_0.67.9-ubuntu3_amd64.deb
ceph-common_0.67.9-ubuntu3_amd64.deb
ceph-dbg_0.67.9-ubuntu3_amd64.deb
ceph-fs-common-dbg_0.67.9-ubuntu3_amd64.deb
ceph-fs-common_0.67.9-ubuntu3_amd64.deb
ceph-fuse-dbg_0.67.9-ubuntu3_amd64.deb
ceph-fuse_0.67.9-ubuntu3_amd64.deb
ceph-mds-dbg_0.67.9-ubuntu3_amd64.deb
ceph-mds_0.67.9-ubuntu3_amd64.deb
ceph-resource-agents_0.67.9-ubuntu3_amd64.deb
ceph-test-dbg_0.67.9-ubuntu3_amd64.deb
ceph-test_0.67.9-ubuntu3_amd64.deb
ceph_0.67.9-ubuntu3_amd64.deb
libcephfs-dev_0.67.9-ubuntu3_amd64.deb
libcephfs-java_0.67.9-ubuntu3_all.deb
libcephfs-jni_0.67.9-ubuntu3_amd64.deb
libcephfs1-dbg_0.67.9-ubuntu3_amd64.deb
libcephfs1_0.67.9-ubuntu3_amd64.deb
librados-dev_0.67.9-ubuntu3_amd64.deb
librados2-dbg_0.67.9-ubuntu3_amd64.deb
librados2_0.67.9-ubuntu3_amd64.deb
librbd-dev_0.67.9-ubuntu3_amd64.deb
librbd1-dbg_0.67.9-ubuntu3_amd64.deb
librbd1_0.67.9-ubuntu3_amd64.deb
python-ceph_0.67.9-ubuntu3_amd64.deb
radosgw-dbg_0.67.9-ubuntu3_amd64.deb
radosgw_0.67.9-ubuntu3_amd64.deb
rbd-fuse-dbg_0.67.9-ubuntu3_amd64.deb
rbd-fuse_0.67.9-ubuntu3_amd64.deb
rest-bench-dbg_0.67.9-ubuntu3_amd64.deb
rest-bench_0.67.9-ubuntu3_amd64.deb:

RPM:
ceph-0.67.9-16.g69a99e6.mira1.src.rpm
ceph-0.67.9-16.g69a99e6.mira1.x86_64.rpm
ceph-debuginfo-0.67.9-16.g69a99e6.mira1.x86_64.rpm
ceph-devel-0.67.9-16.g69a99e6.mira1.x86_64.rpm
ceph-fuse-0.67.9-16.g69a99e6.mira1.x86_64.rpm
ceph-radosgw-0.67.9-16.g69a99e6.mira1.x86_64.rpm
ceph-test-0.67.9-16.g69a99e6.mira1.x86_64.rpm
cephfs-java-0.67.9-16.g69a99e6.mira1.x86_64.rpm
libcephfs1-0.67.9-16.g69a99e6.mira1.x86_64.rpm
libcephfs_jni1-0.67.9-16.g69a99e6.mira1.x86_64.rpm
librados2-0.67.9-16.g69a99e6.mira1.x86_64.rpm
librbd1-0.67.9-16.g69a99e6.mira1.x86_64.rpm
python-ceph-0.67.9-16.g69a99e6.mira1.x86_64.rpm
rbd-fuse-0.67.9-16.g69a99e6.mira1.x86_64.rpm
rest-bench-0.67.9-16.g69a99e6.mira1.x86_64.rpm

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-main (master)

Fix proposed to branch: master
Review: https://review.openstack.org/118184

Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Vladimir Kuklin (vkuklin) wrote :

ceph nodes cannot be safely updated in 5.1 release

we need to write down to release notes that user, if he wants should update ceph manually

for nodes containing monitors he should install new packages and restart monitors

once all the monitors are restarted, he should update code on the OSD nodes one by one waiting for OSD nodes to rebuild cleanly after OSD restart. for this to happen, user should check `ceph pg dump` command output for absence of non-'active+clean' states.

Also, user should reconsider ceph update if there are already unclean pgs in ceph pg dump output.

tags: added: release-notes
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-main (stable/5.0)

Fix proposed to branch: stable/5.0
Review: https://review.openstack.org/118377

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-main (master)

Reviewed: https://review.openstack.org/118184
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=7b5e90893b602d4788ac955f1bc3f7a613766c41
Submitter: Jenkins
Branch: master

commit 7b5e90893b602d4788ac955f1bc3f7a613766c41
Author: Dmitry Pyzhov <email address hidden>
Date: Mon Sep 1 19:43:49 2014 +0400

    Ignore ceph packages in patching

    Blacklist for ceph packages should be considered as a workaround.
    We should create a solution for pre-moderated whitelist of approved
    packages for patching.

    Change-Id: I05ff7eb67efc9e6bbf3813c4ef8ce0774d766112
    Closes-Bug: #1363983

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-main (stable/5.0)

Reviewed: https://review.openstack.org/118377
Committed: https://git.openstack.org/cgit/stackforge/fuel-main/commit/?id=b1e46c8c3ac86a8eafac225507294226322160ea
Submitter: Jenkins
Branch: stable/5.0

commit b1e46c8c3ac86a8eafac225507294226322160ea
Author: Dmitry Pyzhov <email address hidden>
Date: Mon Sep 1 19:43:49 2014 +0400

    Ignore ceph packages in patching

    Blacklist for ceph packages should be considered as a workaround.
    We should create a solution for pre-moderated whitelist of approved
    packages for patching.

    Change-Id: I05ff7eb67efc9e6bbf3813c4ef8ce0774d766112
    Closes-Bug: #1363983

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.