[ceph verifiers] explore a warning rather than failure if ceph-mon returns HEALTH_WARN

Bug #1949600 reported by Alvaro Uria
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
juju-verify
Triaged
Medium
Unassigned

Bug Description

I ran into a Ceph cluster with the following state:
"""
root@juju-733f2f-3-lxd-1:~# ceph health detail
HEALTH_WARN client is using insecure global_id reclaim; mons are allowing insecure global_id reclaim
[WRN] AUTH_INSECURE_GLOBAL_ID_RECLAIM: client is using insecure global_id reclaim
    client.cinder-ceph-hdd at 10.130.21.15:0/133926252 is using insecure global_id reclaim
[WRN] AUTH_INSECURE_GLOBAL_ID_RECLAIM_ALLOWED: mons are allowing insecure global_id reclaim
    mon.juju-733f2f-3-lxd-1 has auth_allow_insecure_global_id_reclaim set to true
    mon.juju-733f2f-5-lxd-1 has auth_allow_insecure_global_id_reclaim set to true
    mon.juju-733f2f-4-lxd-15 has auth_allow_insecure_global_id_reclaim set to true
"""

The above issue is caused by a Ceph client (a Nova instance booting from a cinder volume that uses an old version of the Ceph client), described at:
https://ceph.io/releases/v14-2-20-nautilus-released

This is just a FYI from the Ceph cluster, not anything fatal. In this case, juju-verify would fail as a result of verifier any Ceph OSD or Mon (to see if it could be safely rebooted or shutdown).

We should explore if a "warning" should be raised rather than failing to verify the environment. Please note that other more serious issues such as Ceph service degradation due to some OSDs being out of service will also return HEALTH_WARN.

Alternatively, the "get-health" action in charm-ceph-mon could accept "details=true" to further explore the time of messages (and track a list of known messages that could be ignored if no further "unknown messages" are shared (to start with, the scope would be to evaluate if HEALTH_WARN could raise a "warning" rather than "FAIL", if that's acceptable from an operations perspective).

Changed in juju-verify:
status: New → Triaged
importance: Undecided → Medium
Changed in juju-verify:
assignee: nobody → Robert Gildein (rgildein)
status: Triaged → In Progress
Revision history for this message
Robert Gildein (rgildein) wrote :

This was a partial fix in [1], where I add health message to
result for HEALTH_WARN, HEALTH_ERR and unknown state.

---
[1]: https://github.com/canonical/juju-verify/pull/51

Changed in juju-verify:
status: In Progress → Triaged
assignee: Robert Gildein (rgildein) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.