senlin-health-manager load more cluster make high load request to keystone

Bug #1975440 reported by Bo Tran
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
senlin
Fix Released
Undecided
Unassigned

Bug Description

When I run senlin health manager in my systems. I met an error that is: senlin-health-manager make high load to keystone because it to do health check nova instances

I had see log and found that senlin-health-manager auto reload healthcheck register from database with more cluster than real. This problem, I just met sometime not always.

I running with healthcheck policy with this specs:

```sh
---
description: A policy for maintaining node health from a cluster.
properties:
  detection:
    detection_modes:
    - type: NODE_STATUS_POLLING
    interval: 60
    node_update_timeout: 30
  recovery:
    actions:
    - name: RECREATE
type: senlin.policy.health
version: '1.1'
```

I was running senlin-health-manager with config worker is 32 on one controller node.

Changed in senlin:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to senlin (master)

Reviewed: https://review.opendev.org/c/openstack/senlin/+/842025
Committed: https://opendev.org/openstack/senlin/commit/ee2959af3c1fa63ce7068047105ec66ebcf4f5aa
Submitter: "Zuul (22348)"
Branch: master

commit ee2959af3c1fa63ce7068047105ec66ebcf4f5aa
Author: Erik Olof Gunnar Andersson <email address hidden>
Date: Mon May 16 20:09:53 2022 -0700

    Fixed service manage not cleaning properly

    When spliting the engine up into multiple services
    the cleanup job was left on the conductor, but in
    reality each service needs to handle their own clean
    up, as the service startup order is never guaranteed.

    This patch refactors the service workflow and moves
    all cleanup logic to the base class.

    Closes-Bug: 1975440
    Change-Id: I718910a124780804ec8e14c2e99697025e4c4df0

Changed in senlin:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/senlin 14.0.0.0rc1

This issue was fixed in the openstack/senlin 14.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.