soft-affinity weight not normalized base on server group's maximum
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| OpenStack Compute (nova) |
Fix Released
|
Medium
|
Johannes Kulik | ||
| Pike |
New
|
Undecided
|
Unassigned | ||
| Queens |
New
|
Undecided
|
Unassigned | ||
| Rocky |
New
|
Undecided
|
Unassigned | ||
| Stein |
New
|
Undecided
|
Unassigned | ||
| Train |
New
|
Undecided
|
Unassigned | ||
Bug Description
Description
===========
When using soft-affinity to schedule instances on the same host, the weight is unexpectedly low if a server was previously scheduled to any server-group with more members on a host. This low weight can then be easily outweighed by differences in resources (e.g. RAM/CPU).
Steps to reproduce
==================
Do not restart nova-scheduler in the process or the bug doesn't appear. You need to change the ServerGroupSoft
* Create a server-group with soft-affinity (let's call it A)
* Create 6 servers in server-group A, one after the other so they end up on the same host.
* Create another server-group with soft-affinity (B)
* Create 1 server in server-group B
* Create 1 server in server-group B and look at the scheduler's weights assigned to the hosts by the ServerGroupSoft
Expected result
===============
The weight assigned to the host by the ServerGroupSoft
Actual result
=============
The weight assigned to the host by the ServerGroupSoft
Environment
===========
We noticed this on a queens version of nova a year ago. Can't give the exact commit anymore, but the code still looks broken in current master.
I've opened a review-request for fixing this bug here: https:/
| Changed in nova: | |
| assignee: | nobody → Johannes Kulik (jkulik) |
| status: | New → In Progress |
| description: | updated |
| Changed in nova: | |
| importance: | Undecided → Medium |
| tags: | added: scheduler |
| OpenStack Infra (hudson-openstack) wrote Fix merged to nova (master) | #1 |
| Changed in nova: | |
| status: | In Progress → Fix Released |
Reviewed: https:/
Committed: https:/
Submitter: Zuul
Branch: master
commit 5ab9ef11e27014c
Author: Johannes Kulik <email address hidden>
Date: Thu Mar 19 12:51:25 2020 +0100
Don't recompute weighers' minval/maxval attributes
Changing the minval/maxval attribute to the minimum/maxium of every
weigher run changes the outcome of future runs. We noticed it in the
SoftAffinit
instances for a server-group would make a later run use that maximum.
This resulted in the weight being lower than 1 for a host hosting all
instances of another server-group, if the number of instances of that
server-group on that host is less than a previous server-group's
instances on any host.
Previously, there were two places that computed the maxval/minval - once
in normalize() and once in weigh_objects() - but only the one in
weigh_objects() saved the values to the weigher.
The code now uses the maxval/minval as defined by the weigher and keeps
the weights inside the maxval-minval range. There's also only one place
to compute the minval/maxval now, if the weigher did not set a value:
normalize().
Closes-Bug: 1870096
Change-Id: I60a90dabcd21b4