Canonical Juju

[wishlist] Anti-affinity rules for application unit placement when using openstack cloud provider

Bug #1834060 reported by Drew Freiberger
32
This bug affects 7 people
Affects Status Importance Assigned to Milestone
Canonical Juju
Triaged
Wishlist
Unassigned

Bug Description

It would be helpful for deploying and managing kubernetes (CDK) clusters on top of openstack (FCB) if we had the ability to declare a placement constraint for anti-affinity of a given application's instances.

For instance, if I have a 6 hypervisor cloud, and I deploy 6 kubernetes-worker instances on top of that cloud, I want to be able to place all 6 of those workers separate from each other as much as anti-affinity rules and current available placement weighting allows.

At the moment, we can target AZs for juju bundle machines which would end up in a 6 node, 3 AZ environment, with units possibly double-ing up on a hypervisor.

If we could target hypervisor names, while that would help the initial deployment, the placement api wouldn't track that affinity to that hypervisor through machine evacuations for maintenance. Is there a way to handle this in juju?

I believe anti-affinity rules require a ServerGroup to be defined and deployed via HEAT.

Revision history for this message
Richard Harding (rharding) wrote :

Thanks, it's definitely something we've looked into at some point. There's an array of use cases for more intelligent placement directives.

Changed in juju:
status: New → Triaged
importance: Undecided → Wishlist
Revision history for this message
Drew Freiberger (afreiberger) wrote :

As a workaround, we are implementing a flavor-based placement strategy where we'll target workers per compute node via flavor extra_specs=capabilities:host:<hypervisor_hostname> and then setting the machine constraints for the placement to use those hypervisor-tagged flavors, along with ensuring ComputeCapabilitiesFilter is part of the nova placement filters.

This seems a sufficient strategy for kubernetes-worker nodes, and then using AZ placement flavors for k8s-master, etcd, kubeapi-load-balancer, etc.

Revision history for this message
Alf Ihring (alfihring) wrote :

Hi all,
we would also be interested in a server_group based placement option for k8s on Openstack to regulate Worker Node placement on the hypervisors.

Felipe Reyes (freyes)
tags: added: seg
Revision history for this message
Drew Freiberger (afreiberger) wrote :

Attempted to use hostname in zones constraint with:

juju deploy ubuntu --constraints zones=zone1:myzone1hostname

and the result in juju machines is:

"suitable availability zone for machine 0 not found"

Revision history for this message
John A Meinel (jameinel) wrote :

Further discussion happened on Mattermost. My understanding of it was:

While Openstack does support the syntax AZ[:HOST[:NODE]], it is only supported if you are an administrator on the cloud. And thus not something that would be useful in general. It also puts us further down the road of custom crafting bundles and manually assigning to machines.

What we really need is a better primitive for defining some sort of affinity/anti-affinity rules so that the instances Juju requests can be backed by Openstack primitives available to normal users for spreading VMs across hosts.

Revision history for this message
Jeff Hillman (jhillman) wrote :

The correct syntax to set the property mentioned above is

openstack flavor create --property capabilities:host=<hypervisor_name> <flavor-name>

Revision history for this message
Andre Ruiz (andre-ruiz) wrote :

Ideally you should be able to create an affinity group in advance and just pass it's name for the vm to be added to as soon as it's instantiated. (Or a list of such groups.)

Revision history for this message
Erlon R. Cruz (sombrafam) wrote :

This is especially useful for deployments where all cluster needs to be in one AZ.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.