Ubuntu Cloud Archive

keepalived_use_no_track default=True breaks bionic deploys

  1. Series ussuri
  2. Bug #1896506
Bug #1896506 reported by Corey Bryant
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Ubuntu Cloud Archive
Invalid
Undecided
Unassigned
Stein
New
Undecided
Unassigned
Train
New
Undecided
Unassigned
Ussuri
In Progress
Undecided
Unassigned
neutron
Fix Released
High
Lajos Katona
neutron (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

[Impact]

In the latest Neutron stable release, a new config option was introduced to control using no_track in Keepalived. This config options defaults to true which is broken on releases before Focal.

The proposed fix for this in earlier releases is to change the default to False.

[Test Case]

TBD

[Regression Potential]

The regression potential is low as the fix is changing a configuration default.

--------------------------------------------------------------------------

Presumably the default of keepalived_use_no_track default=True breaks bionic deploys and the patch [1] has been backported to several bionic-based stable branches with a conflicting default config value; it's been backported back to stable/rocky.

[1] https://review.opendev.org/#/c/747862/1

This doesn't seem like a good backport to stable branches without having some logic in it that detects the version of keepalived and sets the default appropriately based on that version.

More discussion with slaweq from #openstack-neutron:

coreycb> Corey Bryant slaweq: about this backport, will it break deployments that are on bionic? https://review.opendev.org/#/c/747857/1
10:40 AM I don't think there are any stein deployments on focal
11:06 AM
<slaweq> Sławek Kapłoński coreycb: what keepalived version is available on Bionic?
11:06 AM coreycb: ahh, ok it's 1.x
11:06 AM
<coreycb> Corey Bryant slaweq: correct 1.x
11:06 AM
<slaweq> Sławek Kapłoński so it should be set to False on Bionic
11:10 AM → armax joined ⇐ priteau quit
11:12 AM
<coreycb> Corey Bryant slaweq: it would be really nice to if we could detect the version and set it appropriately

Can we revert this for all releases prior to Ussuri?

For Ubuntu, Ussuri is available by default with Focal, but we also backport Ussuri to bionic so we would have to change the default for keepalived_use_no_track default to False for bionic.

See original description
Tags: l3-ha
Corey Bryant (corey.bryant)
description: updated
description: updated
summary: - keepalived_us_no_track default=True breaks bionic deploys
+ keepalived_use_no_track default=True breaks bionic deploys
description: updated
Revision history for this message
Brian Haley (brian-haley) wrote :

Slawek - do you want to take this?

Changed in neutron:
importance: Undecided → High
status: New → Confirmed
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in neutron (Ubuntu):
status: New → Confirmed
Revision history for this message
Corey Bryant (corey.bryant) wrote :

Rodlolfo's comment is pertient as well regarding making a run-time check similar to https://review.opendev.org/#/c/726079/1/neutron/agent/linux/dhcp.py.

https://bugs.launchpad.net/neutron/+bug/1890400/comments/1

Revision history for this message
Alexander Gräb (alexander.graeb) wrote :

I am voting against completely reverting the bespoken bug fix since we need this bug fix for our Ubuntu Bionic based Train deployments. We use keepalived < 2.x and set keepalived_use_no_track default=False.

I would also prefer to have a run-time check or setting the default for keepalived_use_no_track to false for older OS releases.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

A run-time check would be ideal. Keep in mind with setting defaults statically that some releases are supported on multiple distro series. For example in ubuntu, ussuri is supported on focal and bionic. We can patch it in distro, of course, but what concerns me if we have to do that is that we can't comb through all future patches to ensure we're setting the right default. The upstream code should do the right thing depending on what version is available.

Slawek Kaplonski (slaweq)
tags: added: l3-ha
Revision history for this message
Bernard Cafarelli (bcafarel) wrote :

If I remember correctly, the root issue is not really https://review.opendev.org/#/c/747857 which adds a config option specifically to mitigate the issue. It was originally added uconditionally with https://review.opendev.org/#/c/721799/ (and similar backports) seen as working fine with keepalived 1.x.
When it was reported to cause issues with some versions, this configuration option was added and backported so distributions can disable it per version and build

And of course all for a runtime detection check if you can get one, I do not remember the discussions around this patch but general opinion was it was not possible or at least not easy (remember checking the version is not enough for LTS versions)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.opendev.org/757620

Changed in neutron:
assignee: nobody → Lajos Katona (lajos-katona)
status: Confirmed → In Progress
Revision history for this message
Lajos Katona (lajos-katona) wrote :

Bernard, Corey could you please check https://review.opendev.org/757620 , I tried to add version check, but not sure if all edge cases handled, and can't find more details of discussion around it.

Chris MacNaughton (chris.macnaughton)
Changed in cloud-archive:
status: New → Invalid
Chris MacNaughton (chris.macnaughton)
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.opendev.org/759657

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.opendev.org/757620
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4dca7fae87d9bea01e01f786465fdb4d8231dc69
Submitter: Zuul
Branch: master

commit 4dca7fae87d9bea01e01f786465fdb4d8231dc69
Author: elajkat <email address hidden>
Date: Tue Oct 6 13:20:06 2020 +0200

    Keepalived version check

    [0] introduced keepalived_use_no_track config option which defaults to
    True.
    Here I add and extra runtime check to check keepalived version, and
    only if that version is bigger than 2.0.0 add the no_track option to
    the keepalived config file.

    [0] https://review.opendev.org/745641

    Change-Id: I16dce5b3faf62ce02535170a1b0e1ad22ee8ac30
    Closes-Bug: #1896506

Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.opendev.org/759657
Committed: https://opendev.org/openstack/neutron/commit/2f66cb8182f6c1f07018584b7a4d721c03b5711a
Submitter: Zuul
Branch: master

commit 2f66cb8182f6c1f07018584b7a4d721c03b5711a
Author: elajkat <email address hidden>
Date: Mon Oct 26 11:07:45 2020 +0100

    Deprecate config option "keepalived_use_no_track"

    Change-Id: I448365bad076e67b32198277101f188fbfc3dece
    Related-Bug: #1896506

Bernard Cafarelli (bcafarel)
tags: added: neutron-proactive-backport-potential
Slawek Kaplonski (slaweq)
tags: removed: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 18.0.0.0rc1

This issue was fixed in the openstack/neutron 18.0.0.0rc1 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/ussuri)

Fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/798335

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/victoria)

Fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/799164

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/victoria)

Change abandoned by "yangjianfeng <email address hidden>" on branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/799164

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron (stable/ussuri)

Change abandoned by "yangjianfeng <email address hidden>" on branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/798335

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.