ovs-vswitchd needs to be forced to reconfigure after adding protocols to bridges

Hello:

I've seen this behavior in OVS 2.12. Previous versions work well (2.11 and 2.10).

I detected this problem using Neutron (master), OVS 2.12 and OVS firewall. The firewall flows are added to the DB using "--bundle". This requires OF14. Because this protocol is added after the bridge is created, it does not work.

But if the protocol is added in the same transaction of the creation [1], then the "--bundle" add-flows commands work.

Regards.

[1] https://github.com/openstack/neutron/blob/4051e0b19dc9f318c2e0dd7c60eaa2c46536ad03/neutron/agent/common/ovs_lib.py#L286-L302

Status changed to 'Confirmed' because the bug affects multiple users.

Related patch in Neutron: https://review.opendev.org/#/c/698715/

Related fix proposed to branch: master
Review: https://review.opendev.org/716538

This has been reported to Kolla via IRC.

Related fix proposed to branch: master
Review: https://review.opendev.org/726010

The attachment "lp1852221-eoan-train.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

unsubscribing ubuntu-sponsors as openstack team will handle uploading this

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/733674

Thanks Ed. A new version of neutron has been uploaded to the eoan unapproved queue with your debdiff applied.
https://launchpad.net/ubuntu/eoan/+queue?queue_state=1&queue_text=neutron

Reviewed: https://review.opendev.org/733674
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7d9742004643d97fe76e9d461ab24090f2f705fa
Submitter: Zuul
Branch: stable/train

commit 7d9742004643d97fe76e9d461ab24090f2f705fa
Author: Jakub Libosvar <email address hidden>
Date: Tue Jan 14 11:30:10 2020 +0000

    Set OpenFlow 1.0, 1.3 and 1.4 by default on bridges

    There is a bug in OVS 2.12 where it's impossible to change protocol on
    a bridge. This patch should be reverted once OVS is fixed. More
    information about the bug at [1].

    [1] https://bugzilla.redhat.com/show_bug.cgi?id=1782834

    Related-Bug: #1852221

    Change-Id: I1ead1eee48a0c56193f20797ab35be36a0458270
    (cherry picked from commit 0643ab44d8204cde78fb7e8713fdd46dad0d87df)

Hello Frode, or anyone else affected,

Accepted neutron into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:15.0.2-0ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Frode, or anyone else affected,

Accepted neutron into train-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:train-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-train-needed to verification-train-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-train-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

eoan verified with [Test Case] and output is:

root@juju-38af90-lp1852221-eoan-6:/home/ubuntu# apt-cache policy neutron-openvswitch-agent
neutron-openvswitch-agent:
  Installed: 2:15.0.2-0ubuntu1.2
  Candidate: 2:15.0.2-0ubuntu1.2
  Version table:
 *** 2:15.0.2-0ubuntu1.2 500
        500 http://archive.ubuntu.com/ubuntu eoan-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
     2:15.0.2-0ubuntu1.1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu eoan-updates/main amd64 Packages
     2:15.0.0-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu eoan/main amd64 Packages
root@juju-38af90-lp1852221-eoan-6:/home/ubuntu# sudo ovs-ofctl -O OpenFlow14 dump-flows br-int| grep -v cookie
OFPST_FLOW reply (OF1.4) (xid=0x2):
root@juju-38af90-lp1852221-eoan-6:/home/ubuntu# grep negotiation /var/log/neutron/neutron-*
root@juju-38af90-lp1852221-eoan-6:/home/ubuntu#

bionic-train verified with [Test Case] and output is:

ubuntu@hopem-bastion:~/stsstack-bundles/openstack$ juju ssh nova-compute/0 sudo -s
root@juju-dc1c82-lp1852221-train-6:~# apt-cache policy neutron-openvswitch-agent
neutron-openvswitch-agent:
  Installed: 2:15.0.2-0ubuntu1.2~cloud0
  Candidate: 2:15.0.2-0ubuntu1.2~cloud0
  Version table:
 *** 2:15.0.2-0ubuntu1.2~cloud0 500
        500 http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-proposed/train/main amd64 Packages
        100 /var/lib/dpkg/status
     2:12.1.0-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
     2:12.0.1-0ubuntu1 500
        500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
root@juju-dc1c82-lp1852221-train-6:~# sudo ovs-ofctl -O OpenFlow14 dump-flows br-int| grep -v cookie
OFPST_FLOW reply (OF1.4) (xid=0x2):
root@juju-dc1c82-lp1852221-train-6:~# grep negotiation /var/log/neutron/neutron-*
root@juju-dc1c82-lp1852221-train-6:~#

The verification of the Stable Release Update for neutron has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package neutron - 2:15.0.2-0ubuntu1.2~cloud0
---------------

 neutron (2:15.0.2-0ubuntu1.2~cloud0) bionic-train; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 neutron (2:15.0.2-0ubuntu1.2) eoan; urgency=medium
 .
   * d/p/set-openflow-protocols-by-default-on-bridges.patch Backport fix to resolve
     issues with neutron adding OF protocols to bridges post-create (LP: #1852221).

Status changed to 'Confirmed' because the bug affects multiple users.

Eoan is EOL in July 2020

This bug was fixed in the package neutron - 2:15.0.2-0ubuntu1.2

---------------
neutron (2:15.0.2-0ubuntu1.2) eoan; urgency=medium

  * d/p/set-openflow-protocols-by-default-on-bridges.patch Backport fix to resolve
    issues with neutron adding OF protocols to bridges post-create (LP: #1852221).

 -- Edward Hope-Morley <email address hidden> Sun, 31 May 2020 16:50:01 +0100

Change abandoned by Frode Nordahl (<email address hidden>) on branch: master
Review: https://review.opendev.org/716538
Reason: The upstream Neutron workaround for this issue makes the charm handling redundant.

Seeing this in Focal, openvswitch version 2.13.0-0ubuntu1

re-reading this, the issue I was seeing was that the protocol wasn't negotiated - did not need to restart ovs to get the 'good' test. Apologies for the noise, it does actually look like this is also fixed in 2.13.0-0ubuntu1 and possibly could be updated for openvswitch (Ubuntu).

2.15.0 contains the fix for this issue - marking Fix Released.

Also backported to >= 2.13.2 so resolved for Ubuntu 20.04 LTS and later

Fixed in v2.12.2 so still impacts the Train UCA (2.12.1 currently).

Earlier releases should not have this issue as don't contain the commit that regressed this feature.

$ git tag --contains a0baa7dfa4fe

v2.12.0
v2.12.1
v2.12.2
v2.12.3
v2.13.0
v2.13.1
v2.13.2
v2.13.3
v2.13.4
v2.14.0
v2.14.1
v2.14.2
v2.15.0
v2.15.1

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/812162

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron/+/812162