[SRU] [OSSA-2019-002] Unable to install new flows on compute nodes when having broken security group rules (CVE-2019-10876)

It's affecting neutron-openvswitch-agent 2:12.0.5-0ubuntu1~cloud0

Hi,
Could you please specify which openstack/neutron release are you using, and add the relevant logs to the bug report as pastebin is not visible for everybody.

It affects qpenstack queens; neutron-openvswitch-agent 2:12.0.5-0ubuntu1~cloud0.

Below the pastes from the public ubuntu pastebin:

https://paste.ubuntu.com/p/s5Z4DNJspV/
https://paste.ubuntu.com/p/8pqkDZmt9N/

Found https://paste.ubuntu.com/p/s5Z4DNJspV/ missing.
New copy here: https://pastebin.ubuntu.com/p/g7Q3nFmhWN/

The thing to note is that they both have the same remote_group_id, 40ee2790-282f-4c6a-8f00-d5ee0b8b66d7 and one has ports_range_min/max=None (which in the code None is replaced with 1 on ports_range_min and 65535 on port_range_max).

The code then translated this remote_group_id into id 38, which in the cur_conj.remove(conj_id) call ended up failing to delete due to there being two different entities with the key 38, hence the KeyError. If I recall correctly from looking at this last week, cur_conj is an array of tuples of ('<json array of security group rule port/proto settings>', conj_id).

Note, this is a Xenial queens install.

I'm working on recreating this but any more details you can provide in the mean time will help speed that up (ie. juju crashdump and ensuing commands that lead up to the error).

As I understand it, creation of a security group rule B with '--remote-group A' specifies that rule B applies to any instances that use rule A. [1]

That said, can you explain why the rule for tcp 5672:5672 is needed along with the rule for tcp port_range_min:port_range_max? ie. Is the former a subset of the latter?

Thanks,
Corey

[1] https://docs.openstack.org/nova/queens/admin/security-groups.html#create-security-group-rules-for-a-cluster-of-instances

@dparv

pastebin is somewhat transient - please attach logs directly to the bug report for persistence!

Marking private as this may be a security issue.

@corey.byrant This is a managed customer cloud so we can't provide the exact commands used to trigger this because we didn't create the security group rules - our customer's customer did. Also, you seem to be assuming these rules make sense but it's not at all a given that they do; I suspect they were created by mistake.

In the paste below is truncated output of 'openstack security group rule list 7ca75f8f-9e32-4e94-b576-fa00ea472d3a' which may help to clarify/recreate the rules:

  https://paste.ubuntu.com/p/TcMxzgNdf3/

Also - super important detail is that the cloud in question is using the openvswitch firewall driver (per /etc/plugins/ml2/openvswitch_agent.ini) and not the default iptables based one.

@elmo Agreed, if a user can crash neutron-ovs-agent on a compute node that's not good. I'll dig more in the morning.

For now we have this marked private as a security issue for that reason ^.

The follow security group commands should resemble what causes the issue:

$ openstack security group create sec_group_A
$ openstack security group create sec_group_B
$ openstack security group rule create --ingress --proto tcp --dst-port 5682:5682 --remote-ip 0.0.0.0/0 sec_group_A
$ openstack security group rule create --ingress --proto tcp --dst-port 5672:5672 --remote-group sec_group_A sec_group_B
$ openstack security group rule create --ingress --proto tcp --remote-group sec_group_A sec_group_B

Which result in:

$ openstack security group rule list sec_group_B
+--------------------------------------+-------------+----------+------------+--------------------------------------+
| ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+----------+------------+--------------------------------------+
| 4823f9d9-1105-4ca8-a1dc-c2edb0cc1fd1 | tcp | None | | 0e212f1a-156b-43bc-bb74-092840f6272d |
| 4dc5446c-5e98-4ee7-be45-0e50c6f9a857 | tcp | None | 5672:5672 | 0e212f1a-156b-43bc-bb74-092840f6272d |
| 593c6d59-7fa9-447c-8c97-fd26578640c7 | None | None | | None |
| 68f8f07a-aa9e-4491-9e43-698873c25f91 | None | None | | None |
+--------------------------------------+-------------+----------+------------+--------------------------------------+

Note that for a charm deployment this requires neutron-gateway to use:

firewall-driver: openvswitch

s/neutron-gateway/neutron-openvswitch

I've not yet been able to recreate this but it's possible I'm doing something wrong.

I've deployed using the ovs firewall, created the security groups, then created a new subnet, router, plugged into ext_net.

One observation; I noticed the line numbers for rpc_loop in /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py in my deployment do not match those in the traceback from the paste in the initial bug description [1]:

1971 def rpc_loop(self, polling_manager=None, bridges_monitor=None):
...
2091 failed_devices = self.process_network_ports(
2092 port_info, provisioning_needed)

Note that I'm using a xenial-queens deployment and the neutron-openvswitch-agent version matches the one that hit the error, 2:12.0.5-0ubuntu1~cloud0.

[1] https://pastebin.canonical.com/p/wvFKjNWydr/
2019-01-18 18:16:29.566 1628577 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File "/usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 2114, in rpc_loop
2019-01-18 18:16:29.566 1628577 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info, provisioning_needed)

Ok I've recreated this, I just needed to deploy an instance. I'll work on getting some clear steps together for recreating.

Here are the steps I used to reproduce this, using juju but of course it should be simple enough to recreate with other deployment tooling.

1) juju-deployer -d -c default.yaml xenial-mitaka (sorry, the bundle is for the old juju-deployer tool rather than the native juju bundles)

See attachment for default.yaml bundle.

This bundle ensures /etc/neutron/plugins/ml2/openvswitch_agent.ini on compute nodes is using firewall_driver=openvswitch:

[securitygroup]
enable_security_group = True
firewall_driver = openvswitch

2) Our initial cloud setup creates a standard flat external network named ext_net and a private gre network. I don't know if that makes a difference or not.

openstack network list
+--------------------------------------+---------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+---------+--------------------------------------+
| 2b28b833-7be9-4dde-afdd-82eccf05955f | ext_net | 34ca55db-e5af-4706-b810-4f128d34bcef |
| ed9728c0-0744-4f14-acd1-420d90de6d27 | private | ea7a07d5-8df4-4977-afd3-f61010ae38b3 |
+--------------------------------------+---------+--------------------------------------+

(clients) ubuntu@coreycb-bastion:~/openstack-charm-testing$ openstack subnet list
+--------------------------------------+----------------+--------------------------------------+-----------------+
| ID | Name | Network | Subnet |
+--------------------------------------+----------------+--------------------------------------+-----------------+
| 34ca55db-e5af-4706-b810-4f128d34bcef | ext_net_subnet | 2b28b833-7be9-4dde-afdd-82eccf05955f | 10.5.0.0/16 |
| ea7a07d5-8df4-4977-afd3-f61010ae38b3 | private_subnet | ed9728c0-0744-4f14-acd1-420d90de6d27 | 192.168.21.0/24 |
+--------------------------------------+----------------+--------------------------------------+-----------------+

3)
openstack security group create sec_group_A
openstack security group create sec_group_B
openstack security group rule create --ingress --proto tcp --dst-port 5682:5682 --remote-ip 0.0.0.0/0 sec_group_A
openstack security group rule create --ingress --proto tcp --dst-port 5672:5672 --remote-group sec_group_A sec_group_B
openstack security group rule create --ingress --proto tcp --remote-group sec_group_A sec_group_B

4) create an instance using sec_group_B

openstack server create x1 --image xenial --flavor m1.tempest --nic net-id=`openstack network list | grep private | awk '{ print $2 }'` --security-group sec_group_B

5) See looping traceback in the compute node that the server gets scheduled to in /var/log/neutron/neutron-openvswitch-agent.log (see attachement)

I was able to recreate this on a bionic-rocky deployment as well.

Currently am unable to test on stein but assuming for now that this is also broken there.

This also occurs on xenial-pike, but not on xenial-ocata.

Apologies for the delay:

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

Since this was public for about a month prior to making private, it doesn't seem worth keeping under embargo here. Getting this fixed quickly is preferable and opening it up publicly would allow more eyes on it and easier communication to help fix this.

Thanks Gage.

I've confirmed this issue also exists in stein.

I'll work on this change, think I have a fix, just need to get some reviews.

Fix proposed to branch: master
Review: https://review.openstack.org/640252

Reviewed: https://review.openstack.org/640252
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=18c578aa10c19a6befdf1f1510645200f173eb44
Submitter: Zuul
Branch: master

commit 18c578aa10c19a6befdf1f1510645200f173eb44
Author: Brian Haley <email address hidden>
Date: Thu Feb 28 22:19:16 2019 -0500

    Fix KeyError in OVS firewall

    When merging port ranges, the code never assumed the
    conjunction ID might not be present in the set due to
    already being removed.

    In this case there were two security groups, both using
    the same remote security group, but the first security
    group does not define a port range and the second one does.
    Or more generally, the first SG port range is a subset
    of the second, as no port-range means the full range.

    Change-Id: I17ab643abbd2ec21eda4ae1dfb9abf2d4b0657f2
    Closes-bug: #1813007

For Ubuntu I've dropped pre-Pike releases from affected series as a fix for https://bugs.launchpad.net/neutron/+bug/1708092 would be a pre-requisite at least for Ocata. That fix is fairly involved and we would have to evaulate the risk of backporting that to Ocata if in fact any fixes were needed back that far.

New versions of the Ubuntu neutron package have been uploaded to disco, cosmic, and bionic unapproved queues where they are awaiting review by the SRU team, and also be uploaded to pike-staging.

Hello Diko, or anyone else affected,

Accepted neutron into pike-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:pike-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-pike-needed to verification-pike-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-pike-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Diko, or anyone else affected,

Accepted neutron into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:13.0.2-0ubuntu3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Diko, or anyone else affected,

Accepted neutron into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:12.0.5-0ubuntu3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Is there a chance anyone's working on backporting I17ab643abbd2ec21eda4ae1dfb9abf2d4b0657f2 upstream to stable/pike through stable/rocky? Is the impact roughly similar in nature to https://security.openstack.org/ossa/OSSA-2019-001.html or can someone make an attempt at describing a viable exploit scenario so we can draft an appropriate impact description?

Hello Diko, or anyone else affected,

Accepted neutron into rocky-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:rocky-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-rocky-needed to verification-rocky-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-rocky-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Diko, or anyone else affected,

Accepted neutron into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

This bug was fixed in the package neutron - 2:14.0.0~rc1-0ubuntu2~cloud0
---------------

 neutron (2:14.0.0~rc1-0ubuntu2~cloud0) bionic-stein; urgency=medium
 .
   * New update for the Ubuntu Cloud Archive.
 .
 neutron (2:14.0.0~rc1-0ubuntu2) disco; urgency=medium
 .
   * d/p/fix-KeyError-in-OVS-firewall.patch: Cherry-picked from upstream
     to prevent neutron ovs agent from crashing due to creation of two
     security groups that both use the same remote security group, where
     the first group's port range is a subset of the second (LP: #1813007).

Red Hat are treating this as a security issue, whilst similar to OSSA-2019-001 it is not considered quite as severe.

Awaiting the VMT decision, this looks like a Class A vulnerability based on the VMT taxonomy.

Thanks for the heads up, Joshua!

The OpenStack VMT is, in turn, waiting for the reporter, Neutron reviewers or, well, anybody really to clarify the impact of this bug and indicate whether a fix will be implemented in stable branches (per my comment #35).

I see backports to pike, queens, rocky and the stein release candidate branches have been proposed according to https://review.openstack.org/#/q/I17ab643abbd2ec21eda4ae1dfb9abf2d4b0657f2 and have been positively scored by some stable branch reviewers, so I'm going to take that as a sign the solution is acceptably backportable. I'll draft an impact description in a forthcoming bug comment to solicit further feedback.

Here's an OSSA impact statement draft. If any of this information is incorrect, please feel free to correct me and I will revise.

Title: Overlapping security group rules prevents compute node network conf
iguration
Reporter: Diko Parvanov (Canonical)
Products: Neutron
Affects: <11.0.7, >=12.0.0 <12.0.6, >=13.0.0 <13.0.3

Description: Diko Parvanov (Canonical) reported a vulnerability in neutron-openvswitch-agent security group rules. By creating two security groups with separate/overlapping port ranges, an authenticated user may prevent neutron from being able to configure networks on any compute nodes where those security groups are present. All neutron deployments utilizing neutron-openvswitch-agent are affected.

I don't see a backport change to Ocata, are there plans on submitting one?

https://review.openstack.org/#/q/I17ab643abbd2ec21eda4ae1dfb9abf2d4b0657f2

Thanks Gage, your proposed impact description in comment #43 looks great (modulo decisions on any stable/ocata backport altering the "affects" line).

@Gage, regarding Ocata can you take a look at comment #30 above? https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1813007/comments/30

In other words I don't think or knoww if this exists on Ocata seeing as pre-reqs aren't there.

The earliest I saw the changes that lead to this were in Pike. I agree with Corey, the pre-reqs are not there in Ocata.

In that case, as we expect stable/ocata is unaffected, the affects line should be revised as follows:

Affects: >=11.0.0 <11.0.7, >=12.0.0 <12.0.6, >=13.0.0 <13.0.3

Updated impact statement draft. If any of this information is incorrect, please feel free to correct and I will revise.

Title: Overlapping security group rules prevents compute node network configuration
Reporter: Diko Parvanov (Canonical)
Products: Neutron
Affects: >=11.0.0 <11.0.7, >=12.0.0 <12.0.6, >=13.0.0 <13.0.3

Description: Diko Parvanov (Canonical) reported a vulnerability in neutron-openvswitch-agent security group rules. By creating two security groups with separate/overlapping port ranges, an authenticated user may prevent neutron from being able to configure networks on any compute nodes where those security groups are present. All neutron deployments utilizing neutron-openvswitch-agent are affected.

Reviewed: https://review.openstack.org/648002
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=f6652f0ee6c51a0238d2b1ce10a86040456c85d9
Submitter: Zuul
Branch: stable/stein

commit f6652f0ee6c51a0238d2b1ce10a86040456c85d9
Author: Brian Haley <email address hidden>
Date: Thu Feb 28 22:19:16 2019 -0500

    Fix KeyError in OVS firewall

    When merging port ranges, the code never assumed the
    conjunction ID might not be present in the set due to
    already being removed.

    In this case there were two security groups, both using
    the same remote security group, but the first security
    group does not define a port range and the second one does.
    Or more generally, the first SG port range is a subset
    of the second, as no port-range means the full range.

    Change-Id: I17ab643abbd2ec21eda4ae1dfb9abf2d4b0657f2
    Closes-bug: #1813007
    (cherry picked from commit 18c578aa10c19a6befdf1f1510645200f173eb44)

Reviewed: https://review.openstack.org/648102
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=a9bc8ab1e1931aac84f30e29412f17264d88a5a0
Submitter: Zuul
Branch: stable/pike

commit a9bc8ab1e1931aac84f30e29412f17264d88a5a0
Author: Brian Haley <email address hidden>
Date: Thu Feb 28 22:19:16 2019 -0500

    Fix KeyError in OVS firewall

    When merging port ranges, the code never assumed the
    conjunction ID might not be present in the set due to
    already being removed.

    In this case there were two security groups, both using
    the same remote security group, but the first security
    group does not define a port range and the second one does.
    Or more generally, the first SG port range is a subset
    of the second, as no port-range means the full range.

    Change-Id: I17ab643abbd2ec21eda4ae1dfb9abf2d4b0657f2
    Closes-bug: #1813007
    (cherry picked from commit 18c578aa10c19a6befdf1f1510645200f173eb44)

Reviewed: https://review.openstack.org/648004
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=9751ebd36dc24f99b1951e928f8b910e7e0e01f7
Submitter: Zuul
Branch: stable/queens

commit 9751ebd36dc24f99b1951e928f8b910e7e0e01f7
Author: Brian Haley <email address hidden>
Date: Thu Feb 28 22:19:16 2019 -0500

    Fix KeyError in OVS firewall

    When merging port ranges, the code never assumed the
    conjunction ID might not be present in the set due to
    already being removed.

    In this case there were two security groups, both using
    the same remote security group, but the first security
    group does not define a port range and the second one does.
    Or more generally, the first SG port range is a subset
    of the second, as no port-range means the full range.

    Change-Id: I17ab643abbd2ec21eda4ae1dfb9abf2d4b0657f2
    Closes-bug: #1813007
    (cherry picked from commit 18c578aa10c19a6befdf1f1510645200f173eb44)

@coreycb please note that the SRU from bug 1818614 that is bundled with this one in proposed has failed verification due to a regression (see that bug for details)

@edward, Yes thanks for pointing that out. That will delay this fix from landing. Looks like your new fix for 1818614 is on track to land so that's good. https://review.openstack.org/#/c/649991

Reviewed: https://review.openstack.org/648003
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=569b3fddaba0737f150b75cf7ec9cb4137b3991b
Submitter: Zuul
Branch: stable/rocky

commit 569b3fddaba0737f150b75cf7ec9cb4137b3991b
Author: Brian Haley <email address hidden>
Date: Thu Feb 28 22:19:16 2019 -0500

    Fix KeyError in OVS firewall

    When merging port ranges, the code never assumed the
    conjunction ID might not be present in the set due to
    already being removed.

    In this case there were two security groups, both using
    the same remote security group, but the first security
    group does not define a port range and the second one does.
    Or more generally, the first SG port range is a subset
    of the second, as no port-range means the full range.

    Change-Id: I17ab643abbd2ec21eda4ae1dfb9abf2d4b0657f2
    Closes-bug: #1813007
    (cherry picked from commit 18c578aa10c19a6befdf1f1510645200f173eb44)

This bug was fixed in the package neutron - 2:14.0.0~rc1-0ubuntu3

---------------
neutron (2:14.0.0~rc1-0ubuntu3) disco; urgency=medium

  * d/p/bug1823038.patch: Cherry pick fix to ensure that None is not
    passed as an argument when spawning the neutron-keepalived-state-change
    agent (LP: #1823038).

 -- James Page <email address hidden> Tue, 09 Apr 2019 10:03:20 +0100

This issue was fixed in the openstack/neutron 11.0.7 release.

This issue was fixed in the openstack/neutron 13.0.3 release.

This issue was fixed in the openstack/neutron 12.0.6 release.

Hello Diko, or anyone else affected,

Accepted neutron into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:13.0.2-0ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

This issue was fixed in the openstack/neutron 14.0.1 release.

Hello Diko, or anyone else affected,

Accepted neutron into pike-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:pike-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-pike-needed to verification-pike-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-pike-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Diko, or anyone else affected,

Accepted neutron into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/neutron/2:12.0.5-0ubuntu4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Verified successfully on cosmic-proposed. Please see attached for results.

The verification of the Stable Release Update for neutron has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package neutron - 2:13.0.2-0ubuntu3.1

---------------
neutron (2:13.0.2-0ubuntu3.1) cosmic; urgency=medium

  * d/p/bug1823038.patch: Cherry pick fix to ensure that None is not
    passed as an argument when spawning the neutron-keepalived-state-change
    agent (LP: #1823038).

neutron (2:13.0.2-0ubuntu3) cosmic; urgency=medium

  * d/p/fix-KeyError-in-OVS-firewall.patch: Cherry-picked from upstream
    to prevent neutron ovs agent from crashing due to creation of two
    security groups that both use the same remote security group, where
    the first group's port range is a subset of the second (LP: #1813007).
  * d/p/set-initial-ha-router-state-in-neutron-keepalived-st.patch:
    Cherry-picked from upstream stable/rocky branch to ensure proper
    detection of MASTER HA router by neutron-keepalived-state-change
    (LP: #1818614).

 -- James Page <email address hidden> Tue, 09 Apr 2019 11:37:29 +0100

Verified successfully on bionic-proposed. Please see attached for results.

s/bionic-proposed/rocky-proposed ^^

Verified successfully on bionic-proposed. Please see attached for results.

Verified successfully on queens-proposed. Please see attached for results.

Verified successfully on pike-proposed. Please see attached for results.

This bug was fixed in the package neutron - 2:12.0.5-0ubuntu4

---------------
neutron (2:12.0.5-0ubuntu4) bionic; urgency=medium

  * d/p/bug1823038.patch: Cherry pick fix to ensure that None is not
    passed as an argument when spawning the neutron-keepalived-state-change
    agent (LP: #1823038).

neutron (2:12.0.5-0ubuntu3) bionic; urgency=medium

  * d/p/fix-KeyError-in-OVS-firewall.patch: Cherry-picked from upstream
    to prevent neutron ovs agent from crashing due to creation of two
    security groups that both use the same remote security group, where
    the first group's port range is a subset of the second (LP: #1813007).
  * d/p/set-initial-ha-router-state-in-neutron-keepalived-st.patch:
    Cherry-picked from upstream stable/rocky branch to ensure proper
    detection of MASTER HA router by neutron-keepalived-state-change
    (LP: #1818614).

 -- James Page <email address hidden> Tue, 09 Apr 2019 10:59:22 +0100

The verification of the Stable Release Update for neutron has completed successfully and the package has now been released to -updates. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package neutron - 2:11.0.6-0ubuntu1~cloud2.1
---------------

 neutron (2:11.0.6-0ubuntu1~cloud2.1) xenial; urgency=medium
 .
   * d/p/bug1823038.patch: Cherry pick fix to ensure that None is not
     passed as an argument when spawning the neutron-keepalived-state-change
     agent (LP: #1823038).
 .
 neutron (2:11.0.6-0ubuntu1~cloud2) xenial-pike; urgency=medium
 .
   * d/p/fix-KeyError-in-OVS-firewall.patch: Cherry-picked from upstream
     to prevent neutron ovs agent from crashing due to creation of two
     security groups that both use the same remote security group, where
     the first group's port range is a subset of the second (LP: #1813007).
   * d/p/set-initial-ha-router-state-in-neutron-keepalived-st.patch:
     Cherry-picked from upstream stable/rocky branch to ensure proper
     detection of MASTER HA router by neutron-keepalived-state-change
     (LP: #1818614).

This issue was fixed in the openstack/neutron 15.0.0.0b1 development milestone.