[RFE] NFTables Firewall Driver

This sounds good. What's the distribution support like for NFT?

Cool!

Ubuntu have NFTables since 14.10 (super easy to backport to 14.04
(Linux 3.13 support it as well), if needed - I have it on my home
firewall now).

Next Ubuntu 16.04 will be the first LTS version of Ubuntu, with NFTables.

I believe that Debian 9 will also include NFTables.

Sorry about typos grammar errors... English isn't my native language...

Best,
Thiago

On 20 October 2015 at 16:36, Kevin Benton <email address hidden> wrote:
> This sounds good. What's the distribution support like for NFT?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1508155
>
> Title:
> [rfe] NFTables Firewall Driver
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/neutron/+bug/1508155/+subscriptions

To be discussed during the drivers meeting.

For some reason the hybrid solution performs equal or sightly better in some conditions (don't ask me why, I don't know). It works, and I agree it has a lot of complexity.

NFT could be a good thing for linuxbridge or the virtual routers. But I'd prefer to see an openflow+CT firewall for OVS. (to avoid mixing too many technologies at once, like we have now)

We also have the OVS/CT initiative I know @jlibova was working on it already [1]. Btw, we have the old spec for this, but I'm not unsure if he filled any rfe.

[1] https://blueprints.launchpad.net/neutron/+spec/ovs-firewall-driver

So are we saying that going forward we could have the following options?

LinuxBridge -> NFTables
Open vSwitch -> Openflow+CT?

Let's reach consensus.

NFTables for Linux bridge sounds good.

For OVS, I tend to agree with ajo.
we use OVS tables for various purposes and NFtables is a wrapper for OVS/CT.
It is better to use OVS table in a native way as long as it becomes too complicated and/or reinvents an existing things.

One concern on ovs-firewall-driver ([1] above) is that security group API change was proposed as a part of the work. This blocked subsequent effort. Changing SG API is not a good idea. The situation might have changed since then. If the situation changed, I think OVS firewall is a good direction.

@amotoki, yes, the situation changed, there are no changes suggested to the sg api now,
although the blueprint could be outdated (checking).

As discussed in [1], there was consensus on strategy as outlined in #5. However we need a sponsor for this initiative.

[1] http://eavesdrop.openstack.org/meetings/neutron_drivers/2016/neutron_drivers.2016-01-12-15.00.log.html

Any volunteers to spec and work on this?

http://eavesdrop.openstack.org/meetings/neutron_drivers/2016/neutron_drivers.2016-02-04-22.01.log.html#l-137

Great request in principle, but no-one seems willing to work on it.

[Expired for neutron because there has been no activity for 60 days.]

I think this should be revisited, nftables may hinder throughput much less than iptables.

Hey guys,

It's quite nice to see that more people thinking about this!

Things changed since 2015 and it might be fairly easy to start using nftables with Neutron today, the hybrid mode! As follows:

https://wiki.debian.org/nftables

https://www.redhat.com/en/blog/using-iptables-nft-hybrid-linux-firewall

This way, no changes are required on Neutron. However, I honestly don't know the limitations of this approach. I'm planning to test this soon!

---

Another thing that is on my radar, is a Load Balancer based on nftables! It seems to be really awesome! So, what about replacing Octavia with something a lot more simpler? Here is it:

https://www.zevenet.com/knowledge-base/nftlb/what-is-nftlb/

---

Time flies and now, Linux has the so-called BPFilter!

BPF comes to firewalls:
https://lwn.net/Articles/747551/

https://news.ycombinator.com/item?id=16421168

What in the hell? The nftables is already obsolete?

No idea... lol

Look, at, this:

BPF at Facebook (and beyond):
https://lwn.net/Articles/801871/

---

What do you guys think?!

:-D

I talked about this at the Ireland snowstorm, I mean, meeting - and now we're hitting the xlock limitations... hard...

I know both iptables and nft but... It could be hard to have the time to figure this out...

It seems other folks in the virt community already switched to native nftables (see libvirt, LXD, firewalld, some ongoing efforts in k8s, etc).

I'm running neutron rocky with debian buster just fine with iptables-nft and ebtables-nft (requires iptables 1.8.5 at least), using linuxbridge.

Basically, neutron is using nftables just fine, without even noticing. It would be awesome if the native driver is implemented to actually gain from the benefits of nftables: native sets, dictionaries and maps, native support for all the address families in a single tool, and more.

Arturo - I don't see Neutron going to a native nftables implementation soon, mostly because the default install uses ML2/OVS with the OVS firewall, which doesn't require iptables for security groups. And with us moving more towards OVN it's not an issue, except when using linuxbridge, which isn't exactly a priority.

Hey guys! Long time!

I'm using in production, nftables via Ubuntu's etc/alternative subsystem, so, my iptables points to iptables-nft and I'm actually using nftables without Neutron realizing it!

It's cool but, what about the ipset? I have it configured on Neutron but, apparently, nftables has its own sets and there is no `ipset` etc/alternative that uses nftables as a backend.

https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_ipset_to_nftables

So, is this a problem? My environment is working okay...

I disagree with Brian, the Linux Bridges are the default in big deployments out there, for example, with OpenStack Ansible. I have deployed a handful of OpenStack clouds in the last 4 years, all based on Linux Bridges.

The OVN is highly experimental and adding OpenvSwitch to the mix only makes things way more complicated.

It would be awesome to see Neutron with a native nftables driver! But I'm happy with the etc/alternative option from Debian/Ubuntu BUT, what about the `ipset`?

Cheers!

We can agree to disagree on using the Linux Bridge driver, but we (Red Hat) have almost no customers using it, and our default is now OVN, so it's not experimental at all IMO.

And yes, most distros have iptables-nft under the hood, so you're using nftables indirectly. It should just continue to work. If someone committed to writing an nftables driver I'm sure the community would take a look at it, but no one has proposed doing it at our planning meetings.

Actually we were discussing that on last PTG and Rodolfo already started slowly some work on that. Please see his early WIP patch https://review.opendev.org/c/openstack/neutron/+/759874

Reviewed: https://review.opendev.org/c/openstack/neutron/+/785177
Committed: https://opendev.org/openstack/neutron/commit/0a931391d8990f3e654b4bfda24ae4119c609bbf
Submitter: "Zuul (22348)"
Branch: master

commit 0a931391d8990f3e654b4bfda24ae4119c609bbf
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Apr 7 13:16:21 2021 +0000

    Make ARP protection commands compatible with "ebtables-nft"

    "nftables" compatible binary, "ebtables-nft", is not 100% compatible
    with the legacy API, as reported in LP#1922892.

    This patch fixes the following issues when using "ebtables-nft" (while
    keeping compatibility with legacy binary):
    - When a new chain is created, a default DROP rule is added at the end
      of the chain (append). This will prevent the error code 4 when the
      chain is listed.
    - The chain rules are added at the begining of the chain (insert),
      before the default DROP rule. This will prioritize the port rules.
    - The MAC rules are cleaned before the new ones are added. That will
      prevent the deletion of any new needed rule, now added after the
      deletion.
    - The "ebtables" command will retry on error code 4. This is the
      error returned when the chains are listed and no rule is present
      in a new created chain (reporeted in LP#1922892).

    This code is backwards compatible, that means it works with the legacy
    "ebtables" binary; this is currently installed in the Neutron CI [1].
    In order to test with the new binary, "ebtables-nft", two new CI jobs
    are added to the periodic queue [2].

    [1]https://github.com/openstack/neutron/blob/1ad9ca56b07ffdc9f7e0bc6a62af61961b9128eb/roles/legacy_ebtables/tasks/main.yaml
    [2]https://review.opendev.org/c/openstack/neutron/+/785144

    Closes-Bug: #1922892
    Related-Bug: #1508155

    Change-Id: I9463b000f6f63e65aaf91d60b30f6c92c01e3baf

Reviewed: https://review.opendev.org/c/openstack/neutron/+/785917
Committed: https://opendev.org/openstack/neutron/commit/dd40bac8b54a7f1c05a573d915c705b0539f0d8f
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit dd40bac8b54a7f1c05a573d915c705b0539f0d8f
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Apr 7 13:16:21 2021 +0000

    Make ARP protection commands compatible with "ebtables-nft"

    "nftables" compatible binary, "ebtables-nft", is not 100% compatible
    with the legacy API, as reported in LP#1922892.

    This patch fixes the following issues when using "ebtables-nft" (while
    keeping compatibility with legacy binary):
    - When a new chain is created, a default DROP rule is added at the end
      of the chain (append). This will prevent the error code 4 when the
      chain is listed.
    - The chain rules are added at the begining of the chain (insert),
      before the default DROP rule. This will prioritize the port rules.
    - The MAC rules are cleaned before the new ones are added. That will
      prevent the deletion of any new needed rule, now added after the
      deletion.
    - The "ebtables" command will retry on error code 4. This is the
      error returned when the chains are listed and no rule is present
      in a new created chain (reporeted in LP#1922892).

    This code is backwards compatible, that means it works with the legacy
    "ebtables" binary; this is currently installed in the Neutron CI [1].
    In order to test with the new binary, "ebtables-nft", two new CI jobs
    are added to the periodic queue [2].

    [1]https://github.com/openstack/neutron/blob/1ad9ca56b07ffdc9f7e0bc6a62af61961b9128eb/roles/legacy_ebtables/tasks/main.yaml
    [2]https://review.opendev.org/c/openstack/neutron/+/785144

    Closes-Bug: #1922892
    Related-Bug: #1508155

    Change-Id: I9463b000f6f63e65aaf91d60b30f6c92c01e3baf
    (cherry picked from commit 0a931391d8990f3e654b4bfda24ae4119c609bbf)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/785144
Committed: https://opendev.org/openstack/neutron/commit/f7d2c3608d1fa6cbc8c562f9121ec0f0d80c3962
Submitter: "Zuul (22348)"
Branch: master

commit f7d2c3608d1fa6cbc8c562f9121ec0f0d80c3962
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Apr 7 10:56:09 2021 +0000

    Add periodic jobs to test "nftables" binaries

    In order to check how the "nftables" binaries work with Neutron, two
    new jobs have been added to the periodic queue:
    - neutron-tempest-plugin-scenario-linuxbridge-nftables
    - neutron-tempest-plugin-scenario-openvswitch-iptables_hybrid-nftables

    In those two jobs, the binaries for "iptables", "ip6tables",
    "arptables" and "ebtables" are replaced with the "nftables"
    counterparts; by default, newer operating systems use the "nftables"
    versions, providing the legacy API to the user but executing the
    new packet handling in Netfilter.

    Change-Id: Idec6d480886298f6d71b1dd649c9255ee6b7bebb
    Related-Bug: #1508155
    Related-Bug: #1922892

Change abandoned by "Rodolfo Alonso <email address hidden>" on branch: master
Review: https://review.opendev.org/c/openstack/neutron/+/775413
Reason: tested patches already merged

Related fix proposed to branch: stable/victoria
Review: https://review.opendev.org/c/openstack/neutron/+/804056

Related fix proposed to branch: stable/ussuri
Review: https://review.opendev.org/c/openstack/neutron/+/804057

Related fix proposed to branch: stable/train
Review: https://review.opendev.org/c/openstack/neutron/+/804058

Reviewed: https://review.opendev.org/c/openstack/neutron/+/804056
Committed: https://opendev.org/openstack/neutron/commit/fafa5dacd5057120562184a734e7345e7c0e9639
Submitter: "Zuul (22348)"
Branch: stable/victoria

commit fafa5dacd5057120562184a734e7345e7c0e9639
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Apr 7 13:16:21 2021 +0000

    Make ARP protection commands compatible with "ebtables-nft"

    "nftables" compatible binary, "ebtables-nft", is not 100% compatible
    with the legacy API, as reported in LP#1922892.

    This patch fixes the following issues when using "ebtables-nft" (while
    keeping compatibility with legacy binary):
    - When a new chain is created, a default DROP rule is added at the end
      of the chain (append). This will prevent the error code 4 when the
      chain is listed.
    - The chain rules are added at the begining of the chain (insert),
      before the default DROP rule. This will prioritize the port rules.
    - The MAC rules are cleaned before the new ones are added. That will
      prevent the deletion of any new needed rule, now added after the
      deletion.
    - The "ebtables" command will retry on error code 4. This is the
      error returned when the chains are listed and no rule is present
      in a new created chain (reporeted in LP#1922892).

    This code is backwards compatible, that means it works with the legacy
    "ebtables" binary; this is currently installed in the Neutron CI [1].
    In order to test with the new binary, "ebtables-nft", two new CI jobs
    are added to the periodic queue [2].

    [1]https://github.com/openstack/neutron/blob/1ad9ca56b07ffdc9f7e0bc6a62af61961b9128eb/roles/legacy_ebtables/tasks/main.yaml
    [2]https://review.opendev.org/c/openstack/neutron/+/785144

    Closes-Bug: #1922892
    Related-Bug: #1508155
    Closes-Bug: #1938670

    Conflicts:
        neutron/tests/unit/plugins/ml2/drivers/linuxbridge/agent/test_arp_protect.py

    Change-Id: I9463b000f6f63e65aaf91d60b30f6c92c01e3baf
    (cherry picked from commit 0a931391d8990f3e654b4bfda24ae4119c609bbf)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/804057
Committed: https://opendev.org/openstack/neutron/commit/f034de001ba224c3add364ca5d00cc15e26c2e96
Submitter: "Zuul (22348)"
Branch: stable/ussuri

commit f034de001ba224c3add364ca5d00cc15e26c2e96
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Apr 7 13:16:21 2021 +0000

    Make ARP protection commands compatible with "ebtables-nft"

    "nftables" compatible binary, "ebtables-nft", is not 100% compatible
    with the legacy API, as reported in LP#1922892.

    This patch fixes the following issues when using "ebtables-nft" (while
    keeping compatibility with legacy binary):
    - When a new chain is created, a default DROP rule is added at the end
      of the chain (append). This will prevent the error code 4 when the
      chain is listed.
    - The chain rules are added at the begining of the chain (insert),
      before the default DROP rule. This will prioritize the port rules.
    - The MAC rules are cleaned before the new ones are added. That will
      prevent the deletion of any new needed rule, now added after the
      deletion.
    - The "ebtables" command will retry on error code 4. This is the
      error returned when the chains are listed and no rule is present
      in a new created chain (reporeted in LP#1922892).

    This code is backwards compatible, that means it works with the legacy
    "ebtables" binary; this is currently installed in the Neutron CI [1].
    In order to test with the new binary, "ebtables-nft", two new CI jobs
    are added to the periodic queue [2].

    [1]https://github.com/openstack/neutron/blob/1ad9ca56b07ffdc9f7e0bc6a62af61961b9128eb/roles/legacy_ebtables/tasks/main.yaml
    [2]https://review.opendev.org/c/openstack/neutron/+/785144

    Closes-Bug: #1922892
    Related-Bug: #1508155
    Closes-Bug: #1938670

    Conflicts:
        neutron/tests/unit/plugins/ml2/drivers/linuxbridge/agent/test_arp_protect.py

    Change-Id: I9463b000f6f63e65aaf91d60b30f6c92c01e3baf
    (cherry picked from commit 0a931391d8990f3e654b4bfda24ae4119c609bbf)
    (cherry picked from commit fafa5dacd5057120562184a734e7345e7c0e9639)

Reviewed: https://review.opendev.org/c/openstack/neutron/+/804058
Committed: https://opendev.org/openstack/neutron/commit/4245963c71280bebd94e9005230d610931a7c2df
Submitter: "Zuul (22348)"
Branch: stable/train

commit 4245963c71280bebd94e9005230d610931a7c2df
Author: Rodolfo Alonso Hernandez <email address hidden>
Date: Wed Apr 7 13:16:21 2021 +0000

    Make ARP protection commands compatible with "ebtables-nft"

    "nftables" compatible binary, "ebtables-nft", is not 100% compatible
    with the legacy API, as reported in LP#1922892.

    This patch fixes the following issues when using "ebtables-nft" (while
    keeping compatibility with legacy binary):
    - When a new chain is created, a default DROP rule is added at the end
      of the chain (append). This will prevent the error code 4 when the
      chain is listed.
    - The chain rules are added at the begining of the chain (insert),
      before the default DROP rule. This will prioritize the port rules.
    - The MAC rules are cleaned before the new ones are added. That will
      prevent the deletion of any new needed rule, now added after the
      deletion.
    - The "ebtables" command will retry on error code 4. This is the
      error returned when the chains are listed and no rule is present
      in a new created chain (reporeted in LP#1922892).

    This code is backwards compatible, that means it works with the legacy
    "ebtables" binary; this is currently installed in the Neutron CI [1].
    In order to test with the new binary, "ebtables-nft", two new CI jobs
    are added to the periodic queue [2].

    [1]https://github.com/openstack/neutron/blob/1ad9ca56b07ffdc9f7e0bc6a62af61961b9128eb/roles/legacy_ebtables/tasks/main.yaml
    [2]https://review.opendev.org/c/openstack/neutron/+/785144

    Closes-Bug: #1922892
    Related-Bug: #1508155
    Closes-Bug: #1938670

    Conflicts:
        neutron/tests/unit/plugins/ml2/drivers/linuxbridge/agent/test_arp_protect.py

    Change-Id: I9463b000f6f63e65aaf91d60b30f6c92c01e3baf
    (cherry picked from commit 0a931391d8990f3e654b4bfda24ae4119c609bbf)
    (cherry picked from commit fafa5dacd5057120562184a734e7345e7c0e9639)

If we use nft and use inet - then ipv4 and ipv6 can be mixed, thus we could get floating ip for ipv6.