| 2016-10-06 02:12:57 |
Ha Van Tu |
bug |
|
|
added bug |
| 2016-10-06 02:19:41 |
Ha Van Tu |
summary |
Using Netlink to improve performance of conntrack-tools |
[RFE] FWaaS: Using Netlink instead of conntrack-tools to improve performance |
|
| 2016-10-06 02:22:46 |
Cao Xuan Hoang |
bug |
|
|
added subscriber Cao Xuan Hoang |
| 2016-10-06 02:29:34 |
Ha Van Tu |
description |
Updating firewall with a large number of firewall rules needs improving performance.
When the Firewall is updated, the conntrack entries will be deleted by conntrack-tools ("conntrack -D" commands) with each rule associated with this firewall. The problem is inside a cloud system with a large number of firewall rules applied. Updating so much rules will lead to call a large number of subprocesses to implement the "conntrack -D" commands. That will consume the system resource and it will take a long time to finish updating firewall[1].
By using Netlink, we can call the subprocess only one time [6], so as to reduce the system resource and time to update firewall.
There should be some critical points need to be discussed:
- The standard Netlink interface for Python. There are 2 sources: [3] and [4] on github, but I don't know these resources are acceptable or not.
- The "conntrack -D" needs *root privilege*. My solution is make the Python module which performs deleting conntrack entries become Linux command (calling "python pythonmodule.py") and wrap by rootwrap.[5]
[1] With the system with Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory, it take 429s to finish removing 10.000 rules. The client is in [2]
[2] http://paste.openstack.org/show/584602/
[3] https://github.com/ei-grad/python-conntrack
[4] https://github.com/regit/pynetfilter_conntrack
[5] https://ask.openstack.org/en/question/60893/rootwrap-python-write-to-root-only-owned-file/
[6] http://paste.openstack.org/show/584603/ |
Updating firewall with a large number of firewall rules needs improving performance.
When the Firewall is updated, the conntrack entries will be deleted by conntrack-tools ("conntrack -D" commands) with each rule associated with this firewall. The problem is inside a cloud system with a large number of firewall rules applied. Updating so much rules will call a large number of subprocesses to implement the "conntrack -D" commands. That will consume the system resource and it will take a long time to finish updating firewall[1]. The client example to delete rules is in [2].
By using Netlink, we can call the subprocess only one time [3], so as to reduce the system resource and time to update firewall[4].
There should be some points need to be discussed:
- The standard Netlink interface for Python. There are 2 sources: [5] and [6] on github, but I don't know these resources are acceptable or not.
- The "conntrack -D" needs *root privilege*. My solution is making the Python module which performs deleting conntrack entries become Linux command (calling "python pythonmodule.py") and then wrapping it by rootwrap.[7]
[1] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "conntrack-tools", it take average 429s to finish removing 10.000 rules.
[2] http://paste.openstack.org/show/584602/
[3] http://paste.openstack.org/show/584603/
[4] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "Netlink", it take average 0.8s to finish removing 10.000 rules.
[5] https://github.com/ei-grad/python-conntrack
[6] https://github.com/regit/pynetfilter_conntrack
[7] https://ask.openstack.org/en/question/60893/rootwrap-python-write-to-root-only-owned-file/ |
|
| 2016-10-06 02:51:44 |
Brian Haley |
bug |
|
|
added subscriber Brian Haley |
| 2016-10-06 02:55:14 |
Ha Van Tu |
description |
Updating firewall with a large number of firewall rules needs improving performance.
When the Firewall is updated, the conntrack entries will be deleted by conntrack-tools ("conntrack -D" commands) with each rule associated with this firewall. The problem is inside a cloud system with a large number of firewall rules applied. Updating so much rules will call a large number of subprocesses to implement the "conntrack -D" commands. That will consume the system resource and it will take a long time to finish updating firewall[1]. The client example to delete rules is in [2].
By using Netlink, we can call the subprocess only one time [3], so as to reduce the system resource and time to update firewall[4].
There should be some points need to be discussed:
- The standard Netlink interface for Python. There are 2 sources: [5] and [6] on github, but I don't know these resources are acceptable or not.
- The "conntrack -D" needs *root privilege*. My solution is making the Python module which performs deleting conntrack entries become Linux command (calling "python pythonmodule.py") and then wrapping it by rootwrap.[7]
[1] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "conntrack-tools", it take average 429s to finish removing 10.000 rules.
[2] http://paste.openstack.org/show/584602/
[3] http://paste.openstack.org/show/584603/
[4] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "Netlink", it take average 0.8s to finish removing 10.000 rules.
[5] https://github.com/ei-grad/python-conntrack
[6] https://github.com/regit/pynetfilter_conntrack
[7] https://ask.openstack.org/en/question/60893/rootwrap-python-write-to-root-only-owned-file/ |
Updating firewall with a large number of firewall rules needs improving performance.
When the Firewall is updated, the conntrack entries will be deleted by conntrack-tools ("conntrack -D" commands) with each rule associated with this firewall. The problem is inside a cloud system with a large number of firewall rules applied. Updating so much rules will call a large number of subprocesses to implement the "conntrack -D" commands. That will consume the system resource and it will take a long time to finish updating firewall[1]. The client example to delete rules is in [2].
By using Netlink, we can call the subprocess only one time [3], so as to reduce the system resource and time to update firewall[4].
There should be some points need to be discussed:
- The standard Netlink interface for Python. There are 2 sources: [5] and [6] on github, but I don't know these resources are acceptable or not. If there is not standard one, we may need to write conntrack library for OpenStack from scratch.
- The "conntrack -D" needs *root privilege*. My solution is making the Python module which performs deleting conntrack entries become Linux command (calling "python pythonmodule.py") and then wrapping it by rootwrap.[7]
[1] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "conntrack-tools", it take average 429s to finish removing 10.000 rules.
[2] http://paste.openstack.org/show/584602/
[3] http://paste.openstack.org/show/584603/
[4] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "Netlink", it take average 0.8s to finish removing 10.000 rules.
[5] https://github.com/ei-grad/python-conntrack
[6] https://github.com/regit/pynetfilter_conntrack
[7] https://ask.openstack.org/en/question/60893/rootwrap-python-write-to-root-only-owned-file/ |
|
| 2016-10-06 03:33:39 |
Armando Migliaccio |
neutron: status |
New |
Confirmed |
|
| 2016-10-06 03:33:43 |
Armando Migliaccio |
neutron: importance |
Undecided |
Wishlist |
|
| 2016-10-19 04:21:50 |
Sarath Chandra Mekala |
bug |
|
|
added subscriber Sarath Chandra Mekala |
| 2016-10-21 10:48:00 |
Ha Van Tu |
description |
Updating firewall with a large number of firewall rules needs improving performance.
When the Firewall is updated, the conntrack entries will be deleted by conntrack-tools ("conntrack -D" commands) with each rule associated with this firewall. The problem is inside a cloud system with a large number of firewall rules applied. Updating so much rules will call a large number of subprocesses to implement the "conntrack -D" commands. That will consume the system resource and it will take a long time to finish updating firewall[1]. The client example to delete rules is in [2].
By using Netlink, we can call the subprocess only one time [3], so as to reduce the system resource and time to update firewall[4].
There should be some points need to be discussed:
- The standard Netlink interface for Python. There are 2 sources: [5] and [6] on github, but I don't know these resources are acceptable or not. If there is not standard one, we may need to write conntrack library for OpenStack from scratch.
- The "conntrack -D" needs *root privilege*. My solution is making the Python module which performs deleting conntrack entries become Linux command (calling "python pythonmodule.py") and then wrapping it by rootwrap.[7]
[1] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "conntrack-tools", it take average 429s to finish removing 10.000 rules.
[2] http://paste.openstack.org/show/584602/
[3] http://paste.openstack.org/show/584603/
[4] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "Netlink", it take average 0.8s to finish removing 10.000 rules.
[5] https://github.com/ei-grad/python-conntrack
[6] https://github.com/regit/pynetfilter_conntrack
[7] https://ask.openstack.org/en/question/60893/rootwrap-python-write-to-root-only-owned-file/ |
Updating firewall with a large number of firewall rules needs improving performance.
When the Firewall is updated, the conntrack entries will be deleted by conntrack-tools ("conntrack -D" commands) with each rule associated with this firewall. The problem is inside a cloud system with a large number of firewall rules applied. Updating so much rules will call a large number of subprocesses to implement the "conntrack -D" commands. That will consume the system resource and it will take a long time to finish updating firewall[1]. The client example to delete rules is in [2].
By using Netlink, we can call the subprocess only one time [3], so as to reduce the system resource and time to update firewall[4].
There should be some points need to be discussed:
- The standard Netlink interface for Python. There are 2 sources: [5] and [6] on github, but I don't know these resources are acceptable or not. If there is not standard one, we may need to write conntrack library for OpenStack from scratch.
- The "conntrack -D" needs *root privilege*. My solution is making the Python module which performs deleting conntrack entries become Linux command (calling "python pythonmodule.py") and then wrapping it by rootwrap.[7]
[1] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "conntrack-tools", it take average 429s to finish removing 10.000 rules.
[2] http://paste.openstack.org/show/584602/
[3] http://paste.openstack.org/show/584603/
[4] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "Netlink", it take average 33s to finish removing 10.000 rules.
[5] https://github.com/ei-grad/python-conntrack
[6] https://github.com/regit/pynetfilter_conntrack
[7] https://ask.openstack.org/en/question/60893/rootwrap-python-write-to-root-only-owned-file/ |
|
| 2016-11-03 02:26:12 |
Ha Van Tu |
description |
Updating firewall with a large number of firewall rules needs improving performance.
When the Firewall is updated, the conntrack entries will be deleted by conntrack-tools ("conntrack -D" commands) with each rule associated with this firewall. The problem is inside a cloud system with a large number of firewall rules applied. Updating so much rules will call a large number of subprocesses to implement the "conntrack -D" commands. That will consume the system resource and it will take a long time to finish updating firewall[1]. The client example to delete rules is in [2].
By using Netlink, we can call the subprocess only one time [3], so as to reduce the system resource and time to update firewall[4].
There should be some points need to be discussed:
- The standard Netlink interface for Python. There are 2 sources: [5] and [6] on github, but I don't know these resources are acceptable or not. If there is not standard one, we may need to write conntrack library for OpenStack from scratch.
- The "conntrack -D" needs *root privilege*. My solution is making the Python module which performs deleting conntrack entries become Linux command (calling "python pythonmodule.py") and then wrapping it by rootwrap.[7]
[1] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "conntrack-tools", it take average 429s to finish removing 10.000 rules.
[2] http://paste.openstack.org/show/584602/
[3] http://paste.openstack.org/show/584603/
[4] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "Netlink", it take average 33s to finish removing 10.000 rules.
[5] https://github.com/ei-grad/python-conntrack
[6] https://github.com/regit/pynetfilter_conntrack
[7] https://ask.openstack.org/en/question/60893/rootwrap-python-write-to-root-only-owned-file/ |
Updating firewall with a large number of firewall rules needs improving performance.
When the Firewall is updated, the conntrack entries will be deleted by conntrack-tools ("conntrack -D" commands) with each rule associated with this firewall. The problem is inside a cloud system with a large number of firewall rules applied. Updating so much rules will call a large number of subprocesses to implement the "conntrack -D" commands. That will consume the system resource and it will take a long time to finish updating firewall[1]. The client example to delete rules is in [2].
By using Netlink, we can call the subprocess only one time [3], so as to reduce the system resource and time to update firewall[4].
There should be some points need to be discussed:
- The standard Netlink interface for Python. There are 2 sources: [5] and [6] on github, but I don't know these resources are acceptable or not. If there is not standard one, we may need to write conntrack library for OpenStack from scratch.
- The "conntrack -D" needs *root privilege*. My solution is using oslo_privsep for instead.[7]
[1] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "conntrack-tools", it take average 429s to finish removing 10.000 rules.
[2] http://paste.openstack.org/show/584602/
[3] http://paste.openstack.org/show/584603/
[4] For example: With the developer system (Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz and 16GiB memory) and using "Netlink", it take average 33s to finish removing 10.000 rules.
[5] https://github.com/ei-grad/python-conntrack
[6] https://github.com/regit/pynetfilter_conntrack
[7] https://review.openstack.org/#/c/389654/ |
|
| 2017-03-15 20:53:06 |
Armando Migliaccio |
neutron: status |
Confirmed |
Triaged |
|
| 2017-03-16 00:53:08 |
Ha Van Tu |
bug |
|
|
added subscriber Cuong Nguyen |
| 2017-03-16 01:24:47 |
Ha Van Tu |
bug |
|
|
added subscriber Nguyen Phuong An |
| 2017-03-16 22:27:39 |
Armando Migliaccio |
neutron: milestone |
|
pike-1 |
|
| 2017-03-16 22:27:47 |
Armando Migliaccio |
neutron: assignee |
|
Ha Van Tu (tuhv) |
|
| 2017-03-16 22:27:51 |
Armando Migliaccio |
tags |
fwaas needs-attention rfe |
fwaas needs-attention rfe-approved |
|
| 2017-03-16 22:31:20 |
Armando Migliaccio |
tags |
fwaas needs-attention rfe-approved |
fwaas loadimpact |
|
| 2017-05-18 01:20:36 |
Armando Migliaccio |
neutron: milestone |
pike-1 |
pike-2 |
|
| 2017-05-30 09:42:47 |
OpenStack Infra |
neutron: status |
Triaged |
In Progress |
|
| 2017-05-30 09:42:47 |
OpenStack Infra |
neutron: assignee |
Ha Van Tu (tuhv) |
Cuong Nguyen (cuongnv) |
|
| 2017-07-14 08:21:04 |
OpenStack Infra |
neutron: assignee |
Cuong Nguyen (cuongnv) |
songminglong (songminglong) |
|
| 2017-07-14 08:32:05 |
songminglong |
neutron: assignee |
songminglong (songminglong) |
|
|
| 2017-07-17 02:52:28 |
OpenStack Infra |
neutron: assignee |
|
songminglong (songminglong) |
|
| 2017-07-17 03:14:46 |
songminglong |
neutron: assignee |
songminglong (songminglong) |
|
|
| 2017-07-17 09:22:51 |
OpenStack Infra |
neutron: assignee |
|
songminglong (songminglong) |
|
| 2017-07-17 09:23:01 |
songminglong |
neutron: assignee |
songminglong (songminglong) |
|
|
| 2017-07-25 07:01:22 |
OpenStack Infra |
neutron: assignee |
|
Cuong Nguyen (cuongnv) |
|
| 2017-08-01 21:47:17 |
Kevin Benton |
neutron: importance |
Wishlist |
Medium |
|
| 2017-11-28 04:47:49 |
OpenStack Infra |
neutron: assignee |
Cuong Nguyen (cuongnv) |
Cao Xuan Hoang (hoangcx) |
|
| 2018-02-18 02:24:37 |
OpenStack Infra |
neutron: assignee |
Cao Xuan Hoang (hoangcx) |
Brian Haley (brian-haley) |
|
| 2018-05-29 08:54:44 |
OpenStack Infra |
neutron: assignee |
Brian Haley (brian-haley) |
Slawek Kaplonski (slaweq) |
|
| 2018-08-31 07:38:00 |
Slawek Kaplonski |
neutron: status |
In Progress |
New |
|
| 2018-08-31 07:38:00 |
Slawek Kaplonski |
neutron: assignee |
Slawek Kaplonski (slaweq) |
|
|
| 2018-08-31 07:38:06 |
Slawek Kaplonski |
tags |
fwaas loadimpact |
fwaas loadimpact timeout-abandon |
|
| 2022-11-08 01:11:26 |
Brian Haley |
neutron: status |
New |
Won't Fix |
|
