Neutron VPNaaS - growing memory consumption
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| neutron |
New
|
High
|
Unassigned | ||
Bug Description
I have problem with neutron vpnaas in Kolla-ansible (Victoria, from source) on Ubuntu 20.04.2 LTS, kernel 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64. Neutron client version 7.1.1, neutron server version 17.1.3.dev22.
After enabling vpnaas plugin everything was ok at first. I was able to create VPN connection and communication is correct. But after a week of running vpnaas (only one VPN connection created/working!) I’ve noticed, that neutron-vpnaas takes more and more memory. I have 5 processes on each controller (there should be always five? Or it is changing dynamically?):
42435 1545384 0.6 8.6 5802516 5712412 ? S Jun15 110:50 /var/lib/
42435 1545389 0.6 8.5 5735832 5645856 ? S Jun15 112:16 /var/lib/
42435 1545378 0.5 8.5 5734192 5643620 ? S Jun15 108:09 /var/lib/
42435 1545372 0.5 8.5 5731128 5641436 ? S Jun15 109:07 /var/lib/
42435 1545369 0.6 8.4 5637084 5547392 ? S Jun15 114:21 /var/lib/
now neutron_server takes over 27G of RAM on each controller:
neutron_server running 10.2 27.2G
and forced controllers to use swapfile.
After stopping all neutron containers and starting it again neutron takes a lot less memory:
neutron_server running 0.5 583M
but memory usage keeps growing (about 5-6 M every minute).
There is only one, bidirectional vpn connection between project networks in two regions.
There is no (or almost no) traffic on this vpn link.
Second region is on one host (kolla all-in-one) and situation looks the same- with vpn service enabled there is excessive Memory use (growing also about 5-6MB/ minute and forcing the system to use swapfile).
There is no big cpu usage (ofcourse also due to no traffic inside VPN)
It's the same case even if there's no vpn connection configured.
| tags: | added: vpnaas |
| Changed in neutron: | |
| importance: | Undecided → High |
| Adam Tomas (bkslash) wrote | #1 |
| Adam Tomas (bkslash) wrote | #2 |
I attach (mem.txt) output of
watch -n 60 "ps -eo pid,rss,
from first controller, where can be exactly seen how the memory consumption grows over time. L3-agent and the first neutron-server process uses almost the same memory size all the time, but 5 additional neutron-server processes takes more and more. Looking at the PIDs it seems that these 5 processes are started at the same time.