Comment 11 for bug 1928744

Thanks for the extensive testing, Mariusz. (I assume -137 doesn't have the performance problem?)

So, for comment #9 - the difference in loaded modules comes down to 20 additional modules that were loaded when performance was bad:

-ip6_tables 28672 0
-iptable_filter 16384 0
-ip_tables 28672 1 iptable_filter
-x_tables 40960 3 iptable_filter,ip6_tables,ip_tables
-tcp_diag 16384 0
-udp_diag 16384 0
-inet_diag 24576 2 tcp_diag,udp_diag
-cpuid 16384 0
-btrfs 1155072 0
-zstd_compress 163840 1 btrfs
-xor 24576 1 btrfs
-raid6_pq 114688 1 btrfs
-ufs 77824 0
-qnx4 16384 0
-minix 32768 0
-ntfs 102400 0
-msdos 20480 0
-jfs 188416 0
-xfs 1204224 0
-libcrc32c 16384 1 xfs

Seems they are mostly filesystem modules and iptables modules. It is possible that this could affect performance - though unlikely to cause performance issues simply by them being loaded. If they are causing the performance hit it would probably mean that some thing else is running not just additional modules loaded in memory.

So, if I understand correctly, -140 seemed initially fine, but after some time, performance degrades. That makes things a bit more interesting. Do you see those additional modules loaded when performance is degraded? Note that all those additional modules don't seem to have any external dependencies other than themselves - they depend only on each other. So it should be easily possible to unload them and see if that "fixes" the performance issue.

By the way, kernels -143 and -144 have been released since this bug was created. Maybe it is worthwhile to also try those kernels?