Comment 9 for bug 1928744

Hi,

In the last 3 days, I have been testing the other kernels. The steps which I took:

1. Boot to kernel-4.15.0-136.
2. Install kernel X, where X = 136,137,139,140:

 sudo apt-get install linux-image-4.15.0-X-generic linux-headers-4.15.0-X-generic linux-modules-4.15.0-X-generic linux-modules-extra-4.15.0-X-generic

3. Boot to kernel X.

4. Manually run:
 sudo depmod

5. Reboot.

4. Run the server's normal workload and see how it was performing.

I haven't seen disk performance issues with those kernels.
But there is one difference, which I don't know how to explain.

Before I raised this bug, we had the same disk performance issues on kernel-4.15.0-140.
I collected the list of kernel modules running on that kernel at that time:

Module Size Used by
joydev 24576 0
input_leds 16384 0
hid_generic 16384 0
usbhid 49152 0
hid 122880 2 usbhid,hid_generic
ip6_tables 28672 0
iptable_filter 16384 0
ip_tables 28672 1 iptable_filter
x_tables 40960 3 iptable_filter,ip6_tables,ip_tables
tcp_diag 16384 0
udp_diag 16384 0
inet_diag 24576 2 tcp_diag,udp_diag
cpuid 16384 0
btrfs 1155072 0
zstd_compress 163840 1 btrfs
xor 24576 1 btrfs
raid6_pq 114688 1 btrfs
ufs 77824 0
qnx4 16384 0
minix 32768 0
ntfs 102400 0
msdos 20480 0
jfs 188416 0
xfs 1204224 0
libcrc32c 16384 1 xfs
intel_rapl 20480 0
sb_edac 24576 0
x86_pkg_temp_thermal 16384 0
intel_powerclamp 16384 0
coretemp 16384 0
kvm_intel 217088 0
kvm 614400 1 kvm_intel
ipmi_ssif 32768 0
irqbypass 16384 1 kvm
crct10dif_pclmul 16384 0
crc32_pclmul 16384 0
ghash_clmulni_intel 16384 0
pcbc 16384 0
aesni_intel 188416 0
aes_x86_64 20480 1 aesni_intel
cdc_ether 16384 0
usbnet 45056 1 cdc_ether
mii 16384 1 usbnet
crypto_simd 16384 1 aesni_intel
glue_helper 16384 1 aesni_intel
cryptd 24576 3 crypto_simd,ghash_clmulni_intel,aesni_intel
intel_cstate 20480 0
intel_rapl_perf 16384 0
ipmi_si 57344 0
ipmi_devintf 20480 0
ipmi_msghandler 53248 3 ipmi_devintf,ipmi_si,ipmi_ssif
shpchp 36864 0
mei_me 40960 0
mei 94208 1 mei_me
lpc_ich 24576 0
acpi_power_meter 20480 0
acpi_pad 180224 0
mac_hid 16384 0
bonding 163840 0
lp 20480 0
parport 49152 1 lp
autofs4 40960 2
mgag200 45056 1
i2c_algo_bit 16384 1 mgag200
ttm 106496 1 mgag200
drm_kms_helper 172032 1 mgag200
syscopyarea 16384 1 drm_kms_helper
sysfillrect 16384 1 drm_kms_helper
tg3 167936 0
sysimgblt 16384 1 drm_kms_helper
fb_sys_fops 16384 1 drm_kms_helper
ptp 20480 1 tg3
drm 401408 4 drm_kms_helper,mgag200,ttm
pps_core 20480 1 ptp
megaraid_sas 143360 12
mxm_wmi 16384 0
wmi 24576 1 mxm_wmi

Now, after the testing we are running on kernel-4.15.0-140 (again) and this time I don't see the issue.
But the list of kernel module is different:
lsmod
Module Size Used by
intel_rapl 20480 0
sb_edac 24576 0
x86_pkg_temp_thermal 16384 0
intel_powerclamp 16384 0
coretemp 16384 0
kvm_intel 217088 0
ipmi_ssif 32768 0
kvm 614400 1 kvm_intel
irqbypass 16384 1 kvm
crct10dif_pclmul 16384 0
crc32_pclmul 16384 0
ghash_clmulni_intel 16384 0
pcbc 16384 0
aesni_intel 188416 0
aes_x86_64 20480 1 aesni_intel
crypto_simd 16384 1 aesni_intel
glue_helper 16384 1 aesni_intel
cryptd 24576 3 crypto_simd,ghash_clmulni_intel,aesni_intel
intel_cstate 20480 0
cdc_ether 16384 0
usbnet 45056 1 cdc_ether
input_leds 16384 0
joydev 24576 0
mii 16384 1 usbnet
intel_rapl_perf 16384 0
ipmi_si 57344 0
ipmi_devintf 20480 0
ipmi_msghandler 53248 3 ipmi_devintf,ipmi_si,ipmi_ssif
mei_me 40960 0
mei 94208 1 mei_me
lpc_ich 24576 0
shpchp 36864 0
mac_hid 16384 0
acpi_pad 180224 0
acpi_power_meter 20480 0
bonding 163840 0
lp 20480 0
parport 49152 1 lp
autofs4 40960 2
mgag200 45056 1
i2c_algo_bit 16384 1 mgag200
ttm 106496 1 mgag200
drm_kms_helper 172032 1 mgag200
syscopyarea 16384 1 drm_kms_helper
sysfillrect 16384 1 drm_kms_helper
sysimgblt 16384 1 drm_kms_helper
fb_sys_fops 16384 1 drm_kms_helper
tg3 167936 0
hid_generic 16384 0
ptp 20480 1 tg3
usbhid 49152 0
drm 401408 4 drm_kms_helper,mgag200,ttm
pps_core 20480 1 ptp
hid 122880 2 usbhid,hid_generic
megaraid_sas 143360 12
mxm_wmi 16384 0
wmi 24576 1 mxm_wmi

Perhaps those additional kernel modules which were loaded in the first time were affecting disk IOs?