Soft lockup - cpu stuck - cifs_fattr_to_inode
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux-signed-hwe (Ubuntu) |
Confirmed
|
Undecided
|
Unassigned |
Bug Description
I'm getting a lot of CPU stuck error when writing on a CIFS (SMBv3) mount and need to reboot the machine(s) every time it happens. This error happens when I have a lot of small processes writing small files to the mount.
The CIFS mount options are
sec=krb5,
I'm running
Description: Ubuntu 16.04.5 LTS
Release: 16.04
on kernel 4.15.0-45-generic #48~16.04.1-Ubuntu SMP Tue Jan 29 18:03:48 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Trace is:
[ 5668.105623] CPU: 20 PID: 12828 Comm: action-suid Tainted: G D I L 4.15.0-45-generic #48~16.04.1-Ubuntu
[ 5668.105623] Hardware name: HP ProLiant DL360 G7, BIOS P68 08/16/2015
[ 5668.105626] RIP: 0010:native_
[ 5668.105626] RSP: 0018:ffffaab55b
[ 5668.105628] RAX: 0000000000000000 RBX: ffff8f91fff5c358 RCX: ffff8f893bca3440
[ 5668.105629] RDX: ffff8f925f9a3440 RSI: 0000000000380101 RDI: ffff8f91fff5c3e0
[ 5668.105629] RBP: ffffaab55b5039b8 R08: 0000000000540000 R09: 0000000000000000
[ 5668.105631] R10: ffff8f92347fa000 R11: ffffffffc086d420 R12: ffff8f91fff5c3e0
[ 5668.105632] R13: ffff8f91fff5c2b0 R14: ffff8f91fff5c310 R15: ffffaab55b503a60
[ 5668.105633] FS: 00007ff93663370
[ 5668.105634] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 5668.105635] CR2: 00007f25bcfe3750 CR3: 00000008f5760003 CR4: 00000000000206e0
[ 5668.105636] Call Trace:
[ 5668.105638] _raw_spin_
[ 5668.105651] cifs_fattr_
[ 5668.105663] cifs_get_
[ 5668.105675] cifs_revalidate
[ 5668.105687] cifs_revalidate
[ 5668.105697] cifs_d_
[ 5668.105699] lookup_
[ 5668.105701] ? __inode_
[ 5668.105702] walk_component+
[ 5668.105704] ? path_init+
[ 5668.105705] path_lookupat+
[ 5668.105707] filename_
[ 5668.105709] ? __check_
[ 5668.105711] ? strncpy_
[ 5668.105713] user_path_
[ 5668.105714] ? user_path_
[ 5668.105715] vfs_statx+0x77/0xe0
[ 5668.105717] SYSC_newlstat+
[ 5668.105719] SyS_newlstat+
[ 5668.105720] do_syscall_
[ 5668.105722] entry_SYSCALL_
[ 5668.105723] RIP: 0033:0x7ff936143c85
[ 5668.105724] RSP: 002b:00007ffd8b
[ 5668.105725] RAX: ffffffffffffffda RBX: 000055f5e961e880 RCX: 00007ff936143c85
[ 5668.105726] RDX: 00007ffd8bacad30 RSI: 00007ffd8bacad30 RDI: 000055f5e961d880
[ 5668.105727] RBP: 00007ffd8bacadf0 R08: 31383130325f7374 R09: 7473616d5f383132
[ 5668.105728] R10: 65343237365f7265 R11: 0000000000000246 R12: 000055f5e961d880
[ 5668.105728] R13: 000055f5e961d8cd R14: 000055f5e961d846 R15: 000055f5e961d86d
[ 5668.105729] Code: 12 48 c1 ea 0c 83 e8 01 83 e2 30 48 98 48 81 c2 40 34 02 00 48 03 14 c5 c0 c6 3a 82 48 89 0a 8b 41 08 85 c0 75 09 f3 90 8b 41 08 <85> c0 7
4 f7 4c 8b 09 4d 85 c9 74 08 41 0f 18 09 eb 02 f3 90 8b
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-
ProcVersionSign
Uname: Linux 4.15.0-45-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.18
Architecture: amd64
Date: Wed Feb 20 13:56:05 2019
SourcePackage: linux-signed-hwe
UpgradeStatus: No upgrade log present (probably fresh install)
Still happening on mainline kernel 4.20.11
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018937] watchdog: BUG: soft lockup - CPU#23 stuck for 22s! [cp:31557] iscsi sunrpc autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear radeon i2c_algo_bit ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect crc32_pclmul sysimgblt ghash_clmulni_intel fb_sys_fops drm drm_panel_ orientation_ quirks aesni_intel cfbfillrect cfbimgblt cfbcopyarea aes_x86_64 crypto_simd fb psmouse cryptd fbdev glue_helper hpsa i2c_core bnx2 scsi_transport_sas 042011- generic #201902200535 queued_ spin_lock_ slowpath+ 0x60/0x1a0 b439c8 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 0(0000) GS:ffff9e233bac 0000(0000) knlGS:000000000 0000000 lock+0x1f/ 0x30 safe_to_ change+ 0x3c/0xc0 [cifs] to_inode+ 0x15e/0x5a0 [cifs] inode_info+ 0x16d/0xb60 [cifs] 0x3a3/0x9f0 [cifs]
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018938] Modules linked in: overlay cmac nls_utf8 cifs ccm fscache ipmi_ssif intel_powerclamp coretemp kvm_intel kvm irqbypass gpio_ich acpi_power_meter intel_cstate hpilo ipmi_si i7core_edac ipmi_devintf ipmi_msghandler lpc_ich serio_raw mac_hid ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018965] CPU: 23 PID: 31557 Comm: cp Tainted: G D I L 4.20.11-
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018966] Hardware name: HP ProLiant DL360 G7, BIOS P68 08/16/2015
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018969] RIP: 0010:native_
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018970] Code: 6e f0 0f ba 2f 08 0f 92 c0 0f b6 c0 c1 e0 08 89 c2 8b 07 30 e4 09 d0 a9 00 01 ff ff 75 48 85 c0 74 0e 8b 07 84 c0 74 08 f3 90 <8b> 07 84 c0 75 f8 b8 01 00 00 00 5d 66 89 07 c3 8b 37 81 fe 00 01
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018971] RSP: 0018:ffffab3356
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018973] RAX: 0000000000540101 RBX: ffff9e173e960000 RCX: 000000000a12a9c6
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018974] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9e1a3334c840
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018974] RBP: ffffab3356b439c8 R08: ffff9e2334ca5800 R09: ffff9e2334ca5100
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018975] R10: ffffddca47d5e400 R11: ffffffffc0a25fc0 R12: ffff9e1a3334c840
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018976] R13: 000000000a12a9c6 R14: ffff9e173e960000 R15: ffff9e1a35ef5300
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018978] FS: 00007f608522280
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018978] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018979] CR2: 00007f60846fb6e0 CR3: 00000011f159a004 CR4: 00000000000206e0
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018980] Call Trace:
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018983] _raw_spin_
Feb 22 16:31:12 imk-node01 kernel: [ 1465.018999] is_size_
Feb 22 16:31:12 imk-node01 kernel: [ 1465.019013] cifs_fattr_
Feb 22 16:31:12 imk-node01 kernel: [ 1465.019027] cifs_get_
Feb 22 16:31:12 imk-node01 kernel: [ 1465.019041] cifs_open+
Feb 22 16:31:12 imk-node01 kernel: [ 1465.019056] ? cifsF...