Ubuntu
open-iscsi package

iSCSI Connection abruptly dies writing to Windows Server 2016 Target

Bug #1729000 reported by Scott
22
This bug affects 4 people
Affects Status Importance Assigned to Milestone
open-iscsi (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

Ubuntu Release: Ubuntu 16.04.2 LTS and Ubuntu 17.10
open-iscsi versions: 2.0.873+git.03b4b4500-14ubuntu3.3/3.4, 20.0.87-4ubuntu3

Issue: When writing data to an from Ubuntu to an iSCSI target hosted on Windows Server 2016, after a certain amount of data is written, the system will temporarily lock up. After the system unlocks, the writing process reports input/output errors, and the iSCSI drive is offlined, though erroneously remains mounted. This issue only happens with Windows Server 2016, and not 2012 R2 or older.

Below are the kernel logs detailed the faults that occur.

Oct 30 16:43:17 ubuntutest kernel: [ 511.488927] scsi host4: iSCSI Initiator over TCP/IP
Oct 30 16:43:17 ubuntutest kernel: [ 511.749528] scsi 4:0:0:0: Direct-Access MSFT Virtual HD 10.0 PQ: 0 ANSI: 5
Oct 30 16:43:17 ubuntutest kernel: [ 511.751907] sd 4:0:0:0: Attached scsi generic sg2 type 0
Oct 30 16:43:17 ubuntutest kernel: [ 511.753257] sd 4:0:0:0: [sdc] 209715200 512-byte logical blocks: (107 GB/100 GiB)
Oct 30 16:43:17 ubuntutest kernel: [ 511.753260] sd 4:0:0:0: [sdc] 4096-byte physical blocks
Oct 30 16:43:17 ubuntutest kernel: [ 511.754874] sd 4:0:0:0: [sdc] Write Protect is off
Oct 30 16:43:17 ubuntutest kernel: [ 511.754877] sd 4:0:0:0: [sdc] Mode Sense: 03 00 10 00
Oct 30 16:43:17 ubuntutest kernel: [ 511.755161] sd 4:0:0:0: [sdc] No Caching mode page found
Oct 30 16:43:17 ubuntutest kernel: [ 511.755176] sd 4:0:0:0: [sdc] Assuming drive cache: write through
Oct 30 16:43:17 ubuntutest kernel: [ 511.762845] sdc: sdc1 sdc2
Oct 30 16:43:17 ubuntutest kernel: [ 511.765822] sd 4:0:0:0: [sdc] Attached SCSI disk
Oct 30 16:43:17 ubuntutest ntfs-3g[17166]: Version 2015.3.14AR.1 integrated FUSE 28
Oct 30 16:43:17 ubuntutest ntfs-3g[17166]: Mounted /dev/sdc2 (Read-Write, label "New Volume", NTFS 3.1)
Oct 30 16:43:17 ubuntutest ntfs-3g[17166]: Cmdline options: rw
Oct 30 16:43:17 ubuntutest ntfs-3g[17166]: Mount options: rw,allow_other,nonempty,relatime,fsname=/dev/sdc2,blkdev,blksize=4096
Oct 30 16:43:17 ubuntutest ntfs-3g[17166]: Ownership and permissions disabled, configuration type 7
Oct 30 16:43:18 ubuntutest iscsid: Connection1:0 to [target: iqn.1991-05.com.microsoft:hosttest-testdisk-target, portal: *.*.*.*,3260] through [iface: default] is operational now
Oct 30 16:43:50 ubuntutest ntfs-3g[17166]: ntfs_mst_post_read_fixup_warn: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument
Oct 30 16:44:57 ubuntutest kernel: [ 611.756200] ------------[ cut here ]------------
Oct 30 16:44:57 ubuntutest kernel: [ 611.756207] WARNING: CPU: 2 PID: 1163 at /build/linux-5EyXrQ/linux-4.4.0/kernel/softirq.c:150 __local_bh_enable_ip+0x64/0x80()
Oct 30 16:44:57 ubuntutest kernel: [ 611.756208] Modules linked in: intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_hdmi snd_hda_codec_realtek kvm_intel snd_hda_codec_generic kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw snd_hda_intel gf128mul snd_hda_codec glue_helper hci_uart snd_hda_core ablk_helper btbcm rtsx_pci_ms snd_hwdep cryptd btqca snd_pcm btintel memstick input_leds snd_timer bluetooth ir_lirc_codec ir_xmp_decoder snd ir_rc6_decoder ir_rc5_decoder ir_sharp_decoder lirc_dev ir_nec_decoder ir_sony_decoder ir_mce_kbd_decoder ir_jvc_decoder ir_sanyo_decoder soundcore rc_rc6_mce ite_cir rc_core intel_lpss_acpi mei_me tpm_crb shpchp intel_lpss mei mac_hid acpi_als kfifo_buf industrialio acpi_pad ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 uas usb_storage hid_generic usbhid rtsx_pci_sdmmc i915_bpo e1000e intel_ips i2c_algo_bit ptp drm_kms_helper pps_core syscopyarea sysfillrect sysimgblt fb_sys_fops ahci rtsx_pci drm libahci wmi video pinctrl_sunrisepoint pinctrl_intel i2c_hid hid fjes
Oct 30 16:44:57 ubuntutest kernel: [ 611.756252] CPU: 2 PID: 1163 Comm: kworker/2:1H Not tainted 4.4.0-87-generic #110-Ubuntu
Oct 30 16:44:57 ubuntutest kernel: [ 611.756254] Hardware name: /NUC7i7BNB, BIOS BNKBL357.86A.0042.2017.0303.1854 03/03/2017
Oct 30 16:44:57 ubuntutest kernel: [ 611.756257] Workqueue: kblockd blk_timeout_work
Oct 30 16:44:57 ubuntutest kernel: [ 611.756258] 0000000000000086 d85973362ff8b0a1 ffff880857f2bc50 ffffffff813f9903
Oct 30 16:44:57 ubuntutest kernel: [ 611.756260] 0000000000000000 ffffffff81cba920 ffff880857f2bc88 ffffffff81081332
Oct 30 16:44:57 ubuntutest kernel: [ 611.756262] 0000000000000200 ffff880856aadbe8 ffff88085739d000 ffff880856aa9420
Oct 30 16:44:57 ubuntutest kernel: [ 611.756263] Call Trace:
Oct 30 16:44:57 ubuntutest kernel: [ 611.756267] [<ffffffff813f9903>] dump_stack+0x63/0x90
Oct 30 16:44:57 ubuntutest kernel: [ 611.756269] [<ffffffff81081332>] warn_slowpath_common+0x82/0xc0
Oct 30 16:44:57 ubuntutest kernel: [ 611.756271] [<ffffffff8108147a>] warn_slowpath_null+0x1a/0x20
Oct 30 16:44:57 ubuntutest kernel: [ 611.756273] [<ffffffff810858c4>] __local_bh_enable_ip+0x64/0x80
Oct 30 16:44:57 ubuntutest kernel: [ 611.756275] [<ffffffff81841a0e>] _raw_spin_unlock_bh+0x1e/0x20
Oct 30 16:44:57 ubuntutest kernel: [ 611.756279] [<ffffffffc01b6cb6>] __iscsi_conn_send_pdu+0x1c6/0x3e0 [libiscsi]
Oct 30 16:44:57 ubuntutest kernel: [ 611.756282] [<ffffffffc01b96ce>] iscsi_send_nopout+0xbe/0x100 [libiscsi]
Oct 30 16:44:57 ubuntutest kernel: [ 611.756285] [<ffffffffc01b99a7>] iscsi_eh_cmd_timed_out+0x297/0x2a0 [libiscsi]
Oct 30 16:44:57 ubuntutest kernel: [ 611.756287] [<ffffffff815c21c2>] scsi_times_out+0x62/0x250
Oct 30 16:44:57 ubuntutest kernel: [ 611.756289] [<ffffffff813d21af>] blk_rq_timed_out+0x1f/0x60
Oct 30 16:44:57 ubuntutest kernel: [ 611.756290] [<ffffffff813d229e>] blk_timeout_work+0xae/0x150
Oct 30 16:44:57 ubuntutest kernel: [ 611.756293] [<ffffffff8109a5e5>] process_one_work+0x165/0x480
Oct 30 16:44:57 ubuntutest kernel: [ 611.756295] [<ffffffff8109a94b>] worker_thread+0x4b/0x4c0
Oct 30 16:44:57 ubuntutest kernel: [ 611.756297] [<ffffffff8109a900>] ? process_one_work+0x480/0x480
Oct 30 16:44:57 ubuntutest kernel: [ 611.756298] [<ffffffff810a0c85>] kthread+0xe5/0x100
Oct 30 16:44:57 ubuntutest kernel: [ 611.756300] [<ffffffff810a0ba0>] ? kthread_create_on_node+0x1e0/0x1e0
Oct 30 16:44:57 ubuntutest kernel: [ 611.756302] [<ffffffff8184224f>] ret_from_fork+0x3f/0x70
Oct 30 16:44:57 ubuntutest kernel: [ 611.756304] [<ffffffff810a0ba0>] ? kthread_create_on_node+0x1e0/0x1e0
Oct 30 16:44:57 ubuntutest kernel: [ 611.756305] ---[ end trace e49e660a26e124c8 ]---
Oct 30 16:46:45 ubuntutest kernel: [ 720.110378] INFO: task kworker/u8:2:17150 blocked for more than 120 seconds.
Oct 30 16:46:45 ubuntutest kernel: [ 720.110396] Tainted: G W 4.4.0-87-generic #110-Ubuntu
Oct 30 16:46:45 ubuntutest kernel: [ 720.110409] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 30 16:46:45 ubuntutest kernel: [ 720.110425] kworker/u8:2 D ffff88085742b628 0 17150 2 0x00000000
Oct 30 16:46:45 ubuntutest kernel: [ 720.110430] Workqueue: writeback wb_workfn (flush-8:32)
Oct 30 16:46:45 ubuntutest kernel: [ 720.110432] ffff88085742b628 ffff88085403b008 ffff88085475ad00 ffff880857eb0000
Oct 30 16:46:45 ubuntutest kernel: [ 720.110433] ffff88085742c000 ffff88087ed96dc0 7fffffffffffffff ffff88085403af50
Oct 30 16:46:45 ubuntutest kernel: [ 720.110434] ffff88085403aef8 ffff88085742b640 ffffffff8183dda5 0000000000000000
Oct 30 16:46:45 ubuntutest kernel: [ 720.110435] Call Trace:
Oct 30 16:46:45 ubuntutest kernel: [ 720.110438] [<ffffffff8183dda5>] schedule+0x35/0x80
Oct 30 16:46:45 ubuntutest kernel: [ 720.110440] [<ffffffff81840ef5>] schedule_timeout+0x1b5/0x270
Oct 30 16:46:45 ubuntutest kernel: [ 720.110443] [<ffffffff813c52a3>] ? queue_unplugged+0xa3/0xb0
Oct 30 16:46:45 ubuntutest kernel: [ 720.110444] [<ffffffff8183d2d4>] io_schedule_timeout+0xa4/0x110
Oct 30 16:46:45 ubuntutest kernel: [ 720.110445] [<ffffffff813c8d41>] get_request+0x411/0x7a0
Oct 30 16:46:45 ubuntutest kernel: [ 720.110447] [<ffffffff810c4390>] ? wake_atomic_t_function+0x60/0x60
Oct 30 16:46:45 ubuntutest kernel: [ 720.110449] [<ffffffff813cb545>] blk_queue_bio+0x125/0x3e0
Oct 30 16:46:45 ubuntutest kernel: [ 720.110450] [<ffffffff813c984f>] generic_make_request+0x10f/0x2c0
Oct 30 16:46:45 ubuntutest kernel: [ 720.110450] [<ffffffff813c9a76>] submit_bio+0x76/0x170
Oct 30 16:46:45 ubuntutest kernel: [ 720.110453] [<ffffffff81247def>] submit_bh_wbc+0x12f/0x160
Oct 30 16:46:45 ubuntutest kernel: [ 720.110454] [<ffffffff81249dc5>] __block_write_full_page.constprop.39+0x115/0x3a0
Oct 30 16:46:45 ubuntutest kernel: [ 720.110455] [<ffffffff8124a3b0>] ? I_BDEV+0x20/0x20
Oct 30 16:46:45 ubuntutest kernel: [ 720.110456] [<ffffffff8124a3b0>] ? I_BDEV+0x20/0x20
Oct 30 16:46:45 ubuntutest kernel: [ 720.110457] [<ffffffff8124a195>] block_write_full_page+0x145/0x170
Oct 30 16:46:45 ubuntutest kernel: [ 720.110459] [<ffffffff8124aef8>] blkdev_writepage+0x18/0x20
Oct 30 16:46:45 ubuntutest kernel: [ 720.110461] [<ffffffff8119a713>] __writepage+0x13/0x30
Oct 30 16:46:45 ubuntutest kernel: [ 720.110462] [<ffffffff8119b20e>] write_cache_pages+0x1ee/0x510
Oct 30 16:46:45 ubuntutest kernel: [ 720.110463] [<ffffffff8119a700>] ? wb_position_ratio+0x1f0/0x1f0
Oct 30 16:46:45 ubuntutest kernel: [ 720.110464] [<ffffffff8119b581>] generic_writepages+0x51/0x80
Oct 30 16:46:45 ubuntutest kernel: [ 720.110466] [<ffffffff8119a85e>] ? __wb_calc_thresh+0x3e/0x140
Oct 30 16:46:45 ubuntutest kernel: [ 720.110467] [<ffffffff8119da1e>] do_writepages+0x1e/0x30
Oct 30 16:46:45 ubuntutest kernel: [ 720.110468] [<ffffffff8123e2d5>] __writeback_single_inode+0x45/0x340
Oct 30 16:46:45 ubuntutest kernel: [ 720.110469] [<ffffffff8123eae2>] writeback_sb_inodes+0x262/0x600
Oct 30 16:46:45 ubuntutest kernel: [ 720.110471] [<ffffffff8123ef0c>] __writeback_inodes_wb+0x8c/0xc0
Oct 30 16:46:45 ubuntutest kernel: [ 720.110472] [<ffffffff8123f263>] wb_writeback+0x253/0x310
Oct 30 16:46:45 ubuntutest kernel: [ 720.110473] [<ffffffff8123fa3d>] wb_workfn+0x24d/0x400
Oct 30 16:46:45 ubuntutest kernel: [ 720.110489] [<ffffffff8109a5e5>] process_one_work+0x165/0x480
Oct 30 16:46:45 ubuntutest kernel: [ 720.110490] [<ffffffff8109a94b>] worker_thread+0x4b/0x4c0
Oct 30 16:46:45 ubuntutest kernel: [ 720.110491] [<ffffffff8109a900>] ? process_one_work+0x480/0x480
Oct 30 16:46:45 ubuntutest kernel: [ 720.110499] [<ffffffff810a0c85>] kthread+0xe5/0x100
Oct 30 16:46:45 ubuntutest kernel: [ 720.110500] [<ffffffff810a0ba0>] ? kthread_create_on_node+0x1e0/0x1e0
Oct 30 16:46:45 ubuntutest kernel: [ 720.110504] [<ffffffff8184224f>] ret_from_fork+0x3f/0x70
Oct 30 16:46:45 ubuntutest kernel: [ 720.110505] [<ffffffff810a0ba0>] ? kthread_create_on_node+0x1e0/0x1e0
Oct 30 16:47:31 ubuntutest kernel: [ 765.782617] sd 4:0:0:0: Device offlined - not ready after error recovery
Oct 30 16:47:31 ubuntutest kernel: [ 765.782619] sd 4:0:0:0: Device offlined - not ready after error recovery
Oct 30 16:47:31 ubuntutest kernel: [ 765.782620] sd 4:0:0:0: Device offlined - not ready after error recovery
Oct 30 16:47:31 ubuntutest kernel: [ 765.782621] sd 4:0:0:0: Device offlined - not ready after error recovery
Oct 30 16:47:31 ubuntutest kernel: [ 765.782622] sd 4:0:0:0: Device offlined - not ready after error recovery
Oct 30 16:47:31 ubuntutest kernel: [ 765.782664] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.782665] sd 4:0:0:0: [sdc] tag#0 CDB: Write(10) 2a 00 01 f3 8f c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.782666] blk_update_request: I/O error, dev sdc, sector 32739264
Oct 30 16:47:31 ubuntutest kernel: [ 765.782683] Buffer I/O error on dev sdc2, logical block 4059384, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782702] Buffer I/O error on dev sdc2, logical block 4059385, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782718] Buffer I/O error on dev sdc2, logical block 4059386, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782734] Buffer I/O error on dev sdc2, logical block 4059387, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782749] Buffer I/O error on dev sdc2, logical block 4059388, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782765] Buffer I/O error on dev sdc2, logical block 4059389, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782782] Buffer I/O error on dev sdc2, logical block 4059390, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782797] Buffer I/O error on dev sdc2, logical block 4059391, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782814] Buffer I/O error on dev sdc2, logical block 4059392, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782830] Buffer I/O error on dev sdc2, logical block 4059393, lost async page write
Oct 30 16:47:31 ubuntutest kernel: [ 765.782984] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.782996] sd 4:0:0:0: [sdc] killing request
Oct 30 16:47:31 ubuntutest kernel: [ 765.782998] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.783087] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.783169] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.807952] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.808188] sd 4:0:0:0: [sdc] FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.808192] sd 4:0:0:0: [sdc] CDB: Write(10) 2a 00 01 f4 cf c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.808195] blk_update_request: I/O error, dev sdc, sector 32821184
Oct 30 16:47:31 ubuntutest kernel: [ 765.808467] sd 4:0:0:0: [sdc] tag#1 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.808469] sd 4:0:0:0: [sdc] tag#1 CDB: Write(10) 2a 00 01 f3 99 c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.808470] blk_update_request: I/O error, dev sdc, sector 32741824
Oct 30 16:47:31 ubuntutest kernel: [ 765.808749] sd 4:0:0:0: [sdc] tag#2 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.808751] sd 4:0:0:0: [sdc] tag#2 CDB: Write(10) 2a 00 01 f3 a3 c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.808752] blk_update_request: I/O error, dev sdc, sector 32744384
Oct 30 16:47:31 ubuntutest kernel: [ 765.809020] sd 4:0:0:0: [sdc] tag#3 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.809022] sd 4:0:0:0: [sdc] tag#3 CDB: Write(10) 2a 00 01 f3 ad c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.809023] blk_update_request: I/O error, dev sdc, sector 32746944
Oct 30 16:47:31 ubuntutest kernel: [ 765.809293] sd 4:0:0:0: [sdc] tag#4 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.809294] sd 4:0:0:0: [sdc] tag#4 CDB: Write(10) 2a 00 01 f3 b7 c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.809295] blk_update_request: I/O error, dev sdc, sector 32749504
Oct 30 16:47:31 ubuntutest kernel: [ 765.809565] sd 4:0:0:0: [sdc] tag#5 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.809566] sd 4:0:0:0: [sdc] tag#5 CDB: Write(10) 2a 00 01 f3 c1 c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.809567] blk_update_request: I/O error, dev sdc, sector 32752064
Oct 30 16:47:31 ubuntutest kernel: [ 765.809841] sd 4:0:0:0: [sdc] tag#6 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.809843] sd 4:0:0:0: [sdc] tag#6 CDB: Write(10) 2a 00 01 f3 cb c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.809844] blk_update_request: I/O error, dev sdc, sector 32754624
Oct 30 16:47:31 ubuntutest kernel: [ 765.810115] sd 4:0:0:0: [sdc] tag#7 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.810116] sd 4:0:0:0: [sdc] tag#7 CDB: Write(10) 2a 00 01 f3 d5 c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.810117] blk_update_request: I/O error, dev sdc, sector 32757184
Oct 30 16:47:31 ubuntutest kernel: [ 765.810397] sd 4:0:0:0: [sdc] tag#8 FAILED Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Oct 30 16:47:31 ubuntutest kernel: [ 765.810398] sd 4:0:0:0: [sdc] tag#8 CDB: Write(10) 2a 00 01 f3 df c0 00 0a 00 00
Oct 30 16:47:31 ubuntutest kernel: [ 765.810399] blk_update_request: I/O error, dev sdc, sector 32759744
Oct 30 16:47:31 ubuntutest kernel: [ 765.810777] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.811018] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.811247] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.811467] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.811668] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.811868] sd 4:0:0:0: rejecting I/O to offline device
Oct 30 16:47:31 ubuntutest kernel: [ 765.812070] sd 4:0:0:0: rejecting I/O to offline device

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in open-iscsi (Ubuntu):
status: New → Confirmed
Revision history for this message
Matt (matts80) wrote :

I'm seeing the same behaviour on 16.04.3 LTS.

A quick test using 14.04 suggests that the 3.16 kernels work without issue whilst 3.19 and later display the reported behaviour when writing to Windows Server 2016 iSCSI targets

Revision history for this message
Alex Tavcar (stfast386) wrote :

This solves a problem for me:

echo 1024 > /sys/block/sdc/queue/max_sectors_kb

Seems this is an old issue they did not resolved for Microsoft ISCSI targets.

https://forum.qnap.com/viewtopic.php?t=108970
https://groups.google.com/forum/m/#!topic/open-iscsi/6bp5FMM8DwI
...

I think kernel ../drivers/scsi/scsi_devinfo.c should be patched with MSFT as a disk HW vendor:

https://github.com/torvalds/linux/blob/master/drivers/scsi/scsi_devinfo.c

something like this:

56: scsi_static_device_list[] __initdata = {
....
     {"QNAP", "iSCSI Storage", NULL, BLIST_MAX_1024},
....
     {"MSFT", "Virtual HD", "10.0", BLIST_MAX_1024},
....
};

Revision history for this message
costinel (costinel) wrote :
Download full text (3.7 KiB)

I am getting similar message at boot but not to Windows target, instead to a DELL EQUALOGIC 100E
Symptom is system hang for a few seconds during boot then it works ok

[ 56.363592] ------------[ cut here ]------------
[ 56.363604] WARNING: CPU: 1 PID: 415 at /build/linux-hwe-edge-zpduMv/linux-hwe-edge-4.13.0/kernel/softirq.c:161 __local_bh_enable_ip+0x46/0x60
[ 56.363606] Modules linked in: mptctl mptbase ebtable_filter ebtables target_core_user uio tcm_qla2xxx qla2xxx nvme_fc nvme_fabrics nvme_core usb_f_tcm tcm_usb_gadget libcomposite udc_core vhost_scsi vhost iscsi_target_mod tcm_loop ib_srpt tcm_fc libfc scsi_transport_fc target_core_file target_core_iblock target_core_pscsi target_core_mod nfnetlink_queue nfnetlink_log nfnetlink bluetooth ecdh_generic dell_rbu bridge stp llc ip6table_filter ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables ipt_REJECT nf_reject_ipv4 xt_conntrack iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_CHECKSUM xt_tcpudp iptable_mangle ip_tables x_tables binfmt_misc zfs(POE) zunicode(POE) zavl(PO) icp(POE) zcommon(POE) znvpair(POE)
[ 56.363658] spl(OE) gpio_ich dcdbas ipmi_ssif joydev input_leds intel_powerclamp coretemp ipmi_si kvm_intel kvm irqbypass intel_cstate ipmi_devintf ipmi_msghandler acpi_power_meter shpchp mac_hid lpc_ich nfsd auth_rpcgss nfs_acl lockd grace sunrpc ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid i2c_algo_bit ttm drm_kms_helper syscopyarea sysfillrect sysimgblt uas fb_sys_fops drm usb_storage pata_acpi mpt3sas bnx2 raid_class scsi_transport_sas wmi
[ 56.363710] CPU: 1 PID: 415 Comm: kworker/1:1H Tainted: P OE 4.13.0-25-generic #29~16.04.2-Ubuntu
[ 56.363710] Hardware name: Dell Inc. PowerEdge T310/02P9X9, BIOS 1.12.0 09/06/2013
[ 56.363715] Workqueue: kblockd blk_timeout_work
[ 56.363716] task: ffff9a74690b8000 task.stack: ffffa83941174000
[ 56.363718] RIP: 0010:__local_bh_enable_ip+0x46/0x60
[ 56.363718] RSP: 0018:ffffa83941177d08 EFLAGS: 00010046
[ 56.363720] RAX: 0000000000000046 RBX: ffff9a746c6c9330 RCX: ffff9a745884c480
[ 56.363720] RDX: ffff9a746c6c93d0 RSI: 0000000000000200 RDI: ffffffffc0560cb3
[ 56.363721] RBP: ffffa83941177d08 R08: 0000000000000008 R09: 00000000000001b8
[ 56.363721] R10: 0000000000000000 R11: 0000000000007259 R12: ffff9a746c6c93c8
[ 56.363722] R13: ffff9a746a499000 R14: ffff9a746c6cac08 R15: ffff9a7468068dd8
[ 56.363723] FS: 0000000000000000(0000) GS:ffff9a7476e40000(0000) knlGS:0000000000000000
[ 56.363724] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 56.363724] CR2: 00007f2d20bb2f80 CR3: 000000022fc0a005 CR4: 00000000000206e0
[ 56.363725] Call Trace:
[ 56.363733] _raw_spin_unlock_bh+0x1e/0x20
[ 56.363738] __iscsi_conn_send_pdu+0x1c3/0x400 [libiscsi]
[ 56.363740] iscsi_send_nopout+0xb7/0x100 [libiscsi]
[ 56.363742] iscsi_eh_cmd_timed_out+0x287/0x290 [libiscsi]
[ 56.363745] scsi_times_...

Read more...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.