os-brick

first iscsi device attach on host fails with multipath enabled

Bug #1854159 reported by Lars on 2019-11-27

This bug affects 2 people

Affects		Status	Importance	Assigned to	Milestone
	os-brick	New	Undecided	Unassigned

Bug Description

On the first iscsi attachment on a host (e.g. newly deployed vm) the multipath device only gets attached with one path[1]. All following attachments work fine, it even works if I delete the first vm with that iscsi device and recreate it.

It looks like there is a race condition in the _connect_to_iscsi_portal[2] function wich gets called from the _connect_vol[3] function and this function is called for each path (4 paths) in a different thread[4].

I currently do not understand why this behaviour is only for the first time, but I was able to solve that problem by adding a random millisecond delay in front of that line[5]. It looks like on the first "iscsiadm -m node -T <target> -p <ip>:<port> --interface default --op new" call there shouldn't be a second call on the same time.

And as this thread never finishes it is stuck in the waiting loop forver: https://github.com/openstack/os-brick/blob/81f26f822d66c71c29ea25fd4158ac41fc162964/os_brick/initiator/connectors/iscsi.py#L733

[1]
[root@node01:~] 1 # multipath -ll
3624a9370d6c5f98b3e94412345678905 dm-3 PURE,FlashArray
size=20G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=50 status=active
`- 1:0:0:1 sdc 8:32 active ready running

[2]
https://github.com/openstack/os-brick/blob/81f26f822d66c71c29ea25fd4158ac41fc162964/os_brick/initiator/connectors/iscsi.py#L1021

[3]
https://github.com/openstack/os-brick/blob/81f26f822d66c71c29ea25fd4158ac41fc162964/os_brick/initiator/connectors/iscsi.py#L592

[4]
https://github.com/openstack/os-brick/blob/81f26f822d66c71c29ea25fd4158ac41fc162964/os_brick/initiator/connectors/iscsi.py#L721

[5]
https://github.com/openstack/os-brick/blob/81f26f822d66c71c29ea25fd4158ac41fc162964/os_brick/initiator/connectors/iscsi.py#L1037