iscsi-targets don't quit session on shutdown
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
curtin |
Fix Released
|
Medium
|
Unassigned | ||
curtin (Ubuntu) |
Fix Released
|
Medium
|
Unassigned | ||
Xenial |
Confirmed
|
Medium
|
Unassigned | ||
Zesty |
Won't Fix
|
Medium
|
Unassigned | ||
Artful |
Fix Released
|
Medium
|
Unassigned |
Bug Description
1. Artful (MAAS Image)[a]
2. open-iscsi 2.0.874-4ubuntu1
3. On shutdown, all iscsi sessions to be stopped and unmounted
4. Iscsi stops but does not stop sessions and shutdown is blocked/hung
[^[[0;32m OK ^[[0m] Reached target Shutdown.
[ 125.666350] connection4:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4294921130, last ping 4294922432, now 4294923712
[ 125.922334] connection3:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4294921211, last ping 4294922496, now 4294923776
[ 126.178553] connection2:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4294921292, last ping 4294922560, now 4294923840
[ 126.434334] connection1:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4294921371, last ping 4294922624, now 4294923904
Note, previously released Artful MAAS images work fine:
http://
which contain
open-iscsi 2.0.873+
a. http://
Related branches
- Server Team CI bot: Approve (continuous-integration)
- Chad Smith: Approve
- Nish Aravamudan: Pending requested
-
Diff: 298 lines (+227/-5)4 files modifiedcurtin/block/iscsi.py (+35/-3)
curtin/commands/install.py (+7/-2)
tests/unittests/test_block_iscsi.py (+181/-0)
tests/vmtests/test_lvm_iscsi.py (+4/-0)
- Scott Moser (community): Approve
-
Diff: 7045 lines (+3340/-962)91 files modifiedcurtin/__init__.py (+2/-0)
curtin/block/__init__.py (+69/-20)
curtin/block/iscsi.py (+44/-3)
curtin/block/mdadm.py (+10/-6)
curtin/commands/apply_net.py (+34/-8)
curtin/commands/apt_config.py (+0/-9)
curtin/commands/curthooks.py (+197/-94)
curtin/commands/extract.py (+6/-0)
curtin/commands/install.py (+44/-4)
curtin/futil.py (+24/-1)
curtin/net/__init__.py (+106/-0)
curtin/reporter/handlers.py (+42/-0)
curtin/util.py (+137/-13)
debian/changelog (+33/-0)
doc/index.rst (+1/-0)
doc/topics/apt_source.rst (+9/-6)
doc/topics/config.rst (+18/-0)
doc/topics/curthooks.rst (+109/-0)
doc/topics/integration-testing.rst (+6/-0)
doc/topics/networking.rst (+2/-0)
doc/topics/overview.rst (+45/-47)
doc/topics/reporting.rst (+29/-0)
doc/topics/storage.rst (+2/-0)
examples/network-ipv6-bond-vlan.yaml (+2/-2)
examples/tests/bonding_network.yaml (+1/-4)
examples/tests/centos_basic.yaml (+2/-1)
examples/tests/centos_defaults.yaml (+91/-0)
examples/tests/journald_reporter.yaml (+20/-0)
examples/tests/network_alias.yaml (+29/-31)
examples/tests/network_static_routes.yaml (+10/-15)
examples/tests/network_v2_passthrough.yaml (+8/-0)
setup.py (+16/-2)
tests/unittests/helpers.py (+36/-0)
tests/unittests/test_apt_custom_sources_list.py (+3/-6)
tests/unittests/test_apt_source.py (+4/-7)
tests/unittests/test_basic.py (+4/-4)
tests/unittests/test_block.py (+20/-36)
tests/unittests/test_block_iscsi.py (+187/-18)
tests/unittests/test_block_lvm.py (+2/-2)
tests/unittests/test_block_mdadm.py (+10/-22)
tests/unittests/test_block_mkfs.py (+2/-2)
tests/unittests/test_clear_holders.py (+5/-5)
tests/unittests/test_commands_apply_net.py (+334/-0)
tests/unittests/test_commands_block_meta.py (+6/-19)
tests/unittests/test_commands_install.py (+22/-0)
tests/unittests/test_config.py (+6/-6)
tests/unittests/test_curthooks.py (+241/-57)
tests/unittests/test_feature.py (+5/-2)
tests/unittests/test_gpg.py (+4/-4)
tests/unittests/test_make_dname.py (+4/-4)
tests/unittests/test_net.py (+99/-24)
tests/unittests/test_partitioning.py (+4/-3)
tests/unittests/test_public.py (+54/-0)
tests/unittests/test_reporter.py (+29/-38)
tests/unittests/test_util.py (+201/-52)
tests/unittests/test_version.py (+7/-19)
tests/vmtests/__init__.py (+59/-7)
tests/vmtests/releases.py (+0/-15)
tests/vmtests/test_apt_config_cmd.py (+0/-4)
tests/vmtests/test_basic.py (+0/-13)
tests/vmtests/test_bcache_basic.py (+0/-4)
tests/vmtests/test_centos_basic.py (+35/-0)
tests/vmtests/test_iscsi.py (+0/-4)
tests/vmtests/test_journald_reporter.py (+52/-0)
tests/vmtests/test_lvm.py (+0/-9)
tests/vmtests/test_lvm_iscsi.py (+4/-4)
tests/vmtests/test_mdadm_bcache.py (+3/-59)
tests/vmtests/test_mdadm_iscsi.py (+4/-4)
tests/vmtests/test_multipath.py (+0/-4)
tests/vmtests/test_network.py (+202/-39)
tests/vmtests/test_network_alias.py (+33/-4)
tests/vmtests/test_network_bonding.py (+47/-22)
tests/vmtests/test_network_bridging.py (+77/-17)
tests/vmtests/test_network_enisource.py (+2/-8)
tests/vmtests/test_network_ipv6.py (+29/-4)
tests/vmtests/test_network_ipv6_enisource.py (+8/-6)
tests/vmtests/test_network_ipv6_static.py (+17/-5)
tests/vmtests/test_network_ipv6_vlan.py (+17/-5)
tests/vmtests/test_network_mtu.py (+61/-8)
tests/vmtests/test_network_static.py (+30/-4)
tests/vmtests/test_network_static_routes.py (+19/-6)
tests/vmtests/test_network_vlan.py (+40/-15)
tests/vmtests/test_nvme.py (+0/-9)
tests/vmtests/test_raid5_bcache.py (+0/-9)
tests/vmtests/test_simple.py (+0/-4)
tests/vmtests/test_uefi_basic.py (+0/-19)
tools/build-deb (+3/-1)
tools/curtainer (+14/-8)
tools/find-tgt (+54/-29)
tools/jenkins-runner (+47/-10)
tools/launch (+46/-7)
- curtin developers: Pending requested
-
Diff: 7045 lines (+3340/-962)91 files modifiedcurtin/__init__.py (+2/-0)
curtin/block/__init__.py (+69/-20)
curtin/block/iscsi.py (+44/-3)
curtin/block/mdadm.py (+10/-6)
curtin/commands/apply_net.py (+34/-8)
curtin/commands/apt_config.py (+0/-9)
curtin/commands/curthooks.py (+197/-94)
curtin/commands/extract.py (+6/-0)
curtin/commands/install.py (+44/-4)
curtin/futil.py (+24/-1)
curtin/net/__init__.py (+106/-0)
curtin/reporter/handlers.py (+42/-0)
curtin/util.py (+137/-13)
debian/changelog (+33/-0)
doc/index.rst (+1/-0)
doc/topics/apt_source.rst (+9/-6)
doc/topics/config.rst (+18/-0)
doc/topics/curthooks.rst (+109/-0)
doc/topics/integration-testing.rst (+6/-0)
doc/topics/networking.rst (+2/-0)
doc/topics/overview.rst (+45/-47)
doc/topics/reporting.rst (+29/-0)
doc/topics/storage.rst (+2/-0)
examples/network-ipv6-bond-vlan.yaml (+2/-2)
examples/tests/bonding_network.yaml (+1/-4)
examples/tests/centos_basic.yaml (+2/-1)
examples/tests/centos_defaults.yaml (+91/-0)
examples/tests/journald_reporter.yaml (+20/-0)
examples/tests/network_alias.yaml (+29/-31)
examples/tests/network_static_routes.yaml (+10/-15)
examples/tests/network_v2_passthrough.yaml (+8/-0)
setup.py (+16/-2)
tests/unittests/helpers.py (+36/-0)
tests/unittests/test_apt_custom_sources_list.py (+3/-6)
tests/unittests/test_apt_source.py (+4/-7)
tests/unittests/test_basic.py (+4/-4)
tests/unittests/test_block.py (+20/-36)
tests/unittests/test_block_iscsi.py (+187/-18)
tests/unittests/test_block_lvm.py (+2/-2)
tests/unittests/test_block_mdadm.py (+10/-22)
tests/unittests/test_block_mkfs.py (+2/-2)
tests/unittests/test_clear_holders.py (+5/-5)
tests/unittests/test_commands_apply_net.py (+334/-0)
tests/unittests/test_commands_block_meta.py (+6/-19)
tests/unittests/test_commands_install.py (+22/-0)
tests/unittests/test_config.py (+6/-6)
tests/unittests/test_curthooks.py (+241/-57)
tests/unittests/test_feature.py (+5/-2)
tests/unittests/test_gpg.py (+4/-4)
tests/unittests/test_make_dname.py (+4/-4)
tests/unittests/test_net.py (+99/-24)
tests/unittests/test_partitioning.py (+4/-3)
tests/unittests/test_public.py (+54/-0)
tests/unittests/test_reporter.py (+29/-38)
tests/unittests/test_util.py (+201/-52)
tests/unittests/test_version.py (+7/-19)
tests/vmtests/__init__.py (+59/-7)
tests/vmtests/releases.py (+0/-15)
tests/vmtests/test_apt_config_cmd.py (+0/-4)
tests/vmtests/test_basic.py (+0/-13)
tests/vmtests/test_bcache_basic.py (+0/-4)
tests/vmtests/test_centos_basic.py (+35/-0)
tests/vmtests/test_iscsi.py (+0/-4)
tests/vmtests/test_journald_reporter.py (+52/-0)
tests/vmtests/test_lvm.py (+0/-9)
tests/vmtests/test_lvm_iscsi.py (+4/-4)
tests/vmtests/test_mdadm_bcache.py (+3/-59)
tests/vmtests/test_mdadm_iscsi.py (+4/-4)
tests/vmtests/test_multipath.py (+0/-4)
tests/vmtests/test_network.py (+202/-39)
tests/vmtests/test_network_alias.py (+33/-4)
tests/vmtests/test_network_bonding.py (+47/-22)
tests/vmtests/test_network_bridging.py (+77/-17)
tests/vmtests/test_network_enisource.py (+2/-8)
tests/vmtests/test_network_ipv6.py (+29/-4)
tests/vmtests/test_network_ipv6_enisource.py (+8/-6)
tests/vmtests/test_network_ipv6_static.py (+17/-5)
tests/vmtests/test_network_ipv6_vlan.py (+17/-5)
tests/vmtests/test_network_mtu.py (+61/-8)
tests/vmtests/test_network_static.py (+30/-4)
tests/vmtests/test_network_static_routes.py (+19/-6)
tests/vmtests/test_network_vlan.py (+40/-15)
tests/vmtests/test_nvme.py (+0/-9)
tests/vmtests/test_raid5_bcache.py (+0/-9)
tests/vmtests/test_simple.py (+0/-4)
tests/vmtests/test_uefi_basic.py (+0/-19)
tools/build-deb (+3/-1)
tools/curtainer (+14/-8)
tools/find-tgt (+54/-29)
tools/jenkins-runner (+47/-10)
tools/launch (+46/-7)
Changed in open-iscsi (Ubuntu): | |
status: | New → Invalid |
Changed in curtin (Ubuntu): | |
assignee: | nobody → Ryan Harper (raharper) |
status: | New → In Progress |
Changed in curtin (Ubuntu): | |
assignee: | Ryan Harper (raharper) → nobody |
status: | In Progress → Fix Committed |
Changed in curtin: | |
status: | New → Fix Committed |
importance: | Undecided → Medium |
Changed in curtin (Ubuntu): | |
importance: | Undecided → Medium |
Changed in curtin (Ubuntu Xenial): | |
status: | New → Confirmed |
Changed in curtin (Ubuntu Zesty): | |
status: | New → Confirmed |
Changed in curtin (Ubuntu Xenial): | |
importance: | Undecided → Medium |
Changed in curtin (Ubuntu Zesty): | |
importance: | Undecided → Medium |
no longer affects: | open-iscsi (Ubuntu) |
no longer affects: | open-iscsi (Ubuntu Xenial) |
no longer affects: | open-iscsi (Ubuntu Zesty) |
no longer affects: | open-iscsi (Ubuntu Artful) |
Changed in curtin (Ubuntu Zesty): | |
status: | Confirmed → Won't Fix |
When I first saw this, I assumed that it was a result of ifupdown being
dropped from the image (bug 1698181). However, it seems (as of the
20170826 cloud image) that that has not yet happened.
Additionally, I've not been able to reproduce this with the dep8 tests
that are in open-iscsi. And those tests *do* a boot and shutdown
on a iscsi target root.
In an effort to describe what happened previously in zesty and how that
might have gone wrong, I wrote the info below.
In iscsi root environment, the initrams brings up the network connection,
and then connects to the iscsi target. It has to communicate with the target
root in order to acheive the following things:
a. make the network device not be bounced on the re-played coldplug after
pivot.
b. do not take the network device *down* on shutdown.
c. update DNS information from the result of a dhcp or other network
configuration.
Artful dropped ifupdown, so the methods which were present in xenial-zesty paste.ubuntu. com/25418884/
to accomplish those things have broken.
I suspect it is possible that 'a' and 'b' can be accomplished by somehow
informing systemd that the root mount was a '_netdev' (mount -O _netdev).
An untested patch for this might look like:
http://
Its possible that resolvconf integration will still work.
iscsi root works in xenial -> zesty like this: open-iscsi. interface. rules.d/ 70-iscsi- network- interface. rules [2] iscsi/net- interface- handler [3]. handler is then responsible for basically mocking the
* in its local-top [1] initramfs hooks, open-iscsi writes the interface
that the root device lives on to /run/initramfs/
* on interface add (coldplug) or remove (shutdown)
udev rules /lib/udev/
call /lib/open-
* net-interface-
interface up and interface down for the root network device. This
includes all of 'a', 'b', and 'c'.
For dns configuration, see the lines around 'command -v resolvconf' at [3].
[1] https:/ /git.launchpad. net/~usd- import- team/ubuntu/ +source/ open-iscsi/ tree/debian/ extra/initramfs .local- top?h=applied/ ubuntu/ zesty /git.launchpad. net/~usd- import- team/ubuntu/ +source/ open-iscsi/ tree/debian/ iscsi-network- interface. rules?h= applied/ ubuntu/ zesty /git.launchpad. net/~usd- import- team/ubuntu/ +source/ open-iscsi/ tree/debian/ net-interface- handler? h=applied/ ubuntu/ zesty
[2] https:/
[3] https:/