Installing/Upgrading multipath-tools leads to multipathd.socket being inactive

Bug #2034471 reported by Mitchell Dzurick
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
multipath-tools (Ubuntu)
Status tracked in Mantic
Jammy
New
Undecided
Unassigned
Lunar
New
Undecided
Unassigned
Mantic
New
Undecided
Mitchell Dzurick

Bug Description

This is an issue with the systemd service not stating the dependency on the socket, but stating a start order with the socket. The fix here is to add

Wants=multipathd.socket

in multipathd.service which is a weak dependency, so the chance of issues arising from it is low, but helps to solve this particular issue.

[Impact]

On Upgrade/Install, the multipathd.socket states that it is inactive. An error is seen during install/upgrade that says

Job failed. See "journalctl -xe" for details.

This issue affects the following packages:

Lunar (0.8.8-1ubuntu2.1)
Jammy (0.8.8-1ubuntu1.22.04.2)
Focal (0.8.3-1ubuntu2.2)
- On upgrade
- On first install

Mantic (0.9.4-5ubuntu3)
- On first install

The issue is because of the order of the systemctl command and the systemd unit files.

multipathd.service has:
...
After=multipathd.socket
...
Also=multipathd.socket

multipath-tools.postinst has:
Jammy/Lunar/Focal (added by dh_systemd_start)
deb-systemd-invoke start 'multipathd.service' 'multipathd.socket' >/dev/null || true

Mantic (added by dh_installsystemd)
if [ -n "$2" ]; then
    _dh_action=restart
else
    _dh_action=start
fi
deb-systemd-invoke $_dh_action 'multipathd.service' 'multipathd.socket' >/dev/null || true

In other words,
systemctl stop multipathd.service multipathd.socket
systemctl start multipathd.service multipathd.socket

will fail

systemctl stop multipathd.service multipathd.socket
systemctl start multipathd.socket multipathd.service

will succeed

If we try to start the socket after the service, we get a failure. This is by design so multiple processes of the socket are not active at once.

[Test Cases]
Due to the nature of this bug/fix, multiple test cases are defined here:
1. Install multipath-tools when it is uninstalled
2. Upgrade multipath-tools to this version
3. Downgrade multipath-tools to this version
4. Reinstall multipath-tools when it is this version
5.
systemctl stop multipathd.service multipathd.socket
systemctl start multipathd.service multipathd.socket

All test cases should result in _no errors_ and both multipathd.service multipathd.socket are active

[Reproducer]
$ lxc launch ubuntu-daily:mantic m --vm
$ lxc shell m
# apt purge -y multipath-tools
# apt update -y
# apt install -y multipath-tools
# systemctl status multipathd.service multipathd.socket | grep Active -B2
● multipathd.service - Device-Mapper Multipath Device Controller
     Loaded: loaded (/lib/systemd/system/multipathd.service; enabled; preset: enabled)
     Active: active (running) since Wed 2023-09-13 19:30:56 UTC; 47s ago
--
○ multipathd.socket - multipathd control socket
     Loaded: loaded (/lib/systemd/system/multipathd.socket; enabled; preset: enabled)
     Active: inactive (dead)

Tags: server-todo

Related branches

Revision history for this message
Mitchell Dzurick (mitchdz) wrote :

This is because postinst attempts to start multipathd.socket while multipathd.service is active.

It's an issue with ordering of stopping/starting the service/socket.

Attempting to start the socket while the service is active will result in a failure due to how the socket will attempt to activate the service. To properly stop/start this socket/service the ordering is important, and we must do:

# systemctl stop multipathd.service multipathd.socket
# systemctl start multipathd.socket multipathd.service

Notice how the service is stopped first, but started last.

Inside of d/rules we have

dh_systemd_start -pmultipath-tools multipathd.service

Which results in the following line of postinst:
# Automatically added by dh_systemd_start/13.11.4ubuntu3
if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
        if [ -z "${DPKG_ROOT:-}" ] && [ -d /run/systemd/system ]; then
                systemctl --system daemon-reload >/dev/null || true
                deb-systemd-invoke start 'multipathd.service' 'multipathd.socket' >/dev/null || true
        fi
fi
# End automatically added section

Which postinst should start the socket first.

description: updated
Revision history for this message
Mitchell Dzurick (mitchdz) wrote (last edit ):

This is not specific to reinstalling the package, it will show up on upgrade as well.

description: updated
description: updated
Revision history for this message
Mitchell Dzurick (mitchdz) wrote :

Adding a dependency requirement in multipathd.service fixes this issue. This is similar to what is done with uuidd.service.

uuidd.service (as an example) has
Required=uuidd.socket

Which enforces a (hard) dependency requirement that systemd works with. Required= makes it so that if the socket does not start, the service will also not start[0].

I would suggest adding a weaker dependency[1] using Want= in the service file. This satisfies fixing the issue in this bug and has a lower chance of causing other issues.

[0] - https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Requires=

[1] - https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Wants=

summary: - Reinstalling multipath-tools leads to multipathd.socket being inactive
+ Installing/Upgrading multipath-tools leads to multipathd.socket being
+ inactive
description: updated
description: updated
summary: - Installing/Upgrading multipath-tools leads to multipathd.socket being
- inactive
+ [FFE] Installing/Upgrading multipath-tools leads to multipathd.socket
+ being inactive
summary: - [FFE] Installing/Upgrading multipath-tools leads to multipathd.socket
- being inactive
+ [FFE+SRU] Installing/Upgrading multipath-tools leads to
+ multipathd.socket being inactive
description: updated
summary: - [FFE+SRU] Installing/Upgrading multipath-tools leads to
- multipathd.socket being inactive
+ Installing/Upgrading multipath-tools leads to multipathd.socket being
+ inactive
description: updated
description: updated
description: updated
Changed in multipath-tools (Ubuntu):
assignee: nobody → Mitchell Dzurick (mitchdz)
description: updated
description: updated
tags: added: ubuntu-server
tags: added: server-todo
removed: ubuntu-server
Revision history for this message
Mitchell Dzurick (mitchdz) wrote :

Attempted to upstream[0] the fix of adding Wants=multipathd.socket in multipathd.service and was rejected, as upstream wants the ability to start the service without the socket starting.

[0] - https://github.com/opensvc/multipath-tools/pull/72

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.