snapd

Failed messages of sd-umount shown with reboot/shutdown

Bug #1920107 reported by Woodrow Shen
36
This bug affects 7 people
Affects Status Importance Assigned to Milestone
snapd
In Progress
Medium
Valentin David
finalrd (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

We were seeing the same sequence of errors heppened on KVM and ARM device running UC20:

"""
[12450.684692] sd-umoun[5922]: Failed to unmount /oldroot: Device or resource busy
[12450.693515] sd-remou[5923]: Failed to remount '/oldroot/run/mnt/data' read-only: Device or resource busy
[12450.704374] sd-umoun[5924]: Failed to unmount /oldroot/run/mnt/data: Device or resource busy
[12450.714225] sd-umoun[5925]: Failed to unmount /oldroot/run: Device or resource busy
[12450.726073] shutdown[1]: Could not detach loopback /dev/loop0: Device or resource busy
[12450.735481] shutdown[1]: Failed to finalize file systems, loop devices, ignoring
[12451.129165] reboot: Power down
"""

[Steps to reproduce]

$ sudo poweroff or $ sudo reboot

[Additional info]

http://cdimage.ubuntu.com/ubuntu-core/20/stable/current/ubuntu-core-20-amd64.img.xz

See original description
Revision history for this message
Woodrow Shen (woodrow-shen) wrote :

It looks like systemd issue so probably I will move this to systemd later if someone has any thoughts.

description: updated
Steve Langasek (vorlon)
tags: added: rls-ff-incoming
Michael Vogt (mvo)
affects: snapd (Ubuntu) → systemd (Ubuntu)
Revision history for this message
Steve Langasek (vorlon) wrote :

These messages appear to be coming from finalrd, and leading to an actual unclean shutdown.

Dan Streetman (ddstreet)
affects: systemd (Ubuntu) → finalrd (Ubuntu)
Brian Murray (brian-murray)
summary: - Failed messages of sd-umoun show with reboot/shutdown
+ Failed messages of sd-umount shown with reboot/shutdown
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

The way uc20 is booted, and how bind mounts are setup means that we do force unmount eventually.

It matters which snaps are installed, and which processes have memory mapped which files.

Many things should probably be marked for lazyunmount.

Please provide details why is held up /oldroot/run/mnt/data

For example output from fuser.

I do not believe any of the linked issues are of relevance here. It matters how snaps have been mounted, as they do get mounted circularly.

Changed in finalrd (Ubuntu):
status: New → Incomplete
Changed in snapd:
status: New → Incomplete
Revision history for this message
Woodrow Shen (woodrow-shen) wrote :

@Dimitri,

The issue can be reproduced by http://cdimage.ubuntu.com/ubuntu-core/20/stable/current/ubuntu-core-20-amd64.img.xz while I previously fetched the image there to test, so if you think it's snap relevant, I can check with snapd team.

Revision history for this message
hugh chao (hugh712) wrote :

I also can reproduce this issue in my amd64 machine.

Woodrow Shen (woodrow-shen)
description: updated
description: updated
Brian Murray (brian-murray)
tags: added: rls-ff-notfixing
removed: rls-ff-incoming
Revision history for this message
Balint Reczey (rbalint) wrote :

IMO it is out of scope for finalrd to unmount mounts which can't be cleanly unmounted.

Changed in finalrd (Ubuntu):
status: Incomplete → Opinion
status: Opinion → Invalid
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for snapd because there has been no activity for 60 days.]

Changed in snapd:
status: Incomplete → Expired
Revision history for this message
Ian Johnson (anonymouse67) wrote :

I continue to see these messages on all UC20 VM's I use, including the most recently released stable image on cdimage when booted in a VM

Changed in snapd:
status: Expired → Confirmed
importance: Undecided → Medium
Revision history for this message
Steve Langasek (vorlon) wrote :

> IMO it is out of scope for finalrd to unmount mounts
> which can't be cleanly unmounted.

Er, this is nonsense. The entire point of finalrd is to ensure we pivot to an initramfs after all other processes have been terminated, so that it is possible to cleanly unmount the root filesystem with no errors. For the root filesystem to be busy and failing to unmount is clearly a bug in either systemd or finalrd.

Changed in finalrd (Ubuntu):
status: Invalid → Confirmed
Revision history for this message
Maciej Borzecki (maciek-borzecki) wrote :

Valentin has 2 PRs up to address this on the snapd/core20 side: https://github.com/snapcore/snapd/pull/11119 and https://github.com/snapcore/core20/pull/124

Changed in snapd:
assignee: nobody → Valentin David (valentin.david)
status: Confirmed → In Progress
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

The choices in core20 pr are good.

systemd-shutdownd should attempt to make things private, and should attempt to move them out of the way, and count that as progress.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

However it already does all that.

Marks things private, tries to move them if they cannot be unmounted.

So not sure what the issue is.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.