direct dependencies of ubiquity should not be autoremovable

same for ubuntu mate 18.10

Status changed to 'Confirmed' because the bug affects multiple users.

This can be a regression from the minimize-manual work in livecd-rootfs. I'll do some digging shortly.

Rerunning the minimize manual script in an xubuntu image with some extra tracking I see the following chain causing it to be marked as automatically installed:

1. xubuntu-desktop Recommends indicator-messages
2. indicator-messages Recommends ubiquity-frontend-gtk (due to indicator-renderer Provides)
3. ubiquity-frontend-gtk Depends on ubiquity
4. ubiquity Depends on cryptsetup, Recommends lvm2

We later on remove ubiquity in cleanup, but don't mark any remaining autoremovable stuff manually installed.

I think we should run

apt-get autoremove -s | sed -nr 's#Remv (.*) \[.*#\1#p' | xargs apt-mark manual

after the install finished otherwise, this way get any packages marked as manually installed that would otherwise be subject to autoremoval.

The alternative here would be to run the minimize-manual script on the "install" chroot, and not the "live" chroot. But that requires further integration with live-build.

Is this expected to not be an issue in bionic as well? I'm hearing similar reports on people who reinstalled from an OEM image.

FYI I tested both generic bionic image and OEM image. Generic bionic image does not suffer this problem. It's only OEM image generated by Canonical OEM infrastructure that suffers it.

In generic bionic image (live cd mode), lvm and cryptsetup set mark as manual.
In oem image, it's not mark manual. Plan to mark so.

I have this bug on Xubuntu 18.10.

The following packages were automatically installed and are no longer required:
  cryptsetup cryptsetup-bin cryptsetup-initramfs cryptsetup-run
Use 'sudo apt autoremove' to remove them.

According to Julian, this does not (currently?) affect bionic, so marking invalid. If the "manually installed" minimization work is SRUed to bionic, this will need to be dealt with as part of that work but not before.

I have this bug on Ubuntu Mate 18.10. Repeated install of standard installation with LUKS LVM on HP 14-DF0023CL. After each install, apt autoremove deletes cryptsetup and lvm2 making computer fail to boot on next reboot. Attempted same install on Acer One netboot with exact same result.

Fix in https://git.launchpad.net/livecd-rootfs/commit/?id=b2db5bf3658f33dfaf3261bdc73f41be72acd374

Status changed to 'Confirmed' because the bug affects multiple users.

I just did run a "apt autoremove" without being careful enough and can't boot anylonger. After rebooting it just prints:

gave up waiting for suspend/resume device.
...
ALERT! /dev/mapper/xxx does not exist. Dropping to a shell!

Is there an easy way to unbreak a broken system like this one?

I think I've been successful recovering the installation. Please let me know if this makes sense or if there are any mistakes. People running into this, will be happy to find recovery instructions here.

See also: https://askubuntu.com/a/1120949/27088

# find root partition
sudo fdisk -l

# unencrypt partition
# Note: replace /dev/nvme0n1p3 with your disk
# replace "nvme0n1p3_crypt" with the correct name
# check by running this in chroot:
# $ cat /etc/crypttab | cut -f1 -d " "
# nvme0n1p3_crypt
sudo cryptsetup luksOpen /dev/nvme0n1p3 nvme0n1p3_crypt

# mount root partition
sudo vgscan
sudo vgchange -ay
sudo mount /dev/mapper/xubuntu--vg-root /mnt

# prepare chroot environment
sudo mount /dev/nvme0n1p2 /mnt/boot/
sudo mount -o rbind /dev/ /mnt/dev/
sudo mount -t proc proc /mnt/proc/
sudo mount -t sysfs sys /mnt/sys/

# make dns available in chroot
sudo cp /etc/resolv.conf /mnt/etc/resolv.conf

# enter chroot
sudo chroot /mnt /bin/bash

# re-install missing packages
apt install cryptsetup lvm2

# re-generate (this might be done also by apt in the step before, I'm not sure)
update-initramfs -u -k all

# Leave chroot environment
exit
# Write buffers to disk
sudo sync
# Unmount file systems
sudo umount /mnt/sys
sudo umount /mnt/proc
sudo umount /mnt/boot

This bug was fixed in the package livecd-rootfs - 2.566

---------------
livecd-rootfs (2.566) disco; urgency=medium

  [ Julian Andres Klode ]
  * Do not mark direct dependencies of ubiquity as auto installed. This caused
    cryptsetup to remain auto on the installed system (LP: #1801629)

  [ Tobias Koch ]
  * When the magic-proxy script could not find a valid InRelease file for the
    configured timestamp, it would fall back to serving the canonical version
    of it. This meant that builds would succeed, even though snap-shotting the
    repository failed.
    .
    This update makes the script return HTTP 404 when an InRelease by-hash
    link for a given combination of mirror, suite and timestamp cannot be
    found.

 -- Julian Andres Klode <email address hidden> Tue, 26 Feb 2019 08:56:02 +0100

This seems to have affected me even though I never ran autoclean. Maybe apt autocleans after I've used it to install things enough times? No idea. Was saved by this answer: https://askubuntu.com/a/1120949/69057

Fix is verified, daily images of xubuntu disco as of today do not produce the issue anymore :)

Julian verified this as not reproducing on disco now, with the livecd-rootfs change. Closing the ubiquity task as invalid.

This issue is still affecting Ubuntu MATE 19.10 beta. See the following for more detail:

  * https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1841672

This bug was fixed in the package ubuntu-mate-meta - 1.253

---------------
ubuntu-mate-meta (1.253) eoan; urgency=medium

  * Refreshed dependencies
  * Added cryptsetup to core, desktop (LP: #11841672, #1801629)
  * Added libnotify-bin to core-recommends, desktop-recommends

 -- Martin Wimpress <email address hidden> Wed, 25 Sep 2019 00:47:17 +0100

This bug has also affected me on Ubuntu

Sorry all. Something happened while I was typing in the comment.

The bug has also affected me on Ubuntu 18.04.3 LTS. I finally fixed it as described in comment #17.

Maybe unrelated but FYI. My installation is using UEFI but I need to keep the bios in legacy mode and then select Ubuntu (UEFI) to boot properly. If I set the BIOS to boot using UEFI directly, the fix still fails for me.

struck Linux mint 20.4 (focal, right?) - "just" lvm2 was removed though, not cryptsetup.

I definitely did not remove lvm2 by hand but usually run autoremove. Maybe I'll actually read the "these packages will be removed" next time o_O

My setup was encrypted root, encrypted raid5 and none of those of course mounted since lvm2 was no longer a thing. Easily fixed by liveUsb boot, unlock+chroot and apt install lvm2