"cryptsetup: unknown fstype, bad password or options?" error unlocking / decrypting LUKS volume at boot

I've gotten this message as well for Kubuntu Wily, but I didn't have issues unlocking it. I first get the prompt screen for cryptsetup to enter password, I then enter correct password for it, I then get: "Unknown Fstype, bad password or options?" Then it gives message "sda5_crypt setup successfully", and then loads up the user /password prompt screen before entering into desktop / Kubuntu Wily. Not sure why I would get "Unknown fstype, bad password or options?" message, when I entered correct password from get go lol.

Status changed to 'Confirmed' because the bug affects multiple users.

I had this after upgrading to wily four days ago. I get the error message as above for Johnny Rogers, but then the system hangs and will go no further... my workaround is to boot up in recovery mode which gives asks for my password then gives this message (about 20 times):

 remove ioctl on sda5_crypt failed: device or resource busy

 Then it lets me then boot up as usual.

I also ran into this problem. I was able to fix it by looking at my dmesg and realizing that the proprietary ati drivers don't work with the 4.X kernel. I had the ones from ATI installed so when it'd try and boot up it'd get past the encryption but then the driver would crash.

I solved this by uninstalling my ati drivers and reinstall/reconfiguring my xorg-server.

This also occurs on Ubuntu MATE 15.04 running cryptsetup 1.6.6-5ubuntu2 (versus the ubuntu1 in the original report).

I'm also experiencing this. I noticed the following in my /var/log/boot.log:

/run/lvm/lvmetad.socket: connect failed: No such file or directory
WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
Reading all physical volumes. This may take a while...
Found volume group "ubuntu-vg" using metadata type lvm2
/run/lvm/lvmetad.socket: connect failed: No such file or directory
WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
2 logical volume(s) in group "ubuntu-vg" now active
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy
Device sda5_crypt is still in use.

and then it boots normally.

After entering the password encryption sends me the Emergency mode. Here you enter "systemctl default" and can enter the desktop Ubuntu 15.10. Now I'm trying to fix the bug with "sudo dpkg --configure -a"

//Sorry for my bad english

I have the same issue as the OP and Johny Rogers. System boots successful and notifies me cryptsetup was successful but first it displays the unkown fs type, bad option password message. This started happening after upgrading lubuntu to wily.

I have the same issue as the OP on Ubuntu 15.10 Desktop AMD64. It spits out "cryptsetup: unknown fstype, bad password or options?" after the correct passphrase is entered then proceeds to boot normally. I thought it was just me, because I have a LUKS-encrypted LVM setup that might be different from what Ubuntu was expecting in a couple of ways.
1. I set it all up using the command line before going into ubiquity.
2. I set it up using the Ubuntu 14.04(.1, maybe) Desktop AMD64 LiveUSB over a year ago. (I did format the partitions containing / and /boot when installing 15.10.)

In case they help, here are my fstab and crypttab with comments removed:

/dev/mapper/kryptvg-root / ext4 errors=remount-ro 0 1
UUID=3a58960b-caa8-4573-8730-812bba9b0a69 /boot ext4 defaults 0 2
/dev/mapper/kryptvg-data /data ext4 defaults 0 2
/dev/mapper/kryptvg-swap none swap sw 0 0

krypt /dev/sda3 none luks

Hello, i hope my first post is helpful.

After some debugging i figured out where this error message "cryptsetup: unknown fstype, bad password or options?" in my case comes from.

The point is that i am having a LVM Partition which holds several logical volumes which need to be mounted (at least one of them as root partition). This partition seems not to be visible/available at this time.

I patched the script and the error is gone for me. So can someone please check and confirm that this helps?!
It would then be great to get this fixed for future versions because i do not want to use a self-patched version which will be overwritten with some new updates.

To sum up. This are the problems with that i think

1) "cryptsetup: unknown fstype, bad password or options?"
Problem: LVM Partitions not fully set up at the time the script tries to detect the File-System type.
Solution: Wait until all partitions fully set up and accessible

2) "device-mapper: remove ioctl on sda5_crypt failed: Device or resource busy"
This error is shown because of the Problem in "1)" because the File-System is not being detected (due to empty FSTYPE variable)

3) /run/lvm/lvmetad.socket: connect failed: No such file or directory + WARNING: Failed to connect to lvmetad. Falling back to internal scanning.

This warning comes from /etc/lvm/lvm.conf because in this configuration file "use_lvmetad" is activated and set to 1.
Having this enabled seems to be fine i think. The problem is that this daemon is not running at the time the lvm commands from the initramfs-tools are being executed.

I solved this "optical problem" of displaying this warnings by creating a hook, which set the use_lvmetad to "use_lvmetad = 0".

$ cat /etc/initramfs-tools/hooks/fix_lvm2_hook
#!/bin/sh

PREREQ="lvm2"

prereqs()
{
        echo "$PREREQ"
}

case $1 in
prereqs)
        prereqs
        exit 0
        ;;
esac

. /usr/share/initramfs-tools/hook-functions

if [ -e ${DESTDIR}/etc/lvm/lvm.conf ]; then
        # Replace use_lvmetad=1 with use_lvmetad=0
        sed -E 's/(^\s*use_lvmetad\s*=\s*)(1)(\s*.*$)/\10\3/' ${DESTDIR}/etc/lvm/lvm.conf
fi

The attachment "Patch for /usr/share/initramfs-tools/scripts/local-top/cryptroot" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

In my comment #11 i did some mistake in "3)"
I copied the wrong command for changing the use_lvmetad parameter temporarily. I missed the "-i" for sed.

To not confuse someone here is the corrected part:

# Replace use_lvmetad=1 with use_lvmetad=0
sed -i -E 's/(^\s*use_lvmetad\s*=\s*)(1)(\s*.*$)/\10\3/' ${DESTDIR}/etc/lvm/lvm.conf

This problem is caused by a race condition, which is why only some people are experiencing it.

At /usr/share/initramfs-tools/scripts/local-top/cryptsetup line 322, the setup_mapping() function in the cryptroot script calls the activate_vg() function, which runs 'lvm vgchange -a y'. At line 340, without waiting for udev to finish creating the device links for the newly activated volume group, 'blkid' is invoked to get the filesystem type of the logical volume. This fails if udev hasn't yet finished creating the device links, which causes the script to log "cryptsetup: unknown fstype, bad password or options?" at line 345 and run 'cryptsetup remove' at line 347. Fortunately 'cryptsetup remove' fails because the volume group was successfully activated (the crypt device is now busy), which allows the next iteration through the loop to succeed.

The attached patch fixes this bug for me.

Thanks for the analysis and patch, Richard. I think you've put your finger on the problem. However, I see that there's only one code path where we call vgchange without already calling udev_settle afterward A proper fix for this should eliminate unnecessary calls to udevadm settle that would slow down the boot.

Actually it looks like the calls to vgchange are unnecessary as a whole, because we have udev in the initramfs to do this for us; and we should be calling *just* udevadm settle.

Attached is a patch that I believe should do the right thing, though it's currently untested.

I uploaded Steve's patch to my PPA:
https://launchpad.net/~rhansen/+archive/ubuntu/bug1481536

I haven't tested it yet. The patch looks good to me, except I wonder whether the call to udev_settle() between the two blkids should be moved up to just after $cryptopen (see attached patch). Thoughts?

On Fri, Dec 04, 2015 at 03:51:50AM -0000, Richard Hansen wrote:
> I uploaded Steve's patch to my PPA:
> https://launchpad.net/~rhansen/+archive/ubuntu/bug1481536

> I haven't tested it yet. The patch looks good to me, except I wonder
> whether the call to udev_settle() between the two blkids should be moved
> up to just after $cryptopen (see attached patch). Thoughts?

Since there was not already a udev_settle call there, I assumed (but did not
verify) that cryptsetup operates synchronously, and only returns once the
device mapping has been set up. We should only add one there if someone
confirms that cryptsetup *doesn't* block until the device is created.

I believe the second udev_settle call that you removed is absolutely
required /at that point/. Even if cryptsetup synchronously ensures the
container device mapping has been set up, I don't think it ensures that the
events for the child device have been handled yet - or even added to the
event queue - so udev_settle could still return before vgchange -a has been
run.

But talking through this, I realize this also means that we still need to
call vgchange ourselves, because otherwise there's still a race condition
here even with udev_settle being called a little bit later.

I'll prepare an updated patch, and test it.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

>> The patch looks good to me, except I wonder whether the call to
>> udev_settle() between the two blkids should be moved up to just
>> after $cryptopen (see attached patch). Thoughts?
>
> Since there was not already a udev_settle call there, I assumed (but
> did not verify) that cryptsetup operates synchronously, and only
> returns once the device mapping has been set up.

You're probably right, given that nobody appears to be complaining
about seeing "cryptsetup: unknown error setting up device mapping".
But maybe there is a race condition there too and we've just been
lucky.

> We should only add one there if someone confirms that cryptsetup
> *doesn't* block until the device is created.

What is the cost of calling udev_settle when unnecessary? If it's
already settled, does it return immediately or does it sleep a bit?

> I believe the second udev_settle call that you removed is absolutely
> required /at that point/.

I don't think it's required at that point. As far as I can tell, the
udev_settle can go anywhere between $cryptopen and the invocation of
blkid after changing NEWROOT, depending on whether $cryptopen
guarantees that the crypt device exists by the time it exits. If so,
udev_settle can go anywhere but should go inside the LVM2_member 'if'
statement to avoid the overhead for non-LVM systems. If not,
udev_settle should go immediately after $cryptopen to avoid a race
with the "unknown error setting up device mapping" check.

> Even if cryptsetup synchronously ensures the container device
> mapping has been set up, I don't think it ensures that the events
> for the child device have been handled yet -

If it did, then this bug wouldn't exist.

> or even added to the event queue -

That's a good point. That would mean that a call to udev_settle
immediately after $cryptopen might be too soon, depending on how
udev_settle behaves when there are no events on the queue. However,
if there's a race by putting udev_settle immediately after $cryptopen,
there's still a race even if the call to udev_settle is kept lower
down in the LVM2_member 'if' statement.

> so udev_settle could still return before vgchange -a has been run.

Do you mean the vgchange -a in the udev rules? If so, I agree.

>
> But talking through this, I realize this also means that we still
> need to call vgchange ourselves, because otherwise there's still a
> race condition here even with udev_settle being called a little bit
> later.

If:
  * $cryptopen can return before the LV device device events are put
    on the queue, and
  * udev_settle immediately returns if there are no events on the
    queue
then it's possible that vgchange -a won't be run unless we run it
ourselves in this script.

However, if vgchange itself doesn't guarantee that the LV device events
are placed on the queue before it returns, then we still have a race.

I think the following are reasonable assumptions to make, but I would
love to have confirmation:
  1. when an encrypted device is unlocked, cryptsetup doesn't create
     the unlocked device node and symlink itself -- udev does that
  2. cryptsetup doesn't return until after the unlocked device event
     has been put on the udev q...

I'm attaching another revision of the patch (v3).

Changes from v2:

  * skip calls to udev_settle() if the device of interest already
    exists
  * don't assume the unlocked device will appear after udev_settle()
    (sleep until the device appears, if necessary)
  * wait for the LVM logical volume device to appear before getting
    the filesystem type
  * remove an unnecessary call to udev_settle()
  * clean up the password retry loop (move out logic that doesn't
    belong there, move in logic that does, simplify some logic)
  * delete commented-out code

You can see the above changes individually and with (somewhat)
descriptive commit messages in my GitHub repo:
https://github.com/rhansen/ubuntu-bug1481536-cryptroot

The patched package has been uploaded to my PPA:
https://launchpad.net/~rhansen/+archive/ubuntu/bug1481536

I have the same problem with Kubuntu 14.04.4 Trusty Tahr. The common LTS updates yesterday contained 2:1.6.7-Ovanir1~14.04(trusty) which replaced 2:1.6.1-1ubuntu1 (trusty). While boot I get the message "cryptsetup: unknown fstype, bad password or options?" although the password was correct. Nevertheless the system boots normally after that.
I solved the problem by forcing Kubuntu to use the old version of cryptsetup by using the Muon-package-manager .

FYI, lvm2 2.02.133-1ubuntu8 dropped the udev rule that runs 'lvm vgscan; lvm vgchange -a y', so cryptsetup must still run those commands in scripts/local-top/cryptroot. In other words, they can't be dropped as proposed in comment #15.

Attached is a new revision (v4) of the patch, this time targeted to
Xenial (16.04) instead of Wily (15.10).

Changes from v3 to v4:

  * restore activate_vg() (which runs 'lvm vgscan' and 'llvm vgchange
    -a y') because Xenial's lvm2 package no longer includes udev rules
    that run those commands (as of 2.02.133-1ubuntu8)
  * improvements to log messages
  * drop to a shell if cryptsetup successfully unlocks the device but
    the unlocked device doesn't appear
  * drop to a shell if an LVM logical volume is expected but doesn't
    appear

A patched package has been uploaded to my PPA:
https://launchpad.net/~rhansen/+archive/ubuntu/bug1481536

I'm on Ubuntu 16.04, added the ppa and updated my cryptsetup package (even did dpkg-reconfigure) but run into the same problem still. Do I need to do something else?

A a skilled friend of mine had a look at my issue and found out that the UUID of my bootable partition got mixed up. after setting things straight in a textfile where this gets looked up my problem was gone and I could boot normally again. Sorry for not being able to provide better terminology, maybe it helps anyway.

mray(mrayyyy) please could you list the steps your friend took to resolve the issue?