Configure encrypted volumes destroys existing data

I wish I had found this bug report sooner... Please take a look at my workaround in
https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/285890

Quietly reinitializing the crypt partition is a release critical bug, IMHO.

There was an email to the bug squad mailing list about this issue (https://lists.ubuntu.com/archives/ubuntu-bugsquad/2011-April/003176.html) which references a thread on the dm-crypt mailing list (http://dir.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt) where a couple of people have also lost their encrypted partitions.

I certainly can see how running into this issue is a severe issue, however we're unfortunately past both UserInterfaceFreeze [1] and DocumentationStringFreeze [2] in Natty, and while hitting this is a PITA, it doesn't impact enough of our users to warrant a freeze exception at this time. We will address this issue in Oneiric, and I have assigned and targeted the bug to ensure it.

As mentioned in the linked Debian bug, selecting "Configure encrypted volumes" ought to make it possible to use your existing encrypted volumes after entry of a passphrase, rather than merely saying "is it OK if we destroy your data?".

 * Selecting "Configure encrypted volumes" in the alternate/server installer initialises encrypted volumes from scratch, and will destroy any encrypted data that was previously present. There is no way to reuse an existing encrypted volume at installation time at the moment, although we do consider this a bug that will be fixed in a future release. (Bug:420080)

(release-noted)

Retargetted to beta 1 and release noted for A3.

I'm making good progress on this, but have a little bit more to clean up before I can upload it. See:

  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=529343#21

This bug was fixed in the package partman-crypto - 47ubuntu3

---------------
partman-crypto (47ubuntu3) oneiric; urgency=low

  * Add an "Activate existing encrypted volumes" option to the
    partman-crypto main menu. If selected, this searches for existing
    volumes, and for each one prompts for its passphrase and attempts to
    open it; it then returns directly to the partitioning menu (closes:
    #451535, LP: #420080).
 -- Colin Watson <email address hidden> Fri, 09 Sep 2011 14:15:32 +0100

Hi Colin,

thanks for your work on this! Unfortunately, there's a problem with the nightly alternate Kubuntu oneiric images (installer can't load kernel modules), so I can only report that yes, the "Activate existing volumes" option is there and asks for the passphrase to the existing crypto partition. Opening the partition fails because of missing modules, I presume.

I'll test the next few nightlies and will report success or failure here.

Jochen

On Mon, Sep 12, 2011 at 08:00:03AM -0000, Jochen Gruse wrote:
> thanks for your work on this! Unfortunately, there's a problem with the
> nightly alternate Kubuntu oneiric images (installer can't load kernel
> modules), so I can only report that yes, the "Activate existing volumes"
> option is there and asks for the passphrase to the existing crypto
> partition. Opening the partition fails because of missing modules, I
> presume.

Hmm. I did see something like this a while back but thought I'd fixed
it. Could you extract /var/log/syslog from the running installer after
this failure (you can use "Save debug logs" from the main menu, or else
run 'anna-install openssh-client-udeb' and then scp it somewhere) and
attach it here?

The last lines in the above syslog at the time that the installer says "No kernel modules were found" are

Sep 12 09:53:49 cdrom-retriever: warning: File /cdrom/dists/oneiric/main/debian-installer/binary-amd64/Packages does not exist.
Sep 12 09:53:49 cdrom-retriever: warning: File /cdrom/dists/oneiric/restricted/debian-installer/binary-amd64/Packages does not exist.
Sep 12 09:53:49 anna[6902]: WARNING **: no packages matching running kernel 3.0.0-11-generic in archive

The mentioned files "Packages" do not exist, but there are files named "Packages.gz". Is that the source of the error? The installer expecting uncompressed files or these files being erroneously compressed?

No, this happened because images were built with out-of-sync kernel
versions. Could you update to the images I manually rebuilt this
morning and try again?

Hmm, I downloaded the image this morning. Well, I'll try again, just a mo'...

Alright, the images with timestamp "12-Sep-2011 09:59" are OK! Right now the installation is running. Crypto partition was found, activated, contained LVM PV found, LVM activated. Great! I'll report back when the installation is done.

There's still a problem. While installing the base system, no installable kernel was found in the configured APT sources. I've included the syslog as before.

Later this morning :-)

Hmmm?!? There's no newer image available. I downloaded oneiric-alternate-amd64.iso from http://cdimage.ubuntu.com/kubuntu/daily/current/, the Ubuntu oneiric images are even older. Some timezone problems?

Well, doesn't really matter. I'll download the new nightly build tomorrow morning and try that image, OK? Apart from that, partitioning worked like a charm, so I'm really happy. Thanks again, Colin!

Our mails crossed in the post. I meant later than the one you
originally tested.

I'll have to poke the kernel people about the secondary problem you
found.

Ah well, crossed mails explain it... B^)

Downloaded the current daily build (Kubuntu oneriric amd64, Timestamp "13-Sep-2011 05:20", size below 700MB, yay!). Partitioner working as expected, but still "No installable kernel found". A check from a shell shows already populated file systems below /target, so that part is definitely working. I'll try the next dailies too and will post here if successfull.

Should I open a new bug for a nonworking alternate install ISO? Is there anywhere to provide additional information like debug logs etc. if needed?

I've filed bug 848907 for the installation failure.

We can't quite assume that this works until we can get a complete installation, because it's important to make sure that your /etc/crypttab and /etc/fstab are correct. But the current problem is definitely unrelated to partman-crypto, as I reproduced it with a straightforward use-entire-disk installation.

Sorry for the delay, but you'll get a success as an excuse! B^)

Tried Kubuntu Oneiric alternate amd64 daily, Timestamp "15-Sep-2011 05:23". Partitioner works great, Oneiric installs, reboots and runs, /etc/fstab and /etc/crypttab are correct.

Two points I'd like to address:
* /etc/fstab uses "/dev/mapper/system-root2" instead of UUID=... This works OK, but is it what you expect it to be?
* GRUB installs OK, but reports an error after update-grub run. Choosing "Continue without installing a bootloader" finishes the installation and Kubunto reboots successfully into Oneiric, i.e. GRUB works. It's really the new installation of grub as it shows the menu to choose between 11.04 and 11.10.

I didn't find a fitting bug report. Want me to open a bug for that?

On Thu, Sep 15, 2011 at 02:03:27PM -0000, Jochen Gruse wrote:
> Tried Kubuntu Oneiric alternate amd64 daily, Timestamp "15-Sep-2011
> 05:23". Partitioner works great, Oneiric installs, reboots and runs,
> /etc/fstab and /etc/crypttab are correct.

Fantastic, thanks!

> Two points I'd like to address:
> * /etc/fstab uses "/dev/mapper/system-root2" instead of UUID=... This
> works OK, but is it what you expect it to be?

Yes, that's intentional. That's a stable name, and we only use UUIDs
when we don't have stable names.

> * GRUB installs OK, but reports an error after update-grub run.
> Choosing "Continue without installing a bootloader" finishes the
> installation and Kubunto reboots successfully into Oneiric, i.e. GRUB
> works. It's really the new installation of grub as it shows the menu
> to choose between 11.04 and 11.10.

That indeed sounds like a bug. Please file it separately, attaching
/var/log/installer/syslog and /var/log/installer/partman.

Opened bug 851704 for the GRUB problem.