Cannot configure 'cryptsetup luksFormat' at install time

Attaching a working patch; for reference.

Testing in Groovy Desktop ISO (daily build 2020-09-30)

Example: usage to specify luks header version,
as LUKS1; the default is LUKS2 on Focal/later.

$ sudo dpkg -i --force-depends --force-overwrite partman-crypto_101ubuntu4+luksopts1_amd64.udeb

Default (no option specified / no changes):
---

$ sudo cryptsetup luksDump /dev/vda4
LUKS header information
Version: 2
...

Option:
---

Preseed file:

"""
ubiquity partman-crypto/luksformat_options string \
  --type luks1
"""

$ sudo debconf-get partman-crypto/luksformat_options
--type luks1

$ grep luksFormat /var/log/partman
/usr/bin/autopartition-crypto: Additional options for luksFormat: '--type luks1'

$ sudo cryptsetup luksDump /dev/vda4
LUKS header information for /dev/vda4

Version: 1
...

In groovy and later, ubiquity vendorizes the udeb sources instead of including them from the archive at build time; so reassigning this to ubiquity since it appears you are targeting fixing this in the desktop installer.

Thanks for mentioning; I wasn't aware of this shipping method.

I'll update the patch for ubiquity/d-i/sources/partman-crypto.

Reported bug on Debian #971599

https://bugs.debian.org/971599

Submitted the change to the two repos (per ubiquity/d-i/source/README)
as I cannot pull into/upload to them. Hopefully this should be correct.

1) the stand-alone repo (~ubuntu-installer/partman-crypto) [1].
2) the bundled code in ubiquity (~ubuntu-installer/ubiquity) [2].

[1] https://code.launchpad.net/~mfo/partman-crypto/+git/partman-crypto/+merge/391766
[2] https://code.launchpad.net/~mfo/ubiquity/+git/ubiquity/+merge/391767

Tested this patch with netinstall by legacy installer(aka debian-installer).
Works well!
Just added into preseed:

d-i preseed/early_command string \
wget -q http://myhost.com/files/packages/partman-crypto_101ubuntu4+luksopts1_amd64.udeb ; \
udpkg -i partman-crypto_101ubuntu4\+luksopts1_amd64.udeb
d-i partman-crypto/luksformat_options string --type luks1

Merged all the required branches and pushed those out to hirsute.

This has been fixed on Hirsute; changing to Fix Released.

"""
ubiquity (21.04.3) hirsute; urgency=medium

  [ Mauricio Faria de Oliveira ]
  * Introduce preseed option partman-crypto/luksformat_options

  [ Łukasz 'sil2100' Zemczak ]
  * Automatic update of included source packages: partman-crypto
    101ubuntu5.

 -- Łukasz 'sil2100' Zemczak <email address hidden> Thu, 03 Dec 2020 15:21:45 +0100
"""

In order to fix this in Focal for the 20.04.3 point release:

Groovy is a no-op, since it has no partman-crypto, and its
ubiquity doesn't need the patch as there won't be new ISOs.
[thus: partman-crypto / Invalid; ubiquity / Won't Fix.]

Focal needs the patch on partman-crypto only (as it is not
yet shipped via ubiquity), then ubiquity installs it later.
[thus: partman-crypto / In Progress; ubiquity / Invalid.]

Actually in Focal partman-crypto is present in ubiquity
(in d-i/source/partman-crypto) so we'll change it again.

However, ubiquity in Focal doesn't yet use git subtrees;
it just `apt-get source` the d-i components, so we need:

1) patch partman-crypto/focal.
2) no-change-rebuild ubiquity/focal, so to pick it up.

I'll try that in a PPA.

Actually, step 2) is not a no-change-rebuild, but running 'debian/rules update', which adds to the changelog properly.

All looks good with the PPA build.

After patch/build partman-crypto, and updating its sources
in ubiquity (adding an apt source to the PPA, of course),
the ubiquitu .deb package includes the partman-crypto changes
in '/lib/partman/lib/crypto-base.sh' as expected.

Next Steps:

1) Upload partman-crypto to focal.
2) Wait for it to be available in focal-proposed.
3) Upload ubiquity to focal (gets focal-proposed.)

We could upload both and coordinate the time that
they are accepted and published (ubiquity must be
after partman-crypto), but plans might go wrong :)

In order to fix this for Focal / 20.04.2 point release:

Groovy is a no-op, because:

1) it has no partman-crypto as stand-alone src pkg anymore.
2) its ubiquity (w/ bundled partman-crypto) doesn't need a
   patch as there won't be new ISOs/Desktop install for it.
[thus, partman-crypto: Invalid / ubiquity: Won't Fix.]

Focal needs the fix in both, in two steps:

1) patch partman-crypto (stills exists as stand-alone src pkg.)
2) update ubiquity w/ the patched partman-crypto (in proposed.)
[thus, partman-crypto: In Progress / ubiquity: In Progress.]

Uploaded partman-crypto to focal.

After it makes it to focal-proposed,
updated ubiquity upload will follow.

Hello Mauricio, or anyone else affected,

Accepted partman-crypto into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/partman-crypto/101ubuntu4.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Verification done for focal-proposed (partman-crypto)
---

1) Booted the Ubuntu 20.04.1 Desktop ISO.
2) Launched a terminal, extracted the .udeb and copied over the crypto-base.sh file.
3) Performed installation to encrypted disk without the preseed option (default).
4) Performed installation to encrypted disk with the preseed option (testing).

On both cases, the installation finishes successfully, and the system can boot.
The LUKS header version is used as expected (LUKS2 by default, LUKS1 w/ option)

Details:
---

Launch terminal:

$ wget https://launchpad.net/ubuntu/+archive/primary/+files/partman-crypto_101ubuntu4.1_amd64.udeb
$ dpkg-deb -x partman-crypto_101ubuntu4.1_amd64.udeb udeb
$ sudo cp udeb/lib/partman/lib/crypto-base.sh /lib/partman/lib/crypto-base.sh

$ grep luksopts /lib/partman/lib/crypto-base.sh
 local mapping device cipher iv size pass luksopts
   luksopts="$RET"
   log "Additional options for luksFormat: '$luksopts'"
  log-output -t partman-crypto /sbin/cryptsetup -c $cipher-$iv -h $hash -s $size $luksopts luksFormat $device $pass

Launch ubiquity / Install Ubuntu 20.04.1 Desktop to Encrypted LVM device.
(In 'Installation type', select 'Erase disk and install Ubuntu', click in 'Advanced features', select 'Use LVM ...', select 'Encrypt ...', and move on to 'Install Now')

Launch terminal:

Without the option:

$ sudo debconf-get partman-crypto/luksformat_options
$

$ lsblk --ascii | grep -B1 crypt
`-vda6 252:6 0 8.8G 0 part
  `-vda6_crypt 253:0 0 8.8G 0 crypt

$ sudo cryptsetup luksDump /dev/vda6 | head -n2
LUKS header information
Version: 2

With the option:

$ sudo debconf-get partman-crypto/luksformat_options
--type luks1

$ lsblk --ascii | grep -B1 crypt
`-vda6 252:6 0 8.8G 0 part
  `-vda6_crypt 253:0 0 8.8G 0 crypt

$ sudo cryptsetup luksDump /dev/vda6 | head -n3
LUKS header information for /dev/vda6

Version: 1

Based on good test results with partman-crypto (comment #19)
now moving on with ubiquity (to include the updated sources).

Tests with the patched packaged built from a PPA are positive.
The results are identical to the reported for partman-crypto
(LUKS2/LUKS1 by default/as requested; installed system boots.)

Testing ubiquity from PPA:
-------------------------

Booted the Ubuntu 20.04.1 Desktop ISO.
Launch terminal:

$ sudo add-apt-repository ppa:mfo/lp1898129prop2
$ sudo apt install -y ubiquity

$ dpkg -s ubiquity | grep Version:
Version: 20.04.15.3

$ grep luksopts /lib/partman/lib/crypto-base.sh
 local mapping device cipher iv size pass luksopts
   luksopts="$RET"
   log "Additional options for luksFormat: '$luksopts'"
  log-output -t partman-crypto /sbin/cryptsetup -c $cipher-$iv -h $hash -s $size $luksopts luksFormat $device $pass

Without the option:

$ sudo debconf-get partman-crypto/luksformat_options

$

$ lsblk --ascii | grep -B1 crypt
`-vda6 252:6 0 8.8G 0 part
  `-vda6_crypt 253:0 0 8.8G 0 crypt

$ sudo cryptsetup luksDump /dev/vda6 | head -n2
LUKS header information
Version: 2

With the option:

$ sudo debconf-get partman-crypto/luksformat_options
--type luks1

$ lsblk --ascii | grep -B1 crypt
`-vda6 252:6 0 8.8G 0 part
  `-vda6_crypt 253:0 0 8.8G 0 crypt

$ sudo cryptsetup luksDump /dev/vda6 | head -n3
LUKS header information for /dev/vda6

Version: 1

Uploaded ubiquity to focal.

Two notes:

1) The 'debian/rules update' process picked up the new partman-crypto from focal-proposed, and also shim-signed.
Discussed this with Laney and it shouldn't be an issue, as the ISO .list file already shows the new version, so it's being used for a while now.
Nonetheless, I'll test a secure boot scenario w/ a VM and OVMF/UEFI.

2) There's new pyflakes in focal-proposed that cause ubiquity to FTBFS.
This is a simple change already in Groovy, so just picking that up too.

Thanks for including the FTBFS fix!

Hello Mauricio, or anyone else affected,

Accepted ubiquity into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubiquity/20.04.15.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Verification done for focal-proposed
---

All good, the package from -proposed works correctly in both scenarios -- without the option (ie, default behavior) and with the option (ie, opt-in behavior.)

Note: tested on VM with UEFI OVMF firmware with secure boot enabled (OVMF_CODE_4M.ms.fd), as shim-signed is also updated in the upload.
All good -- both scenarios install/boot to login screen w/ secboot.

Steps:
=====

On install, select Try Ubuntu, and launch Terminal.

$ sudo add-apt-repository 'deb http://archive.ubuntu.com/ubuntu focal-proposed main' && sudo apt install -y ubiquity && apt policy ubiquity
...
ubiquity:
  Installed: 20.04.15.3
  Candidate: 20.04.15.3
  Version table:
 *** 20.04.15.3 500
        500 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
...

$ grep -c luksopts /lib/partman/lib/crypto-base.sh
4

$ dmesg | grep -i secure
[ 0.000000] secureboot: Secure boot enabled
[ 0.000000] Kernel is locked down from EFI Secure Boot mode; see man kernel_lockdown.7
[ 0.008398] secureboot: Secure boot enabled

Move on with installer, select install to LVM/Encrypt.

Check on Terminal:

$ lsblk --ascii | grep -B1 crypt
`-vda3 252:3 0 8.8G 0 part
  `-vda3_crypt 253:0 0 8.8G 0 crypt

Without option (default)
---

$ sudo debconf-get partman-crypto/luksformat_options

$

$ sudo cryptsetup luksDump /dev/vda3 | head -n2
LUKS header information
Version: 2

With option (opt-in)
---

$ sudo debconf-get partman-crypto/luksformat_options
--type luks1
$

$ sudo cryptsetup luksDump /dev/vda3 | head -n3
LUKS header information for /dev/vda3

Version: 1

$ grep luks /var/log/partman
/usr/bin/autopartition-crypto: Additional options for luksFormat: '--type luks1'

The verification of the Stable Release Update for partman-crypto has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package partman-crypto - 101ubuntu4.1

---------------
partman-crypto (101ubuntu4.1) focal; urgency=medium

  * Add preseed option 'partman-crypto/luksformat_options' to
    provide more options for 'cryptsetup luksFormat' (LP: #1898129)
    - d/partman-crypto.templates: add preseed option.
    - lib/crypto-base.sh: check for, log, and use it.

 -- Mauricio Faria de Oliveira <email address hidden> Thu, 07 Jan 2021 16:51:37 -0300

This bug was fixed in the package ubiquity - 20.04.15.3

---------------
ubiquity (20.04.15.3) focal; urgency=medium

  * Automatic update of included source packages:
    partman-crypto 101ubuntu4.1, shim-signed 1.40.4. (LP: #1898129)
  * Fix FTBFS with new pyflakes (fixed in Groovy/20.10.4, no LP bug):
    autopilot/ubiquity_autopilot_tests/configs/english_label_conf.py:48:
    .format(...) has unused arguments at position(s): 0

 -- Mauricio Faria de Oliveira <email address hidden> Wed, 13 Jan 2021 11:42:57 -0300

The functionality is present in the desktop ISO daily build of 2021-01-21 [1] , per the ISO manifest file [2]:

    ubiquity 20.04.15.3

It has been verified to work correctly on testing of that ISO, with the following in a preseed file:

    ubiquity partman-crypto/luksformat_options string \
        --type luks1

cheers,
Mauricio

[1] https://cdimage.ubuntu.com/focal/daily-live/20210121/
[2] https://cdimage.ubuntu.com/focal/daily-live/20210121/focal-desktop-amd64.manifest