[Ubuntu Server 20.04 LTS]: Failed Install (subiquity...install_fail/add_info) during partitioning

Attached: subiquity-info.log

Attached: subiquity-dubug.log

Hi, thanks for the report. Unfortunately the logs you've provided don't provide much help. Can you upload the crash file from /var/crash? It should be saved to the install media too.

Are you being offered an installer update as part of the install? Do you get the error if you take it?

Attached Crash File

Yes, I am being offered an installer update as part of the install.
Yes, I do get the error if I use the update

I also get the error if I don't update.

I attached the crash file in my comment above.

This seems to be related to:

https://bugs.launchpad.net/ubuntu/+source/subiquity/+bug/1875560

https://bugs.launchpad.net/curtin/+bug/1875903

Unless something else is going on with my installation, this bug hasn't been fixed...

Congratulations, you have found a new problem :)

Will look in my morning.

No worries. Thanks very much :)

From the crash:

Verifying /dev/sda2 exists
 Running command ['sfdisk', '--json', '/dev/sda'] with allowed return codes [0] (capture=True)
 Verifying /dev/sda2 size, expecting 86564143104 bytes, found 86564143104 bytes
 Device /dev/sda2 ptable entry: {
  "node": "/dev/sda2",
  "size": 169070592,
  "start": 61442048,
  "type": "83"
 }
 Verifying /dev/sda2 partition flag, expecting linux, found None
 An error occured handling 'partition-sda2': RuntimeError - Verifying /dev/sda2 partition flag, expecting linux, found None

Curtin has a logic bug here for msdos ptable and primary partitions; we should lookup the type found in ptable_uuid_to_flag_entry().

Status changed to 'Confirmed' because the bug affects multiple users.

This bug is fixed with commit cb60d8a7 to curtin on branch master.
To view that commit see the following URL:
https://git.launchpad.net/curtin/commit/?id=cb60d8a7

Thanks for fixing this @raharper!
How can I make use of this so I can finally successfully run the installation?
Do I just need to select the "Update Installer" option again?

@Michael

I'll sync with mwhudson, subiquity will need to pull in the fix and release an update to edge, once that's done you can select the Update Installer option.

This bug was fixed in the package curtin - 20.1-0ubuntu1

---------------
curtin (20.1-0ubuntu1) groovy; urgency=medium

  * New upstream release.
    - Release 20.1 (LP: #1880741)
    - Handle multiple separators which were found in TestAllindata vmtest
    - verify_ptable_flag: dos primary partitions use ptable_uuid map for flag
      (LP: #1878890)
    - net_meta: add disabled mode to skip writing any network config
      [Lucas Moura]
    - vmtest: trigger guest panic to fail fast
    - Replace grub-shell-helper with install_grub command
    - vmtest-sync-images: update the URL of the maas streams [Paride Legovini]
    - Replace references to old team manager with new team manager
      [James Falcon]
    - tox: pin flake8 to version and add a tip-flake8 environment
      [Paride Legovini]
    - Fix flake8 E741 warning [Lucas Moura]

 -- Ryan Harper <email address hidden> Tue, 26 May 2020 16:31:39 -0500

With Ubuntu 20.05.2, the curtin/subiquity installer continues to fail to install onto existing partitions.

With the following partition table, subiquity/curtin refuses to use /dev/sda1 as boot partition and /dev/sda3 as root partition - something it previously accepted as choices (but it failed during the process).

{
   "partitiontable": {
      "label":"dos",
      "id":"0xd9859edb",
      "device":"/dev/sda",
      "unit":"sectors",
      "partitions": [
         {"node":"/dev/sda1", "start":2048, "size":1998047, "type":"83", "bootable":true},
         {"node":"/dev/sda2", "start":2000896, "size":2000896, "type":"83"},
         {"node":"/dev/sda3", "start":4001792, "size":64000000, "type":"83"}
      ]
   }
}

@raharper I'd like to have this bug reopened, as it is still not fixed

Still not fixed

@Kai

Can you provide the installer crash or logs?

"refuses to use /dev/sda1 as boot partition and /dev/sda3 as root partition"

I wonder if recent changes in subiquity around ESPs may have something to do
with this.

@Ryan,

Since the installer refuses to install, there are no crash logs that I can provide you with. If you try to install 20.04 onto a disk with the mentioned layout (the "Custom layout" option), you should find that you are unable to choose sda1 and sda3 as destinations for /boot and /. Don't you have a system where you can emulate this (that would be very handy)?

Is the installer log located in the same folder as the crash logs (/run/casper or something like that)?

A rant for another time is that the 20.04 installer system has never stored the install and crash logs on installation USB stick. In order to retrieve the logs I've had to use a second USB stick and manually copy files over to it.

@Ryan

I booted the server into the 20.04 installer and copied off the /var/log directory.
/var/log/installer/subiquity-debug.log.3179 ends with:

2020-06-02 21:22:21,793 INFO root:39 start: subiquity/Filesystem: starting UI
2020-06-02 21:22:21,793 DEBUG root:39 start: subiquity/Filesystem/convert_autoinstall_config:
2020-06-02 21:22:21,794 DEBUG subiquitycore.controller.filesystem:170 self.ai_data = {}
2020-06-02 21:22:21,794 DEBUG root:39 finish: subiquity/Filesystem/convert_autoinstall_config: SUCCESS:
2020-06-02 21:22:44,255 DEBUG subiquity.ui.filesystem.filesystem:314 _action DeviceAction.EDIT partition-sda1
2020-06-02 21:23:05,415 DEBUG subiquity.ui.filesystem.filesystem:314 _action DeviceAction.EDIT partition-sda3

The subiquity-info.log file just reports that it has started the filesystem stuff:
2020-06-02 21:22:21,793 INFO root:39 start: subiquity/Filesystem: starting UI

Maybe I should point out that I cannot select /dev/sda1 at all as /boot, but I can select sda3, but just not as root directory.

If you have some kind of emulation environment to test out disk layout scenarios, then now is the time to pull it out.

> @Ryan,
>
> Since the installer refuses to install, there are no crash logs that I can
> provide you with. If you try to install 20.04 onto a disk with the mentioned
> layout (the "Custom layout" option), you should find that you are unable to
> choose sda1 and sda3 as destinations for /boot and /. Don't you have a system
> where you can emulate this (that would be very handy)?

Thank you for the repro steps.

Booting the curtin live focal iso, I see that the curtin
version included with subiquity is 19.3-67-g145e49393

And this fix for this issue is in curtin 19.3-76-gcb60d8a7.

The fix has landed in *curtin* but we still need subiquity to pull
in this fix (subiquity bundles curtin into the snap).

>
> Is the installer log located in the same folder as the crash logs
> (/run/casper or something like that)?

It should be in /var/log/installer/* and /var/log/curtin/* (if it gets that
far, which in your case it doesnt)

>
> A rant for another time is that the 20.04 installer system has never stored
> the install and crash logs on installation USB stick. In order to retrieve the
> logs I've had to use a second USB stick and manually copy files over to it.
>

Crashes typically are uploaded to the crash service; however in the case
where it doesn't actually crash; there's not a way to "report an issue";
that I know of, so please do file a bug for that so we can track getting a fix
for that.

Are there any plans to incorporate the fix to curtin? It was released 27 May 2020 but the status of this bug doesn't seem to have changed.

It is now (as of a few minutes) in the snap currently in edge, so you should be able to test following the instructions in https://discourse.ubuntu.com/t/how-to-test-the-latest-version-of-subiquity/12428

I followed the instructions to get the new installer from edge. When I try to edit an existing LVM volume to make it root for the install the installer crashes as soon as I hit return on "Edit". There are some lines of backtrace, but they disappear before I can see them. I did upload a bug report, which hopefully is visible somewhere.

Sorry for the delay in trying this out, I seem to have gotten unsubscribed from the bug report so I didn't get an email saying there was a new comment.

I have a similar message trying to install 20.04.1:

An error occured handling 'partions-sda5': RuntimeError - Verifying /dev/sda5 partition flag, expecting swap, found None

It appears this is still happening to me many months / years later.

Even when I picked to upgrade the installer snap to the latest version.

I have never managed to get the new Ubuntu Server installer to work, either on a VBox VM like this, or on bare metal. It is very unfortunate since the d-i approach basically always succeeded.

Crash log attached.

For the sake of root cause analysis, I bind mounted a manually patched copy of block_meta.py over the top of the existing version and commented out the partition type exception logic, and the install proceeded. So there is definitely something broken about curtin's logic for handling DOS partition tables as opposed to GPT, and the bug is not fully fixed.