[arm64] nova instances can't boot with 3.13.0-92

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1608854

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

I think all the files are here ?

Junien - can you use lts-xenial until I get this sorted out ? The arm64 backport to enable EFI as well as support secure boot signed module enforcement is going to require quite a bit of effort.

Now I remember why I thought it was OK to disable EFI in arm64 Trusty. v3.13 had no arm64 support for EFI to begin with. However, I backported the UEFI patchset from Utopic (v3.16) which _did_ have arm64 support. I thought it would be OK to simply disable EFI in Trusty, but clearly something went awry. I think I can get away with reverting arch/arm64 EFI commits since they appear to be well isolated.

Please try this test kernel at http://people.canonical.com/~rtg/arm64-efi-lp1608854/

Never mind the test kernel. Its bogus according to Dann Frazier. Back to the drawing board.

OK. For now I'm using 3.13.0-91 or inferior, which works fine.

I've refreshed the test kernel at http://people.canonical.com/~rtg/arm64-efi-lp1608854. Please try arm64 which now has a backport from Utopic (v3.16).

Kernel from comment #15 boots fine in EFI mode in QEMU:

ubuntu@ubuntu:~$ cat /proc/version
Linux version 3.13.0-94-generic (rtg@gloin) (gcc version 4.8.2 20140110 (prerelease) [ibm/gcc-4_8-branch merged from gcc-4_8-branch, revision 205847] (Ubuntu/Linaro 4.8.2-13ubuntu1) ) #141 SMP Mon Aug 8 15:06:28 UTC 2016

git://kernel.ubuntu.com/rtg/ubuntu-trusty.git arm64-efi-lp1608854

I backported EFI functionality for arm64 and amd64 from Utopic (v3.16) to Trusty (v3.13) because much of the arch dependent infrastructure in v3.16 is required in order to support signed module enforcement with a MOK managed key override (for externally built modules such as DKMS). See the "UBUNTU: SAUCE: UEFI:" series of patches. The approach that I took was to revert all EFI functionality in Trusty back to vanilla v3.13, then cherry-picked and backported all EFI patches from Utopic (including stable updates). Ultimately, there is little diff between the Trusty backport and Utopic in the directories and files that matter (arch/x86/platform/efi, drivers/firmware/efi, arch/arm64/kernel/efi*). This backport is possible because EFI is almost completely isolated from the rest of the kernel. However, the patchset is so large that it is essentially unreviewable. I think the only way to verify this patchset is through testing. Fortunately it is one of those features that either works or it doesn't. Thus far my test results and those of #16 are positive for both arches.

So, just to be clear, you want us to test http://people.canonical.com/~rtg/arm64-efi-lp1608854/ ? This is the kernel built from git://kernel.ubuntu.com/rtg/ubuntu-trusty.git arm64-efi-lp1608854 ?

Junien - yes, that is the one.

Looks good to me :

$ cat /proc/version
Linux version 3.13.0-94-generic (rtg@gloin) (gcc version 4.8.2 20140110 (prerelease) [ibm/gcc-4_8-branch merged from gcc-4_8-branch, revision 205847] (Ubuntu/Linaro 4.8.2-13ubuntu1) ) #141 SMP Tue Aug 9 14:58:41 UTC 2016

This backport patchset is not successful. There are multiple platform regressions that appear intransigent. Therefore, I've reverted all patches related to Secure Boot and signed module enforcement. This should restore arm64 EFI functionality. Platforms wishing to use secure boot and signed module enforcement must install lts-xenial.

git://kernel.ubuntu.com/rtg/ubuntu-trusty.git arm64-efi-lp1608854-revert

test kernel at http://people.canonical.com/~rtg/arm64-efi-lp1608854-revert/

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-trusty' to 'verification-done-trusty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Junien - please try the refreshed kernel with boot issue fix at http://people.canonical.com/~rtg/arm64-efi-lp1608854/ - You should hopefully not see any arm64 regressions.

Hi Tim,

I can't see any arm64 kernel in http://people.canonical.com/~rtg/arm64-efi-lp1608854/, only amd64.

Junien - oops, there are arm64 binaries there now.

It boots fine !

$ cat /proc/version
Linux version 3.13.0-97-generic (rtg@gloin) (gcc version 4.8.2 20140110 (prerelease) [ibm/gcc-4_8-branch merged from gcc-4_8-branch, revision 205847] (Ubuntu/Linaro 4.8.2-13ubuntu1) ) #144 SMP Mon Sep 12 16:38:12 UTC 2016

This bug was fixed in the package linux - 3.13.0-96.143

---------------
linux (3.13.0-96.143) trusty; urgency=low

  [ Tim Gardner ]

  * Release Tracking Bug
    - LP: #1618083

  * CVE-2015-8767
    - sctp: Prevent soft lockup when sctp_accept() is called during a timeout
      event

  * MacBookPro11,4 fails to poweroff or suspend (LP: #1587714)
    - SAUCE: PCI: Workaround to enable poweroff on Mac Pro 11

  * 3.13: libvirtd: page allocation failure: order:4, mode:0x1040d0
    (LP: #1616193)
    - vhost-net: extend device allocation to vmalloc
    - vhost-net: don't open-code kvfree

  * [arm64] nova instances can't boot with 3.13.0-92 (LP: #1608854)
    - Revert "UBUNTU: [Config] CONFIG_EFI=n for arm64"
    - Revert "UBUNTU: SAUCE: UEFI: Set EFI_SECURE_BOOT bit in x86_efi_facility"
    - Revert "UBUNTU: SAUCE: UEFI: Add secure boot and MOK SB State disabled
      sysctl"
    - Revert "UBUNTU: SAUCE: UEFI: Display MOKSBState when disabled"
    - Revert "UBUNTU: SAUCE: UEFI: efi: Disable secure boot if shim is in insecure
      mode"
    - Revert "UBUNTU: SAUCE: UEFI MODSIGN: Import certificates from UEFI Secure
      Boot"
    - Revert "UBUNTU: SAUCE: UEFI: efi: Make EFI_SECURE_BOOT_SIG_ENFORCE depend on
      EFI"
    - Revert "UBUNTU: SAUCE: UEFI: Add option to automatically enforce module
      signatures when in Secure Boot mode"
    - Revert "UBUNTU: [Config] CONFIG_EFI_SECURE_BOOT_SIG_ENFORCE=y"
    - Revert "UBUNTU: SAUCE: UEFI: x86: Restrict MSR access when module loading is
      restricted"
    - Revert "UBUNTU: SAUCE: UEFI: kexec: Disable at runtime if the kernel
      enforces module loading restrictions"
    - Revert "UBUNTU: SAUCE: UEFI: acpi: Ignore acpi_rsdp kernel parameter when
      module loading is restricted"
    - Revert "UBUNTU: SAUCE: UEFI: Restrict /dev/mem and /dev/kmem when module
      loading is restricted"
    - Revert "UBUNTU: SAUCE: UEFI: asus-wmi: Restrict debugfs interface when
      module loading is restricted"
    - Revert "UBUNTU: SAUCE: UEFI: ACPI: Limit access to custom_method"
    - Revert "UBUNTU: SAUCE: UEFI: x86: Lock down IO port access when module
      security is enabled"
    - Revert "UBUNTU: SAUCE: UEFI: PCI: Lock down BAR access when module security
      is enabled"
    - Revert "UBUNTU: SAUCE: UEFI: Add secure_modules() call"
    - Revert "x86/efi: Build our own EFI services pointer table"
    - Revert "efi: Add separate 32-bit/64-bit definitions"

  * [Hyper-V] storvsc messages for CD-ROM medium not present tray closed
    (LP: #1590655)
    - scsi: storvsc: Filter out storvsc messages CD-ROM medium not present

  * CVE-2016-3841
    - ipv6: add complete rcu protection around np->opt

 -- Kamal Mostafa <email address hidden> Tue, 16 Aug 2016 10:20:51 -0700

Status changed to 'Confirmed' because the bug affects multiple users.

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-trusty' to 'verification-done-trusty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

ubuntu@ubuntu:~$ archdetect
arm64/efi
ubuntu@ubuntu:~$ cat /proc/version
Linux version 3.13.0-99-generic (buildd@bos01-arm64-032) (gcc version 4.8.4 (Ubuntu/Linaro 4.8.4-2ubuntu1~14.04.3) ) #146-Ubuntu SMP Wed Oct 12 20:56:37 UTC 2016

The most recent d-i for trusty was built w/ 3.13.0-110, so I'll go ahead and mark fix released. I suspect, however, that this kernel still won't boot on any modern OpenStack deployment, since the EFI firmware will require ACPI support that was not available in 3.13.