snapd cannot refresh on some SD cards due to uboot bug

Fixing this issue is complicated by the unfortunate update of the pi/18 gadget for armhf to 256MB boot partition. Some devices in the field have 128MB partitions, other have 256MB, depending on just when the particular image was built.

This prevents us from updating the gadget / bootloader as the sanity check logic will refuse this update.

The second issue is that updating kernel and gadget separately is impossible. Updating the boot assets manually shows that the old kernel (revision 44) cannot boot correctly with gadget 74 (current stable). This requires the ability to update those in sync.

We could also issue a repair assertion that patches uboot.bin with the correct timing code. We could generate sufficient number of xdeltas to cover the revisions used in the field.

Information about the card with this particular write behavior:

U-Boot> mmcinfo
Device: mmc@7e202000
Manufacturer ID: 1b
OEM: 534d
Name: 00000
Bus Speed: 50000000
Mode : SD High Speed (50MHz)
Rd Block Len: 512
SD version 3.0
High Capacity: Yes
Capacity: 7.4 GiB
Bus Width: 4-bit
Erase Group Size: 512 Bytes

I've re-constructed the u-boot matching revision 17 and managed to modify it (it still boots). I will now attempt to apply the patch discussed in the thread.

The patch does work. I can now reliably write to the SD card and can update the core18 snap.

This issue is fixed upstream https://github.com/u-boot/u-boot/commit/b1125802a524641ad1ac803b4a617756d26f007d
 but the corresponding patch is not available in bionic or focal. Groovy u-boot package is recent enough to contain this fix.

So I think the plan of action here should be:
 - Backport the corresponding patch to focal and bionic
 - Respin pi gadgets to pull in the latest uboot
 - Refresh UC18 pi images to get the latest changes, so that no new users would experience this bug
 - Due to the unfeasability of other options, look into creating a repair assertion for existing users

Thank you Zygmunt for your thorough investigation!

Ok, so it seems Dave Jones will backport the current u-boot version to groovy, focal and bionic, so this would solve the issue of the patch missing. In the meantime, I'll try to investigate the repair assertion.

Fixed in Groovy
---

u-boot (2020.04+dfsg-2ubuntu1) groovy; urgency=medium

  * Merge with 2020.04+dfsg-2 from Debian unstable.

[...]

u-boot (2019.10~rc4+dfsg-1) experimental; urgency=medium

  * New upstream release candidate.
  * u-boot-rockchip: Add rock64-rk3328 target.
  * u-boot-rockchip: Install rockchip_make_fit_atf and Recommend python3.
  * generate-rksd: Add support for rk3328 and TPL images.
  * debian/patches:
    - Update ensure-config-sandbox-for-make-env.
    - Update qemu-riscv64-raw-initrd for.
    - Add description and forwarded links for qemu-riscv64-raw-initrd.
  * u-boot-qemu: Drop support for qemu_mips, qemu_mipsel and
    qemu_mips64el as the cross compilers are no longer available.
  * u-boot-tools:
    - Remove fw_printenv and fw_setenv and Recommend libubootenv-tool.
      (Closes: #939598)
    - Update package description.
  * README.Debian: Drop references to fw_setenv and fw_printenv.
  * debian/control:
    - [arm64] Drop Build-Depends on python-pyelftools; no longer needed
      for fit image creation.
    - Update Standards-Version to 4.4.0, no changes needed.
    - Update to debhelper-compat 12.
  * debian/copyright: Drop stanza and license for pcmcia; code was
    removed.

 -- Dave Jones <email address hidden> Sat, 16 May 2020 01:02:53 +0000

Hello Zygmunt, or anyone else affected,

Accepted u-boot into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/u-boot/2020.10+dfsg-1ubuntu0~18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

I've updated pi gadget to latest/stable and then manually updated the assets by copying files fro the gadget boot assets to the appropriate place on the FAT partition. Upon reboot uboot version was

U-Boot> version
U-Boot 2019.07+dfsg-1ubuntu4~18.04.1 (Dec 19 2019 - 22:06:27 +0000)

gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0
GNU ld (GNU Binutils for Ubuntu) 2.30

Subsequently I was able to successfully write to FAT from uboot:

U-Boot> saveenv
Saving Environment to FAT... OK
U-Boot> saveenv
Saving Environment to FAT... OK

I think this worked.

Note that without bumping the edition field, this will require manual intervention.

Hello Zygmunt, or anyone else affected,

Accepted u-boot into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/u-boot/2020.10+dfsg-1ubuntu0~18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

This is the same as https://bugs.launchpad.net/ubuntu/+source/u-boot/+bug/1910094, so verification can be performed on either one of these bugs.

Is this fix released in focal? The other mentioned bug is marked as such.

Yes, this was released in focal a while ago. And since the other bug has been verified for bionic as well, marking this as such too.

This bug was fixed in the package u-boot - 2020.10+dfsg-1ubuntu0~18.04.2

---------------
u-boot (2020.10+dfsg-1ubuntu0~18.04.2) bionic; urgency=medium

  * SRU of changes through to 2020.10+dfsg-1ubuntu6 to fix Pi4-8GB & CM4 support:

  * Add d/p/rpi-8gb-pci.patch for Pi400 and Pi4-8Gb support (LP: #1906552)
  * Add d/p/rpi-cm4-sdhci.patch for CM4 eMMC support
  * Add d/p/rpi-maxargs.patch for new Core 18 boot-env (LP: #1910094)
  * Remove redundant d/targets entries

u-boot (2020.10+dfsg-1ubuntu0~18.04.1) bionic; urgency=low

  * SRU of 2020.10+dfsg-1 to support Raspberry Pi 4 (LP: #1903054), and
    fix a failure to write issue (LP: #1900693)
  * Re-added d/p/lzo-to-lzno.patch as lzop is in universe prior to Groovy
  * Updated d/targets to remove arm-trusted-firmware references
  * Updated d/control for compatibility with bionic debhelper
  * Reinstate rpi-config-migration script for bionic

  [ Dave Jones ]
  * Merge from Debian unstable. Remaining changes:
    - Enable Ubuntu support for the Nitrogen6x board (LP: #1838064)
      - Add d/p/ubuntu-nitrogen6q2g-config-tweaks.patch to tweak the
        nitrogen6q2g configs to better fit our Ubuntu usage.
      - Start building the nitrogen6x2g target for u-boot.
    - Enable FIT signing support (LP: #1831942)
      - Enable CONFIG_FIT_SIGNATURE so we can sign FIT images.
      - Add libssl-dev to Build-Depends: to enable crypto functionality.
      - Limit key names to keys within the keydir.
    - d/p/rpi-config-tweaks.patch: Configuration adjustments to the RPi
      configs
    - d/p/rpi-board-dt.patch: use the board's device-tree instead of an
      embedded one

  * Removed obsolete patches/changes:
    - d/p/lzo-to-lzno.patch: use gzip instead of lzo compression for FIT
      images as lzop in Ubuntu is in universe. This should be temporary and in
      the next releases ideally we should follow what Debian does.

u-boot (2020.10+dfsg-1) unstable; urgency=medium

  * New upstream release.

u-boot (2020.10~rc5+dfsg-1) experimental; urgency=medium

  [ Vagrant Cascadian ]
  * New upstream release candidate.
  * debian/control: u-boot-qemu: Set Multi-Arch: foreign.

  [ Uwe Kleine-König ]
  * control: Use https for upstream homepage

u-boot (2020.10~rc3+dfsg-1) experimental; urgency=medium

  * New upstream release candidate.
  * debian/patches: Refresh and remove obsolete patches.
  * [armhf] u-boot-sunxi: Add Bananapi_M2_Ultra (Closes: #962931). Thanks
    to Bernhard Wörner.
  * [arm64] u-boot-rockchip: Update rock-pi-4 .dtb names.
  * u-boot-qemu: Build firmware for qemu ppc e500 (Closes: #966624).

u-boot (2020.07+dfsg-2) unstable; urgency=medium

  [ Vagrant Cascadian ]
  * u-boot-install-rockchip: Update with additional RockPro64 names from
    linux 5.7 device-trees.

  [ Denis Pynkin ]
  * u-boot-rpi: Add rpi_arm64 target for arm64 (Closes: #966078).

  [ Vagrant Cascadian ]
  * debian/patches: Fix reproducibility of mx6cuboxi target.

u-boot (2020.07+dfsg-1) unstable; urgency=medium

  * New upstream release.
  * [armhf] u-boot-omap: Drop omap3_pandora, removed upstream.
  * [arm64] u-boot-amlogic: Install new board-specific documentation.
  * debian/patches: Fix riscv64 suppor...

The verification of the Stable Release Update for u-boot has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Closing per comment 15