snappy update: fsck errors

ok, so few things I believe that should be looked at:

1. reduce risk of corruption of other partition during update -> mount it with "sync" option as well

2. try harder to recover from corruption
  -> if normal fsck fails, maybe try harder
  -> if that also fails or if try harder is considered not clean -> mkfs and recreate from scratch

also we might want to think about what we do in worst case where even mkfs cannot do its job anymore... in that case I would assume we need to have a special system mode like "requires maintenance" that then can be surfaced somehow to user by appliance manufacturer...

along these lines we should think about how to reproduce these cases (e.g. how to forcefully corrupt the FS to ensure we can test this works).

Lool mentioned on irc that mounting "sync":

<lool> (it will slow down the updates greatly and will wear the media faster)

so maybe a sync at the end of the update is the right compromise.

for testing we might want to play with:

11:05 < asac> like dd if=/dev/random of=/dev/partition :)
or /dev/null

err, /dev/zero not /dev/null for getting zeros

subscribed pitti so he can help us on our testing story here as part of our system update testing thread.

> # system bring up correctly
Do you recall seeing any errors on boot when you first booted the image written using ubuntu-device-flash?

Is it possible you had previously attempted to update from 185 -> 186 and the power failed or the device was unplugged?

If not, either your media has "gone bad", or was always bad but you were lucky enough to boot from the "system-a" partition initially.

To resolve the situation, you have a few options. Tthe normal caveats apply - if I were you, I'd back up any writable data you have on the device before proceeding:

(1) Run fsck manually:

# check that the partition really is one of the read-only rootfs's first
$ sudo blkid|grep "/dev/mmcblk0p2.*LABEL=\"system-a\"" && echo OK || echo FAIL

$ sudo umount /dev/mmcblk0p2
$ sudo fsck -M -yv /dev/mmcblk0p2
(answer any prompts - the 'y' option should do everything for you though)
$ sudo reboot
$ sudo snappy-go update

(2) Reformat the "system-a" rootfs:

# check that the partition really is one of the read-only rootfs's first
$ sudo blkid|grep "/dev/mmcblk0p2.*LABEL=\"system-a\"" && echo OK || echo FAIL

$ sudo umount /dev/mmcblk0p2
$ sudo mkfs.ext4 -v -L system-a /dev/mmcblk0p2
$ sudo reboot
$ sudo snappy-go update

(3) Reflash the entire disk using ubuntu-device-flash

(4) Flash an image to a different disk using ubuntu-device-flash.

We no longer have separate partitions. We have been testing many times a day the update of ubuntu-core without any errors like this one. Marking as invalid.