Unable to use fsck on RHEL/CentOS Images deployed using Bionic Ephemeral Image

Bug #1825729 reported by Pedro Principeza
32
This bug affects 6 people
Affects Status Importance Assigned to Milestone
MAAS
Invalid
High
Unassigned
curtin
Triaged
Medium
Unassigned

Bug Description

From Bionic on, e2fsprogs implemented the "metadata_csum" as a default feature for ext4 filesystem creation [1].

With that change in place, deployments of CentOS / RHEL images (either through MIB or other means, such as packer) fail to execute fsck against ext4 filesystems of these, given these distros do not ship e2fsprogs version higher than 1.43 (or, better said, e2fsprogs versions that support metadata_csum).

The following message is seen, upon attempting to execute fsck against the filesystem:

 Apr 19 19:25:03 TOROON63I6W systemd-fsck: /dev/mapper/vg0-lv3 has unsupported feature(s):
 metadata_csum
 Apr 19 19:25:03 TOROON63I6W systemd-fsck: e2fsck: Get a newer version of e2fsck!
 Apr 19 19:25:03 TOROON63I6W systemd-fsck: fsck failed with error code 8.

The workaround, as of now, is to deploy the CentOS/RHEL images using a Xenial Ephemeral Image, thus avoiding the e2fsprogs version that takes "metadata_csum" as a default feature.

This setup uses MAAS at:

ii maas-cli 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS client and command-line interface
ii maas-common 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS server common files
ii maas-proxy 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS Caching Proxy
ii maas-region-api 2.4.2-7034-g2f5deb8b8-0ubuntu1 all Region controller API service for MAAS
ii maas-region-controller 2.4.2-7034-g2f5deb8b8-0ubuntu1 all Region Controller for MAAS
ii python3-django-maas 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS python API client (Python 3)
ii python3-maas-provisioningserver 2.4.2-7034-g2f5deb8b8-0ubuntu1 all MAAS server provisioning libraries (Python 3)

But I believe this affects MAAS at any version.

To reproduce this issue:

-- Set Bionic as the Commissioning Image to be used;
-- Deploy a CentOS/RHEL 7 image using either MIB or packer, defining at least one ext4 fileystem on Curtin.
-- Check the messages file on the firstboot of the deployed image, or attempt to run fsck against the ext4 filesystem.

I'm attaching curtin debug logs, the yaml used on the deployment, and the messages file outlining the error message on the fsck execution.

[1] https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/1365874

Revision history for this message
Pedro Principeza (pprincipeza) wrote :
Ryan Harper (raharper)
Changed in curtin:
importance: Undecided → Medium
status: New → Triaged
Lee Trager (ltrager)
Changed in maas:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Lee Trager (ltrager)
milestone: none → 2.7.0alpha1
Changed in maas:
milestone: 2.7.0b1 → 2.7.0b2
Changed in maas:
milestone: 2.7.0b2 → none
Changed in maas:
assignee: Lee Trager (ltrager) → nobody
Revision history for this message
Adam Collard (adam-collard) wrote :

We plan to address this as a feature that explicitly models the dependencies between target deployment image and the ephemeral image that MAAS uses to run curtin in (and install that image).

This will, for example, give MAAS the knowledge that CentOS 6 requires Xenial, CentOS 7 requires Focal etc.

Internally, see PF-3237

Revision history for this message
Jerzy Husakowski (jhusakowski) wrote :

On MAAS feature backlog

Changed in maas:
status: Triaged → Invalid
Revision history for this message
Igor Gnip (igorgnip) wrote :

Alternative solution which appears to me easier is to turn off ext4 metadata checksum using tune2fs during deployment of OS which does not have e2fsprogs recent enough to support it.
That would be rhel7 and centos7 at least.

This could be part of curtin step as well and I think it can be done in the maas preseeds.

I strongly urge / advise you not to mess with default ephemeral versions cause there are other issues potentially opening that pandora's box - example: network or storage drivers not present in xenial.

Revision history for this message
Igor Gnip (igorgnip) wrote :

Not being able to reliably know which ephemeral image maas runs would be much worse than the current issue.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.