| 2018-12-06 03:22:19 |
Ryan Finnie |
bug |
|
|
added bug |
| 2018-12-06 03:34:17 |
Ryan Finnie |
attachment added |
|
lp1807077-trusty.debdiff https://bugs.launchpad.net/ubuntu/+source/mountall/+bug/1807077/+attachment/5219619/+files/lp1807077-trusty.debdiff |
|
| 2018-12-06 03:34:33 |
Ryan Finnie |
attachment added |
|
lp1807077-xenial.debdiff https://bugs.launchpad.net/ubuntu/+source/mountall/+bug/1807077/+attachment/5219620/+files/lp1807077-xenial.debdiff |
|
| 2018-12-06 03:54:44 |
Ryan Finnie |
description |
Running bionic's 4.15 kernel on trusty on an HPE DL385 Gen10 results in a device node for the NVMe controller, /devices/pci0000:40/0000:40:03.1/0000:43:00.0/nvme/nvme0/nvme0c33n1 which itself does not have a devname. When mountall gets to it:
fsck_update: updating check priorities
try_mount: /srv/nova/instances waiting for device
try_udev_device: ignored /dev/loop5 (not yet ready?)
try_udev_device: ignored /dev/loop6 (not yet ready?)
try_udev_device: ignored /dev/loop1 (not yet ready?)
try_udev_device: ignored /dev/loop0 (not yet ready?)
try_udev_device: block (null) (null) (null)
and then crashes, leaving the boot at an emergency root shell. A successful scan looks like this for comparison:
try_udev_device: block /dev/sdb (null) (null)
try_udev_device: block /dev/sdb (null) (null)
try_udev_device: block /dev/sda (null) (null)
try_udev_device: block /dev/nvme0n1 ed56e3a9-60f7-4636-85a2-b53137b598e7 (null)
try_udev_device: block /dev/bcache0 756cb2c6-b999-4905-a021-c2e688e81a86 instances
The (forthcoming) debdiffs check for a null devname in try_udev_device() and will not attempt to process it. |
[Impact]
* udev block nodes without a devname will crash mountall, resulting in an unbootable system (emergency root shell)
* While this is not likely to happen in a matched distro/kernel environment (it was discovered while needing to run a bionic 4.15 kernel on trusty), it is possible.
* The code in try_udev_device() assumes a block subsystem will always have a devname; the SRU patches explicitly check for a devname and return if null.
[Test Case]
* HPE DL385 Gen10, Samsung a822 NVMe controller, trusty install
* Kernels <4.15 (or possibly 4.14) do not expose nvme0c33n1 and do not trigger the bug. Tested on 4.13, 4.4 and 3.13.
* Kernel 4.15 exposes nvme0c33n1 in udev but does not have a devname, mountall crash ensues.
[Regression Potential]
* Patch might ignore legitimate block devices on existing installations. Unlikely, since the logic path for null devname leads directly to a program crash.
[Other Info]
* Additional context for Canonical employees: PS4.5 is a trusty backend cloud, but we now have Gen10 hardware incoming (this was discovered while adding new nova-compute hardware). Older kernels are not usable because Gen10 requires ilorest, which requires a >4.4 kernel (at least artful 4.13 is known good). So trusty+4.15 is the only viable combination for continued support of the cloud while adding new hardware. This is done via apt pinning of bionic for the kernel packages, and, mountall notwithstanding, is working fine so far.
Original description:
Running bionic's 4.15 kernel on trusty on an HPE DL385 Gen10 results in a device node for the NVMe controller, /devices/pci0000:40/0000:40:03.1/0000:43:00.0/nvme/nvme0/nvme0c33n1 which itself does not have a devname. When mountall gets to it:
fsck_update: updating check priorities
try_mount: /srv/nova/instances waiting for device
try_udev_device: ignored /dev/loop5 (not yet ready?)
try_udev_device: ignored /dev/loop6 (not yet ready?)
try_udev_device: ignored /dev/loop1 (not yet ready?)
try_udev_device: ignored /dev/loop0 (not yet ready?)
try_udev_device: block (null) (null) (null)
and then crashes, leaving the boot at an emergency root shell. A successful scan looks like this for comparison:
try_udev_device: block /dev/sdb (null) (null)
try_udev_device: block /dev/sdb (null) (null)
try_udev_device: block /dev/sda (null) (null)
try_udev_device: block /dev/nvme0n1 ed56e3a9-60f7-4636-85a2-b53137b598e7 (null)
try_udev_device: block /dev/bcache0 756cb2c6-b999-4905-a021-c2e688e81a86 instances
The debdiffs check for a null devname in try_udev_device() and will not attempt to process it. |
|
| 2018-12-06 03:55:08 |
Ryan Finnie |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
| 2018-12-06 04:48:12 |
Ryan Finnie |
bug |
|
|
added subscriber The Canonical Sysadmins |
| 2018-12-06 05:54:50 |
Launchpad Janitor |
mountall (Ubuntu): status |
New |
Confirmed |
|
| 2018-12-06 07:49:08 |
Ryan Finnie |
tags |
|
patch trusty xenial |
|
| 2018-12-06 07:49:57 |
Ryan Finnie |
nominated for series |
|
Ubuntu Trusty |
|
| 2018-12-06 07:49:57 |
Ryan Finnie |
bug task added |
|
mountall (Ubuntu Trusty) |
|
| 2018-12-06 07:49:57 |
Ryan Finnie |
nominated for series |
|
Ubuntu Xenial |
|
| 2018-12-06 07:49:57 |
Ryan Finnie |
bug task added |
|
mountall (Ubuntu Xenial) |
|
| 2018-12-06 07:53:25 |
Ryan Finnie |
summary |
mountall crashes on udev node with missing devname |
[SRU] mountall crashes on udev node with missing devname |
|
| 2018-12-06 10:47:28 |
Launchpad Janitor |
mountall (Ubuntu Trusty): status |
New |
Confirmed |
|
| 2018-12-06 10:47:28 |
Launchpad Janitor |
mountall (Ubuntu Xenial): status |
New |
Confirmed |
|
| 2018-12-06 11:13:30 |
Steve Langasek |
mountall (Ubuntu Xenial): status |
Confirmed |
Won't Fix |
|
| 2018-12-06 11:13:53 |
Steve Langasek |
mountall (Ubuntu): status |
Confirmed |
Invalid |
|
| 2018-12-06 19:22:19 |
Ryan Finnie |
tags |
patch trusty xenial |
patch trusty |
|
| 2018-12-06 19:22:41 |
Ryan Finnie |
attachment removed |
lp1807077-xenial.debdiff https://bugs.launchpad.net/ubuntu/+source/mountall/+bug/1807077/+attachment/5219620/+files/lp1807077-xenial.debdiff |
|
|
| 2018-12-06 19:34:01 |
Ryan Finnie |
mountall (Ubuntu Trusty): importance |
Undecided |
High |
|
| 2018-12-11 20:12:24 |
Brian Murray |
mountall (Ubuntu Trusty): status |
Confirmed |
Fix Committed |
|
| 2018-12-11 20:12:26 |
Brian Murray |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
| 2018-12-11 20:12:29 |
Brian Murray |
bug |
|
|
added subscriber SRU Verification |
| 2018-12-11 20:12:31 |
Brian Murray |
tags |
patch trusty |
patch trusty verification-needed verification-needed-trusty |
|
| 2018-12-11 20:12:41 |
Brian Murray |
removed subscriber Ubuntu Sponsors Team |
|
|
|
| 2018-12-13 09:56:46 |
William Grant |
bug |
|
|
added subscriber William Grant |
| 2018-12-25 04:31:35 |
Ryan Finnie |
tags |
patch trusty verification-needed verification-needed-trusty |
patch trusty verification-done verification-done-trusty |
|
| 2018-12-25 04:38:12 |
Ryan Finnie |
description |
[Impact]
* udev block nodes without a devname will crash mountall, resulting in an unbootable system (emergency root shell)
* While this is not likely to happen in a matched distro/kernel environment (it was discovered while needing to run a bionic 4.15 kernel on trusty), it is possible.
* The code in try_udev_device() assumes a block subsystem will always have a devname; the SRU patches explicitly check for a devname and return if null.
[Test Case]
* HPE DL385 Gen10, Samsung a822 NVMe controller, trusty install
* Kernels <4.15 (or possibly 4.14) do not expose nvme0c33n1 and do not trigger the bug. Tested on 4.13, 4.4 and 3.13.
* Kernel 4.15 exposes nvme0c33n1 in udev but does not have a devname, mountall crash ensues.
[Regression Potential]
* Patch might ignore legitimate block devices on existing installations. Unlikely, since the logic path for null devname leads directly to a program crash.
[Other Info]
* Additional context for Canonical employees: PS4.5 is a trusty backend cloud, but we now have Gen10 hardware incoming (this was discovered while adding new nova-compute hardware). Older kernels are not usable because Gen10 requires ilorest, which requires a >4.4 kernel (at least artful 4.13 is known good). So trusty+4.15 is the only viable combination for continued support of the cloud while adding new hardware. This is done via apt pinning of bionic for the kernel packages, and, mountall notwithstanding, is working fine so far.
Original description:
Running bionic's 4.15 kernel on trusty on an HPE DL385 Gen10 results in a device node for the NVMe controller, /devices/pci0000:40/0000:40:03.1/0000:43:00.0/nvme/nvme0/nvme0c33n1 which itself does not have a devname. When mountall gets to it:
fsck_update: updating check priorities
try_mount: /srv/nova/instances waiting for device
try_udev_device: ignored /dev/loop5 (not yet ready?)
try_udev_device: ignored /dev/loop6 (not yet ready?)
try_udev_device: ignored /dev/loop1 (not yet ready?)
try_udev_device: ignored /dev/loop0 (not yet ready?)
try_udev_device: block (null) (null) (null)
and then crashes, leaving the boot at an emergency root shell. A successful scan looks like this for comparison:
try_udev_device: block /dev/sdb (null) (null)
try_udev_device: block /dev/sdb (null) (null)
try_udev_device: block /dev/sda (null) (null)
try_udev_device: block /dev/nvme0n1 ed56e3a9-60f7-4636-85a2-b53137b598e7 (null)
try_udev_device: block /dev/bcache0 756cb2c6-b999-4905-a021-c2e688e81a86 instances
The debdiffs check for a null devname in try_udev_device() and will not attempt to process it. |
[Impact]
* udev block nodes without a devname will crash mountall, resulting in an unbootable system (emergency root shell)
* While this is not likely to happen in a matched distro/kernel environment (it was discovered while needing to run a bionic 4.15 kernel on trusty), it is possible.
* The code in try_udev_device() assumes a block subsystem will always have a devname; the SRU patches explicitly check for a devname and return if null.
[Test Case]
* HPE DL385 Gen10, Samsung a822 NVMe controller, trusty install
* Kernels <4.15 (or possibly 4.14) do not expose nvme0c33n1 and do not trigger the bug. Tested on 4.13, 4.4 and 3.13.
* Kernel 4.15 exposes nvme0c33n1 in udev but does not have a devname, mountall crash ensues.
[Regression Potential]
* Patch might ignore legitimate block devices on existing installations. Unlikely, since the logic path for null devname leads directly to a program crash.
[Other Info]
* Additional context for Canonical employees: PS4.5 is a trusty backend cloud, but we now have Gen10 hardware incoming (this was discovered while adding new nova-compute hardware). Older kernels are not usable because Gen10 requires ilorest, which requires a >4.4 kernel (at least artful 4.13 is known good). So trusty+4.15 is the only viable combination for continued support of the cloud while adding new hardware. This is done via apt pinning of bionic for the kernel packages, and, mountall notwithstanding, is working fine so far.
TEST CASE:
1. Enable -proposed
2. apt-get install mountall=2.53ubuntu1
3. update-initramfs -k all -u
4. Reboot
VERIFICATION DONE
Rebooted successfully on affected Gen10 systems. Confirmed no regression on unaffected systems.
Original description:
Running bionic's 4.15 kernel on trusty on an HPE DL385 Gen10 results in a device node for the NVMe controller, /devices/pci0000:40/0000:40:03.1/0000:43:00.0/nvme/nvme0/nvme0c33n1 which itself does not have a devname. When mountall gets to it:
fsck_update: updating check priorities
try_mount: /srv/nova/instances waiting for device
try_udev_device: ignored /dev/loop5 (not yet ready?)
try_udev_device: ignored /dev/loop6 (not yet ready?)
try_udev_device: ignored /dev/loop1 (not yet ready?)
try_udev_device: ignored /dev/loop0 (not yet ready?)
try_udev_device: block (null) (null) (null)
and then crashes, leaving the boot at an emergency root shell. A successful scan looks like this for comparison:
try_udev_device: block /dev/sdb (null) (null)
try_udev_device: block /dev/sdb (null) (null)
try_udev_device: block /dev/sda (null) (null)
try_udev_device: block /dev/nvme0n1 ed56e3a9-60f7-4636-85a2-b53137b598e7 (null)
try_udev_device: block /dev/bcache0 756cb2c6-b999-4905-a021-c2e688e81a86 instances
The debdiffs check for a null devname in try_udev_device() and will not attempt to process it. |
|
| 2019-01-07 11:02:03 |
Łukasz Zemczak |
mountall (Ubuntu Trusty): status |
Fix Committed |
Incomplete |
|
| 2019-01-11 05:59:11 |
Ryan Finnie |
mountall (Ubuntu Trusty): status |
Incomplete |
Fix Committed |
|
| 2019-01-14 08:58:34 |
Łukasz Zemczak |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
| 2019-01-14 09:08:37 |
Launchpad Janitor |
mountall (Ubuntu Trusty): status |
Fix Committed |
Fix Released |
|
