Nouveau driver crash - Ubuntu 22.04.3 LTS stuck on power-off/reboot screen

This bug also affects other users, as described in:

https://askubuntu.com/questions/1482315/ubuntu-22-04-03-lts-stuck-on-power-off-reboot-screen
https://askubuntu.com/questions/1426205/ubuntu-22-04-stuck-at-shutdown

Can you please attach some logs from a time span where you encountered this issue? E.g., if you just rebooted, you could do something like:

$ journalctl -b -1 --since "1 hour ago"

to get the logs the previous boot, that are no older than 1 hour (from the current time).

Status changed to 'Confirmed' because the bug affects multiple users.

I'm having the same issue with a fresh install for Ubuntu 22.04.3 using Kernel 6.2.0-26-generic.

Attached last shutdown logs

> journalctl -b -1 -r > logs-failed-shutdown.txt

I'm having the same issue with a Ubuntu 22.04.3 using Kernel 6.2.0-26-generic.

Hi, I got also sudden issue with shutdown or reboot.

```~$ journalctl -b -1 --since "1 hour ago"```

Same issue here on a Thinkpad with fresh 22.04.3 install.

Same issue here, 22.04.3.
But I`m completely stuck on login screen, after typing my password I get black screen and can`t use my computer at all. Anyone got a workaround to at least get the UI back.

Ronan,

This bug relates to the system not resetting or powering off after restart or shutdown. Not saying what you are experiencing isn’t a bug but I don’t think it is related to THIS specific bug.

Manny

It's not a systemd issue.

@juergh Please, provide further information to help-us with the bug.

It's not trivial since it happens so infrequently. I need to think about this.

@ianrussel your log shows:

Aug 22 09:57:02 ianrussel kernel: nouveau 0000:01:00.0: fb: VPR locked, but no scrubber binary!
Aug 22 09:57:02 ianrussel kernel: BUG: kernel NULL pointer dereference, address: 0000000000000000
Aug 22 09:57:02 ianrussel kernel: #PF: supervisor instruction fetch in kernel mode
Aug 22 09:57:02 ianrussel kernel: #PF: error_code(0x0010) - not-present page
Aug 22 09:57:02 ianrussel kernel: PGD 0 P4D 0
Aug 22 09:57:02 ianrussel kernel: Oops: 0010 [#1] PREEMPT SMP NOPTI
Aug 22 09:57:02 ianrussel kernel: CPU: 14 PID: 479 Comm: systemd-udevd Not tainted 6.2.0-26-generic #26~22.04.1-Ubuntu
Aug 22 09:57:02 ianrussel kernel: Hardware name: Dell Inc. XPS 15 9510/01V4T3, BIOS 1.22.0 07/13/2023
Aug 22 09:57:02 ianrussel kernel: RIP: 0010:0x0
Aug 22 09:57:02 ianrussel kernel: Code: Unable to access opcode bytes at 0xffffffffffffffd6.
Aug 22 09:57:02 ianrussel kernel: RSP: 0018:ffffb3a381f4b658 EFLAGS: 00010246
Aug 22 09:57:02 ianrussel kernel: RAX: 0000000000000000 RBX: ffff9d14de9198b0 RCX: 0000000000000000
Aug 22 09:57:02 ianrussel kernel: RDX: ffff9d14de9198d8 RSI: ffff9d14de9198c8 RDI: ffff9d14de919800
Aug 22 09:57:02 ianrussel kernel: RBP: ffffb3a381f4b6b8 R08: ffff9d14de9198b0 R09: 0000000000000000
Aug 22 09:57:02 ianrussel kernel: R10: 0000000000000000 R11: 0000000000000000 R12: ffff9d14de9198c8
Aug 22 09:57:02 ianrussel kernel: R13: 0000000000000000 R14: ffff9d14de9198b0 R15: ffff9d14de919808
Aug 22 09:57:02 ianrussel kernel: FS: 00007f13d5a4d8c0(0000) GS:ffff9d183f780000(0000) knlGS:0000000000000000
Aug 22 09:57:02 ianrussel kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug 22 09:57:02 ianrussel kernel: CR2: ffffffffffffffd6 CR3: 0000000117606004 CR4: 0000000000770ee0
Aug 22 09:57:02 ianrussel kernel: PKRU: 55555554
Aug 22 09:57:02 ianrussel kernel: Call Trace:
Aug 22 09:57:02 ianrussel kernel: <TASK>
Aug 22 09:57:02 ianrussel kernel: nvkm_acr_oneinit+0x502/0x710 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvkm_subdev_oneinit_+0x4d/0x120 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvkm_subdev_init_+0x3e/0x150 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvkm_subdev_init+0x55/0x80 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvkm_device_init+0x17b/0x300 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvkm_udevice_init+0x50/0x60 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvkm_object_init+0x40/0x210 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvkm_ioctl_new+0x196/0x2c0 [nouveau]
Aug 22 09:57:02 ianrussel kernel: ? __pfx_nvkm_client_child_new+0x10/0x10 [nouveau]
Aug 22 09:57:02 ianrussel kernel: ? __pfx_nvkm_udevice_new+0x10/0x10 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvkm_ioctl+0x12e/0x290 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvkm_client_ioctl+0xe/0x20 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvif_object_ctor+0x136/0x1a0 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nvif_device_ctor+0x24/0x90 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nouveau_cli_init+0x161/0x5e0 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nouveau_drm_device_init+0x67/0x310 [nouveau]
Aug 22 09:57:02 ianrussel kernel: nouveau_drm_probe+0x131/0x280 [nouveau]
Aug 22 09:57:02 ianrussel kernel: local_pci_probe+0x48/0xb0
A...

5 peoples using the same hardware have the same issue after upgrading to 22.04.03, see https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2033267

yes I'm experiencing the same issue.

A few of my colleagues are also affected with 6.2.

This is really a bad situation.

As workaround we stepped back to linux-oem-22.04c (6.1) which works without problems.

[1.545289] VGA switcheroo: detected Optimus DSM method \_SB_.PC00.PEG1.PEGP handle
[ 1.545290] nouveau: detected PR support, will not use DSM
[ 1.545320] nouveau 0000:01:00.0: enabling device (0006 -> 0007)
[ 1.545649] nouveau 0000:01:00.0: NVIDIA GA107 (b77000a1)
[ 1.567694] ACPI Error: Cannot release Mutex [ECMX], not acquired (20221020/exmutex-357)
[ 1.567699] fbcon: Taking over console
[ 1.567703] No Local Variables are initialized for Method [_Q66]
[ 1.567704] No Arguments are initialized for method [_Q66]
[ 1.567705] ACPI Error: Aborting method \_SB.PC00.LPCB.ECDV._Q66 due to previous error (AE_AML_MUTEX_NOT_ACQUIRED) (20221020/psparse-529)
[ 1.629655] usb 3-3: New USB device found, idVendor=0c45, idProduct=6720, bcdDevice=82.68
[ 1.629660] usb 3-3: New USB device strings: Mfr=2, Product=1, SerialNumber=0
[ 1.629661] usb 3-3: Product: Integrated_Webcam_HD
[ 1.629662] usb 3-3: Manufacturer: CN0V976R8LG001C1A5ANA01
[ 1.655115] nouveau 0000:01:00.0: bios: version 94.07.3b.00.e4
[ 1.655549] nouveau 0000:01:00.0: acr: firmware unavailable
[ 1.655693] nouveau 0000:01:00.0: gr: firmware unavailable
[ 1.655706] nouveau 0000:01:00.0: sec2: firmware unavailable
[ 1.655798] nouveau 0000:01:00.0: fb: 4096 MiB GDDR6
[ 1.659174] nouveau 0000:01:00.0: fb: VPR locked, but no scrubber binary!
[ 1.661968] BUG: kernel NULL pointer dereference, address: 0000000000000000
[ 1.661970] #PF: supervisor instruction fetch in kernel mode
[ 1.661972] #PF: error_code(0x0010) - not-present page
[ 1.661973] PGD 0 P4D 0
[ 1.661975] Oops: 0010 [#1] PREEMPT SMP NOPTI
[ 1.661977] CPU: 14 PID: 233 Comm: systemd-udevd Not tainted 6.2.0-26-generic #26~22.04.1-Ubuntu
[ 1.661980] Hardware name: Dell Inc. Precision 7560/0186RY, BIOS 1.20.0 02/10/2023
[ 1.661981] RIP: 0010:0x0
[ 1.661984] Code: Unable to access opcode bytes at 0xffffffffffffffd6.
[ 1.661985] RSP: 0018:ffffaaf340e0b628 EFLAGS: 00010246
[ 1.661986] RAX: 0000000000000000 RBX: ffff906ce208dcb0 RCX: 0000000000000000
[ 1.661988] RDX: ffff906ce208dcd8 RSI: ffff906ce208dcc8 RDI: ffff906ce208dc00
[ 1.661989] RBP: ffffaaf340e0b688 R08: ffff906ce208dcb0 R09: 0000000000000000
[ 1.661990] R10: 0000000000000000 R11: 0000000000000000 R12: ffff906ce208dcc8
[ 1.661991] R13: 0000000000000000 R14: ffff906ce208dcb0 R15: ffff906ce208dc08
[ 1.661992] FS: 00007f2e66a658c0(0000) GS:ffff90742ff80000(0000) knlGS:0000000000000000
[ 1.661994] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1.661995] CR2: ffffffffffffffd6 CR3: 00000001015d4006 CR4: 0000000000770ee0
[ 1.661997] PKRU: 55555554
[ 1.661999] Call Trace:
[ 1.662000] <TASK>
[ 1.662001] nvkm_acr_oneinit+0x502/0x710 [nouveau]
[ 1.662057] nvkm_subdev_oneinit_+0x4d/0x120 [nouveau]
[ 1.662103] nvkm_subdev_init_+0x3e/0x150 [nouveau]
[ 1.662145] nvkm_subdev_init+0x55/0x80 [nouveau]
[ 1.662186] nvkm_device_init+0x17b/0x300 [nouveau]
[ 1.662259] nvkm_udevice_init+0x50/0x60...

Without digging too deep, maybe this:
https://lore.kernel<email address hidden>/T/

I've builtt a test kernel with the above patch: https://kernel.ubuntu.com/~juergh/lp2031352/

Can someone give it a try, please? Note that the kernel is not signed so you need to disable SecureBoot to be able to boot it. Install linux-image-unsigned, linux-modules and maybe linux-modules-extra.

Same problem on my father's laptop: stuck on reboot or shutdown with 6.2 kernel (tested with 6.2.0-26-generic and 6.2.0-31-generic).

Model : ASUS TUF GAMMING F17
Graphic card : Nvidia GeForce RTX 2050

It uses the Nouveau driver.

You'll find attached the output for the below command:

  journalctl -b -1 --since "1 hour ago"

I cannot test the patched kernel at the moment.

@juergh, we tested your kernel on one of our notebooks where the problem occurred. Shut down and reboot hasn't shown this behaviour, yet.

Looks promising!

Thanks for the confirmation, will send the patch to our mailing list.

After following @juergh updates. I realized the problem was the nouveau drivers, also I saw my fresh installation (Ubuntu 22.04.3) was not detecting my RTX 3070. So I fixed both problems installing the drivers.

> sudo apt install nvidia-driver-535

Now, I can shutdown and/or restart my machine without issues. The only problem is this driver is the propertary one. However in my case is not a problem.

Regards
Juan

I'm affected: LENOVO THINKPAD 16P, nVidia RTX 3060 Mobile.

I can't shutdown the system. Also as @juanma-v82 states no nVidia card is detected and "Software and Updates"->"More Drivers" UI doesn't show the nVidia drivers to install them as if the card is not detected at all; so this could be related to #2034068 I've reported resently.

@juergh,

I want to express my gratitude for your dedication and efforts in investigating the bug. Your involvement has been invaluable, and I truly appreciate it.

I've recently updated to the new kernel version, 6.2.0-32-generic, and will be monitoring.

Additionally, for clarity, I'd like to mention that I'm utilizing the "NVIDIA proprietary driver" rather than the "Nouveau" driver.

This also fixed the problem on my end. My Dell xps 15 9510 NVIDIA GA107m
RTX3050 Ti
Linux version 6.2.0-31-generic (buildd@lcy02-amd64-032)
(x86_64-linux-gnu-gcc-11 (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU
Binutils for Ubuntu) 2.38)

On Tue, Sep 5, 2023 at 12:50 AM Juan Manuel Vicente <
<email address hidden>> wrote:

> After following @juergh updates. I realized the problem was the nouveau
> drivers, also I saw my fresh installation (Ubuntu 22.04.3) was not
> detecting my RTX 3070. So I fixed both problems installing the drivers.
>
> > sudo apt install nvidia-driver-535
>
> Now, I can shutdown and/or restart my machine without issues. The only
> problem is this driver is the propertary one. However in my case is not
> a problem.
>
> Regards
> Juan
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2031352
>
> Title:
> Nouveau driver crash - Ubuntu 22.04.3 LTS stuck on power-off/reboot
> screen
>
> Status in linux package in Ubuntu:
> Confirmed
> Status in linux-hwe-6.2 package in Ubuntu:
> Confirmed
> Status in systemd package in Ubuntu:
> Invalid
> Status in linux-hwe-6.2 source package in Jammy:
> Confirmed
> Status in linux source package in Lunar:
> Confirmed
>
> Bug description:
> [Impact]
>
> After updating to Kernel 6.2 a few days ago, I have been experiencing
> issues with my system's shutdown and reboot functions. During these
> processes, the system becomes unresponsive and hangs on a black
> screen, which displays both the Dell and Ubuntu logos. This issue is
> inconsistent; it happens sporadically. Currently, the only workaround
> I've found to successfully shut down the system is to forcibly power
> off the machine by holding down the power button for 5 seconds.
>
> I've also tested a fresh installation of Ubuntu 22.04.3.
>
> [Fix]
>
> Updated patch from linux-next:
> https://patchwork.freedesktop.org/patch/538562/
>
> [Test Case]
>
> Suspend,resume,shutdown,reboot should all work correctly. No nouveau
> stack trace in the kernel log.
>
> [Where Problems Could Occur]
>
> Limited to nouveau driver that wants to load nonexistent ACR firmware.
> Only nvidia GPUs are affected.
>
> [Additional information]
>
> ----
>
> ProblemType: Bug
> DistroRelease: Ubuntu 22.04
> Package: systemd 249.11-0ubuntu3.9
> ProcVersionSignature: Ubuntu 6.2.0-26.26~22.04.1-generic 6.2.13
> Uname: Linux 6.2.0-26-generic x86_64
> NonfreeKernelModules: nvidia_modeset nvidia
> ApportVersion: 2.20.11-0ubuntu82.5
> Architecture: amd64
> CasperMD5CheckResult: pass
> CurrentDesktop: ubuntu:GNOME
> Date: Mon Aug 14 22:41:14 2023
> InstallationDate: Installed on 2023-08-14 (1 days ago)
> InstallationMedia: Ubuntu 22.04.3 2023.08.13 LTS (20230813)
> MachineType: Dell Inc. XPS 8930
> ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-6.2.0-26-generic
> root=UUID=14d1ee7a-565f-4ba4-b6dd-7bc16e487451 ro quiet splash vt.handoff=7
> SourcePackage: systemd
> UpgradeStatus: No upgrade log present (probably fresh install)
> dmi.bios.date: 03/14/2023
> dmi.bios.release: 1.1
> dmi.bios.vendor: Dell Inc...

@juanma-v82 this also fixed my issue.

> sudo apt install nvidia-driver-535

Linux version 6.2.0-31-generic (buildd@lcy02-amd64-032) (x86_64-linux-gnu-gcc-11 (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38)

Dell xps 15 9510 NVIDIA Corporation GA107M [GeForce RTX 3050 Ti Mobile]

Regards,

Ian

Unfortunately, the bug happened today with me, again.

See the attached log using: journalctl -b -1 --since "1 hour ago".

I use:
- Kernel: 6.2.0-32-generic #32~22.04.1-Ubuntu
- NVIDIA Driver Version: 535.86.05

@juergh, it appears that this bug isn't exclusive to the "Nouveau" driver. I'm using the Nvidia proprietary driver and I am experiencing the same issue.

@cfgnunes This bug report is exclusively about the nouveau driver crash. Yours is a different issue. See bug 2031969.

Hi @juergh,

I'd like to inform you that I was the one who reported this bug. All attached logs are from my machine.

I noticed that you changed the bug description, pointing to an issue with the "Nouveau" driver.

But I'd like to clarify that at no point did I use the Nouveau driver.

Best regards,

Derp, yes, you're right. There seem to be two different issues that result in similarly looking reboot/poweroff/suspend/resume problems. I shouldn't have used your bug to track the nouveau issue, since you're not using nouveau but I've missed that. Apologies for the mess. There's the other bug 2031969 that is probably your problem.

Hi @juergh, more and more users got affected, we step back to 6.1 as a workaround.

What is the state of the patch we tested (see #24).

Thanks!

The patch has been applied to lunar:

https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/lunar/commit/?h=master-next&id=1bf3ed1c23a1

and will show up with kernel 6.2.0-35 (unless there is a security respin before that) from the next SRU cycle 2023.10.02 (https://kernel.ubuntu.com/).

This bug is awaiting verification that the linux/6.2.0-36.37 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-lunar-linux' to 'verification-done-lunar-linux'. If the problem still exists, change the tag 'verification-needed-lunar-linux' to 'verification-failed-lunar-linux'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!