That's what I attached. When I could not get back to a proper graphical environment, I switched to a TTY, ran `sudo journalctl -b0 > journalctl-b0-20230829.log`, and I attached it to that bug. As you can see in the logs, the device had been running since August 21, and on Aug 28 evening I suspended it:
Aug 28 19:34:36 coltrane systemd[1]: Reached target Sleep.
Aug 28 19:34:36 coltrane systemd[1]: Starting Record successful boot for GRUB...
Aug 28 19:34:36 coltrane systemd[1]: Starting Restart Syncthing after resume...
Aug 28 19:34:36 coltrane systemd[1]: Starting System Suspend...
Aug 28 19:34:36 coltrane systemd-sleep[903135]: Entering sleep state 'suspend'...
Aug 28 19:34:36 coltrane kernel: PM: suspend entry (deep)
Aug 28 19:34:36 coltrane systemd[1]: grub-common.service: Deactivated successfully.
Aug 28 19:34:36 coltrane systemd[1]: Finished Record successful boot for GRUB.
Aug 28 19:34:36 coltrane systemd[1]: Starting GRUB failed boot detection...
and then resumed it on Aug 29 at 09:12, and this is when the stack trace appears:
Aug 29 09:12:26 coltrane kernel: Filesystems sync: 0.018 seconds
Aug 29 09:12:31 coltrane kernel: Freezing user space processes
Aug 29 09:12:31 coltrane kernel: Freezing user space processes completed (elapsed 0.003 seconds)
Aug 29 09:12:31 coltrane kernel: OOM killer disabled.
Aug 29 09:12:32 coltrane kernel: Freezing remaining freezable tasks
Aug 29 09:12:32 coltrane kernel: Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
Aug 29 09:12:32 coltrane kernel: printk: Suspending console(s) (use no_console_suspend to debug)
Aug 29 09:12:32 coltrane kernel: sd 1:0:0:0: [sda] Synchronizing SCSI cache
Aug 29 09:12:32 coltrane kernel: sd 1:0:0:0: [sda] Stopping disk
Aug 29 09:12:32 coltrane kernel: sd 2:0:0:0: [sdb] Synchronizing SCSI cache
Aug 29 09:12:32 coltrane kernel: sd 3:0:0:0: [sdc] Synchronizing SCSI cache
Aug 29 09:12:32 coltrane kernel: sd 4:0:0:0: [sdd] Synchronizing SCSI cache
Aug 29 09:12:32 coltrane kernel: sd 2:0:0:0: [sdb] Stopping disk
Aug 29 09:12:32 coltrane kernel: sd 4:0:0:0: [sdd] Stopping disk
Aug 29 09:12:32 coltrane kernel: sd 3:0:0:0: [sdc] Stopping disk
(...)
Aug 29 09:12:32 coltrane kernel: kworker/u8:24: page allocation failure: order:5, mode:0x40d00(GFP_NOIO|__GFP_COMP|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0
Aug 29 09:12:32 coltrane kernel: CPU: 3 PID: 903164 Comm: kworker/u8:24 Not tainted 6.2.0-26-generic #26~22.04.1-Ubuntu
Aug 29 09:12:32 coltrane kernel: Hardware name: ASUS All Series/B85M-K, BIOS 3602 03/26/2018
Aug 29 09:12:32 coltrane kernel: Workqueue: events_unbound async_run_entry_fn
Aug 29 09:12:32 coltrane kernel: Call Trace:
Aug 29 09:12:32 coltrane kernel: <TASK>
Aug 29 09:12:32 coltrane kernel: dump_stack_lvl+0x48/0x70
Aug 29 09:12:32 coltrane kernel: dump_stack+0x10/0x20
Aug 29 09:12:32 coltrane kernel: warn_alloc+0x14b/0x1c0
Aug 29 09:12:32 coltrane kernel: ? __alloc_pages_direct_compact+0xa7/0x240
Aug 29 09:12:32 coltrane kernel: __alloc_pages_slowpath.constprop.0+0x910/0x990
Aug 29 09:12:32 coltrane kernel: __alloc_pages+0x32c/0x360
Aug 29 09:12:32 coltrane kernel: __kmalloc_large_node+0x89/0x170
Aug 29 09:12:32 coltrane kernel: kmalloc_large+0x21/0xc0
Aug 29 09:12:32 coltrane kernel: dc_set_power_state+0x49/0x190 [amdgpu]
Aug 29 09:12:32 coltrane kernel: dm_suspend+0x9f/0x290 [amdgpu]
Aug 29 09:12:32 coltrane kernel: ? vi_common_set_clockgating_state+0xd7/0x190 [amdgpu]
Aug 29 09:12:32 coltrane kernel: amdgpu_device_ip_suspend_phase1+0xb9/0x1d0 [amdgpu]
Aug 29 09:12:32 coltrane kernel: amdgpu_device_suspend+0xca/0x1d0 [amdgpu]
Aug 29 09:12:32 coltrane kernel: amdgpu_pmops_suspend+0x33/0x50 [amdgpu]
Aug 29 09:12:32 coltrane kernel: pci_pm_suspend+0x8a/0x1c0
Aug 29 09:12:32 coltrane kernel: ? __pfx_pci_pm_suspend+0x10/0x10
Aug 29 09:12:32 coltrane kernel: dpm_run_callback+0x54/0x1a0
Aug 29 09:12:32 coltrane kernel: __device_suspend+0x14b/0x400
Aug 29 09:12:32 coltrane kernel: async_suspend+0x1f/0x80
Aug 29 09:12:32 coltrane kernel: async_run_entry_fn+0x33/0x130
Aug 29 09:12:32 coltrane kernel: process_one_work+0x21f/0x440
Aug 29 09:12:32 coltrane kernel: worker_thread+0x50/0x3f0
Aug 29 09:12:32 coltrane kernel: ? __pfx_worker_thread+0x10/0x10
Aug 29 09:12:32 coltrane kernel: kthread+0xee/0x120
Aug 29 09:12:32 coltrane kernel: ? __pfx_kthread+0x10/0x10
Aug 29 09:12:32 coltrane kernel: ret_from_fork+0x2c/0x50
Aug 29 09:12:32 coltrane kernel: </TASK>
Aug 29 09:12:32 coltrane kernel: Mem-Info:
Aug 29 09:12:32 coltrane kernel: active_anon:6 inactive_anon:2748750 isolated_anon:0 active_file:309 inactive_file:232 isolated_file:0 unevictable:41 dirty:1 writeback:8 slab_reclaimable:60650 slab_unreclaimable:80021 mapped:1 shmem:70425 pagetables:29671 sec_pagetables:0 bounce:0 kernel_misc_reclaimable:0 free:106406 free_pcp:0 free_cma:0
Aug 29 09:12:32 coltrane kernel: Node 0 active_anon:24kB inactive_anon:10995000kB active_file:1236kB inactive_file:928kB unevictable:164kB isolated(anon):0kB isolated(file):0kB mapped:4kB dirty:4kB writeback:32kB shmem:281700kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 2048kB writeback_tmp:0kB kernel_stack:42288kB pagetables:118684kB sec_pagetables:0kB all_unreclaimable? no
Aug 29 09:12:32 coltrane kernel: Node 0 DMA free:13308kB boost:0kB min:64kB low:80kB high:96kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15988kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Aug 29 09:12:32 coltrane kernel: lowmem_reserve[]: 0 3380 15822 15822 15822
Aug 29 09:12:32 coltrane kernel: Node 0 DMA32 free:197068kB boost:32448kB min:46872kB low:50476kB high:54080kB reserved_highatomic:0KB active_anon:0kB inactive_anon:2122496kB active_file:0kB inactive_file:0kB unevictable:80kB writepending:0kB present:3606752kB managed:3541040kB mlocked:80kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Aug 29 09:12:32 coltrane kernel: lowmem_reserve[]: 0 0 12442 12442 12442
Aug 29 09:12:32 coltrane kernel: Node 0 Normal free:215248kB boost:119452kB min:172544kB low:185816kB high:199088kB reserved_highatomic:2048KB active_anon:24kB inactive_anon:8872504kB active_file:1040kB inactive_file:1136kB unevictable:84kB writepending:36kB present:13090816kB managed:12748956kB mlocked:84kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Aug 29 09:12:32 coltrane kernel: lowmem_reserve[]: 0 0 0 0 0
Aug 29 09:12:32 coltrane kernel: Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 1*512kB (U) 0*1024kB 2*2048kB (UM) 2*4096kB (M) = 13308kB
Aug 29 09:12:32 coltrane kernel: Node 0 DMA32: 49272*4kB (UME) 1*8kB (U) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 197096kB
Aug 29 09:12:32 coltrane kernel: Node 0 Normal: 53516*4kB (UMEH) 44*8kB (UMH) 19*16kB (H) 16*32kB (UH) 5*64kB (H) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 215552kB
Aug 29 09:12:32 coltrane kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Aug 29 09:12:32 coltrane kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Aug 29 09:12:32 coltrane kernel: 71214 total pagecache pages
Aug 29 09:12:32 coltrane kernel: 245 pages in swap cache
Aug 29 09:12:32 coltrane kernel: Free swap = 176108kB
Aug 29 09:12:32 coltrane kernel: Total swap = 2097148kB
Aug 29 09:12:32 coltrane kernel: 4178389 pages RAM
Aug 29 09:12:32 coltrane kernel: 0 pages HighMem/MovableOnly
Aug 29 09:12:32 coltrane kernel: 102050 pages reserved
Aug 29 09:12:32 coltrane kernel: 0 pages hwpoisoned
(...)
That's what I attached. When I could not get back to a proper graphical environment, I switched to a TTY, ran `sudo journalctl -b0 > journalctl- b0-20230829. log`, and I attached it to that bug. As you can see in the logs, the device had been running since August 21, and on Aug 28 evening I suspended it:
Aug 28 19:34:36 coltrane systemd[1]: Reached target Sleep. sleep[903135] : Entering sleep state 'suspend'... service: Deactivated successfully.
Aug 28 19:34:36 coltrane systemd[1]: Starting Record successful boot for GRUB...
Aug 28 19:34:36 coltrane systemd[1]: Starting Restart Syncthing after resume...
Aug 28 19:34:36 coltrane systemd[1]: Starting System Suspend...
Aug 28 19:34:36 coltrane systemd-
Aug 28 19:34:36 coltrane kernel: PM: suspend entry (deep)
Aug 28 19:34:36 coltrane systemd[1]: grub-common.
Aug 28 19:34:36 coltrane systemd[1]: Finished Record successful boot for GRUB.
Aug 28 19:34:36 coltrane systemd[1]: Starting GRUB failed boot detection...
and then resumed it on Aug 29 at 09:12, and this is when the stack trace appears:
Aug 29 09:12:26 coltrane kernel: Filesystems sync: 0.018 seconds GFP_NOIO| __GFP_COMP| __GFP_ZERO) , nodemask= (null), cpuset= /,mems_ allowed= 0 lvl+0x48/ 0x70 0x10/0x20 0x14b/0x1c0 pages_direct_ compact+ 0xa7/0x240 pages_slowpath. constprop. 0+0x910/ 0x990 pages+0x32c/ 0x360 large_node+ 0x89/0x170 large+0x21/ 0xc0 power_state+ 0x49/0x190 [amdgpu] 0x9f/0x290 [amdgpu] set_clockgating _state+ 0xd7/0x190 [amdgpu] device_ ip_suspend_ phase1+ 0xb9/0x1d0 [amdgpu] device_ suspend+ 0xca/0x1d0 [amdgpu] pmops_suspend+ 0x33/0x50 [amdgpu] suspend+ 0x8a/0x1c0 pm_suspend+ 0x10/0x10 callback+ 0x54/0x1a0 suspend+ 0x14b/0x400 0x1f/0x80 entry_fn+ 0x33/0x130 one_work+ 0x21f/0x440 thread+ 0x50/0x3f0 thread+ 0x10/0x10 0x10/0x10 fork+0x2c/ 0x50 anon:2748750 isolated_anon:0
active_ file:309 inactive_file:232 isolated_file:0
unevictab le:41 dirty:1 writeback:8
slab_ reclaimable: 60650 slab_unreclaima ble:80021
mapped: 1 shmem:70425 pagetables:29671
sec_ pagetables: 0 bounce:0
kernel_ misc_reclaimabl e:0
free: 106406 free_pcp:0 free_cma:0 anon:10995000kB active_file:1236kB inactive_file:928kB unevictable:164kB isolated(anon):0kB isolated(file):0kB mapped:4kB dirty:4kB writeback:32kB shmem:281700kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 2048kB writeback_tmp:0kB kernel_ stack:42288kB pagetables:118684kB sec_pagetables:0kB all_unreclaimable? no highatomic: 0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15988kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB highatomic: 0KB active_anon:0kB inactive_ anon:2122496kB active_file:0kB inactive_file:0kB unevictable:80kB writepending:0kB present:3606752kB managed:3541040kB mlocked:80kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB highatomic: 2048KB active_anon:24kB inactive_ anon:8872504kB active_file:1040kB inactive_ file:1136kB unevictable:84kB writepending:36kB present:13090816kB managed:12748956kB mlocked:84kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB size=1048576kB size=2048kB
Aug 29 09:12:31 coltrane kernel: Freezing user space processes
Aug 29 09:12:31 coltrane kernel: Freezing user space processes completed (elapsed 0.003 seconds)
Aug 29 09:12:31 coltrane kernel: OOM killer disabled.
Aug 29 09:12:32 coltrane kernel: Freezing remaining freezable tasks
Aug 29 09:12:32 coltrane kernel: Freezing remaining freezable tasks completed (elapsed 0.001 seconds)
Aug 29 09:12:32 coltrane kernel: printk: Suspending console(s) (use no_console_suspend to debug)
Aug 29 09:12:32 coltrane kernel: sd 1:0:0:0: [sda] Synchronizing SCSI cache
Aug 29 09:12:32 coltrane kernel: sd 1:0:0:0: [sda] Stopping disk
Aug 29 09:12:32 coltrane kernel: sd 2:0:0:0: [sdb] Synchronizing SCSI cache
Aug 29 09:12:32 coltrane kernel: sd 3:0:0:0: [sdc] Synchronizing SCSI cache
Aug 29 09:12:32 coltrane kernel: sd 4:0:0:0: [sdd] Synchronizing SCSI cache
Aug 29 09:12:32 coltrane kernel: sd 2:0:0:0: [sdb] Stopping disk
Aug 29 09:12:32 coltrane kernel: sd 4:0:0:0: [sdd] Stopping disk
Aug 29 09:12:32 coltrane kernel: sd 3:0:0:0: [sdc] Stopping disk
(...)
Aug 29 09:12:32 coltrane kernel: kworker/u8:24: page allocation failure: order:5, mode:0x40d00(
Aug 29 09:12:32 coltrane kernel: CPU: 3 PID: 903164 Comm: kworker/u8:24 Not tainted 6.2.0-26-generic #26~22.04.1-Ubuntu
Aug 29 09:12:32 coltrane kernel: Hardware name: ASUS All Series/B85M-K, BIOS 3602 03/26/2018
Aug 29 09:12:32 coltrane kernel: Workqueue: events_unbound async_run_entry_fn
Aug 29 09:12:32 coltrane kernel: Call Trace:
Aug 29 09:12:32 coltrane kernel: <TASK>
Aug 29 09:12:32 coltrane kernel: dump_stack_
Aug 29 09:12:32 coltrane kernel: dump_stack+
Aug 29 09:12:32 coltrane kernel: warn_alloc+
Aug 29 09:12:32 coltrane kernel: ? __alloc_
Aug 29 09:12:32 coltrane kernel: __alloc_
Aug 29 09:12:32 coltrane kernel: __alloc_
Aug 29 09:12:32 coltrane kernel: __kmalloc_
Aug 29 09:12:32 coltrane kernel: kmalloc_
Aug 29 09:12:32 coltrane kernel: dc_set_
Aug 29 09:12:32 coltrane kernel: dm_suspend+
Aug 29 09:12:32 coltrane kernel: ? vi_common_
Aug 29 09:12:32 coltrane kernel: amdgpu_
Aug 29 09:12:32 coltrane kernel: amdgpu_
Aug 29 09:12:32 coltrane kernel: amdgpu_
Aug 29 09:12:32 coltrane kernel: pci_pm_
Aug 29 09:12:32 coltrane kernel: ? __pfx_pci_
Aug 29 09:12:32 coltrane kernel: dpm_run_
Aug 29 09:12:32 coltrane kernel: __device_
Aug 29 09:12:32 coltrane kernel: async_suspend+
Aug 29 09:12:32 coltrane kernel: async_run_
Aug 29 09:12:32 coltrane kernel: process_
Aug 29 09:12:32 coltrane kernel: worker_
Aug 29 09:12:32 coltrane kernel: ? __pfx_worker_
Aug 29 09:12:32 coltrane kernel: kthread+0xee/0x120
Aug 29 09:12:32 coltrane kernel: ? __pfx_kthread+
Aug 29 09:12:32 coltrane kernel: ret_from_
Aug 29 09:12:32 coltrane kernel: </TASK>
Aug 29 09:12:32 coltrane kernel: Mem-Info:
Aug 29 09:12:32 coltrane kernel: active_anon:6 inactive_
Aug 29 09:12:32 coltrane kernel: Node 0 active_anon:24kB inactive_
Aug 29 09:12:32 coltrane kernel: Node 0 DMA free:13308kB boost:0kB min:64kB low:80kB high:96kB reserved_
Aug 29 09:12:32 coltrane kernel: lowmem_reserve[]: 0 3380 15822 15822 15822
Aug 29 09:12:32 coltrane kernel: Node 0 DMA32 free:197068kB boost:32448kB min:46872kB low:50476kB high:54080kB reserved_
Aug 29 09:12:32 coltrane kernel: lowmem_reserve[]: 0 0 12442 12442 12442
Aug 29 09:12:32 coltrane kernel: Node 0 Normal free:215248kB boost:119452kB min:172544kB low:185816kB high:199088kB reserved_
Aug 29 09:12:32 coltrane kernel: lowmem_reserve[]: 0 0 0 0 0
Aug 29 09:12:32 coltrane kernel: Node 0 DMA: 1*4kB (U) 1*8kB (U) 1*16kB (U) 1*32kB (U) 1*64kB (U) 1*128kB (U) 1*256kB (U) 1*512kB (U) 0*1024kB 2*2048kB (UM) 2*4096kB (M) = 13308kB
Aug 29 09:12:32 coltrane kernel: Node 0 DMA32: 49272*4kB (UME) 1*8kB (U) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 197096kB
Aug 29 09:12:32 coltrane kernel: Node 0 Normal: 53516*4kB (UMEH) 44*8kB (UMH) 19*16kB (H) 16*32kB (UH) 5*64kB (H) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 215552kB
Aug 29 09:12:32 coltrane kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_
Aug 29 09:12:32 coltrane kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_
Aug 29 09:12:32 coltrane kernel: 71214 total pagecache pages
Aug 29 09:12:32 coltrane kernel: 245 pages in swap cache
Aug 29 09:12:32 coltrane kernel: Free swap = 176108kB
Aug 29 09:12:32 coltrane kernel: Total swap = 2097148kB
Aug 29 09:12:32 coltrane kernel: 4178389 pages RAM
Aug 29 09:12:32 coltrane kernel: 0 pages HighMem/MovableOnly
Aug 29 09:12:32 coltrane kernel: 102050 pages reserved
Aug 29 09:12:32 coltrane kernel: 0 pages hwpoisoned
(...)