Hang at end of loading, then reboot, on Framework 16

nic-hartley · June 24, 2024, 8:55pm

I was able to install Qubes 4.2.1 without major issues (the trackpad doesn’t work as documented and Ventoy is apparently incompatible with Qubes, but that’s it) but post-install I’ve had a bunch of issues, I think to do with a graphics race condition. They appear to be resolved after the initial setup, but it could be the same root cause so I’m mentioning it.

Currently, I’m hanging at the end of the initial loading screen. I’m able to enter my LUKS password and all that seems to be working, but once the progress bar at the bottom of the screen fills up, everything hangs; after maybe 60 seconds, the laptop just reboots.

I was able to boot Qubes successfully during install and (after a bunch of trouble with black screens and boot loops) for the first-time setup. Then I was able to boot into Qubes successfully exactly once. I added the kernel parameters described in the HCL, and now I’m getting the loop described above, even when I remove the parameters from the grub boot menu.

Removing quiet, adding console=hvc0 earlyprintk=xen for Linux and adding loglvl=all guest_loglvl=all vga=,keep to Xen gives me more logs before the LUKS/loading screen, but after that I end up with the grey loading screen, hang, reboot, no logs. Ctrl+Alt+F1-F7 don’t bring up alternate TTYs. I’ve seen mention of pressing F2 at the LUKS prompt to get a TTY version but that just toggles how the password renders on my machine.

I’ve just tried reinstalling with the “latest kernel” option on my install media (“Install Qubes OS R4.2.1 using kernel-latest (6.7.7-1.qubes.fc37)”). Similar outcome, but instead of the graphical LUKS password prompt it’s a text one.

After trying another reinstall with a fresh download (though the version looked the same) and a different USB stick, it’s back to the same graphical issues as before – i.e. while it’s waiting for the LUKS password to be entered, the screen abruptly goes black. It seems like I can blindly enter my password and continue onto a black screen with a mouse cursor, then the machine reboots.

(Also obligatory first post apology if I’m putting this in the wrong category, missing tags, etc.)

apparatus · June 25, 2024, 5:48am

Remove quiet and add plymouth.enable=0 to the kernel command line options in GRUB, then check the messages that you see when it boots and hangs. Are there any errors or failures? What are the last messages?
You can try to boot with qubes.skip_autostart so your qubes won’t automatically startup at boot in case it’s an issue with your qubes.
Do you have external display connected to the laptop?

nic-hartley · July 6, 2024, 8:12pm

Unfortunately I entirely forgot about this thread after seeing your reply, my bad (I installed a distro I’m more familiar with to get the laptop working quickly, meaning to come back to Qubes in a bit, and… that’s now, I guess, oops.)

From memory at the time:

I tried with and without an external display; I didn’t see any extra logs or different behavior.
There weren’t any seemingly significant error messages. I saw (and still see) things like e.g. pciback 0000:01:00.0: not ready [...]ms after FLR: waiting (and then : giving up after it waits 60s), but they scroll too fast to read and my phone camera is hilariously bad.

In the time since I did a BIOS update, which might confuse the issue. Now when booting without quiet and with plymouth.enable=0 qubes.skip_autostart, it dumps me into the first-time setup screen. I’m currently trying working back through it to see if it’ll work after the BIOS update; failing that I’m going to reinstall from scratch again and try to retrace my steps with better notetaking.

nic-hartley · July 6, 2024, 11:35pm

Alright, I did a fresh reinstall with 4.2.2-rc1 as a hail mary and I’m getting exactly the same behavior.

By default, changing nothing about the boot (for a control):

It hangs for ~2 minutes on a black screen before popping up the grey-background Qubes logo
After maybe another minute it shows the LUKS decrypt prompt, then almost instantly without me even entering the password the screen goes black.
I can then enter the password, which takes me to the black screen+mouse, but then the machine isn’t rebooting, it just sits there, screen black. (Or it hangs for so long I don’t have the patience to wait for it.)

Removing quiet and adding plymouth.enable=0:

Those ~2 minutes it spends hanging are revealed to be that pciback line from before (wow this is getting really annoying)
I get a whole bunch of logs scrolling by too fast to read (and wow I wish my phone camera wasn’t hilariously awful, maybe I’d be able to take a video and read it back)
I end up at a nongraphical LUKS prompt where I can put in my password.
Eventually I end up back at the initial setup screen; if I try to go through it I hit an error about the default logical volume already existing.
Then I boot into Qubes, and everything… seems to work fine, though I’m hardly testing exhaustively
When I shut down it hangs indefinitely on a black screen.

Those changes, then adding qubes.skip_autostart:

Same ~2 minute hang
Nongraphical LUKS prompt
Dropped straight into normal (again seemingly functional) Qubes
Same issue on shutting down (I see the usual “the system is shutting down NOW!” console message, but then the laptop itself just… doesn’t… turn off.)

So it sure seems like one of the autostarted qubes is causing problems. How can I dig into which one? Is there a way to bifurcate em, try autostarting only half? Or look at logs from prior boots? (I promise I did look myself, but I’m having a ton of trouble navigating Qubes’ docs.)

apparatus · July 7, 2024, 2:40am

It should be an issue with PCI device attached to a qube, so it’s an issue with either sys-net or sys-usb.
You can try to start them manually after you boot into dom0 with qubes.skip_autostart and see which one will fail to start.
It’s probably an issue with this PCI device 0000:01:00.0:

So you can check which device is it and to which qube is it attached using this command in dom0 terminal:

qvm-pci

Maybe it’s this issue:

Need to see the boot log to check what is causing this issue.
Maybe it’s an issue with RAID. Check the BIOS settings and change storage mode from RAID to AHCI. Check for other RAID configuration there as well.

nic-hartley · July 7, 2024, 3:43am

despite noticing the pci in the log line it didn’t even once occur to me to look into which PCI device that is. moving on–

lspci shows 01:00.0 is my WiFi card (for reference, a MediaTek MT7922, aka AMD RZ616). That’s odd because my WiFi works. And when trying to boot all my qubes manually after logging in, none of them fail to start.

So now I’m suspecting something’s happening out of order: Something in Xen or Qubes is correctly configuring WiFi once it’s running, but that only happens after the pciback and qube autostart bits.

Not a hundred percent sure if that doc is my exact issue, but I’m going to try fiddling with passthrough regardless. (If you’ve got any resources on doing that for pre-boot, I’d appreciate it greatly, I’m not sure where pciback is being triggered)

Oh, and I’ve just realized, now that the machine is successfully booting I can actually pull the logs! Unfortunately even without quiet I’m not seeing anything indicative of an error, just the “not ready after FLR” lines – probably because autostart is off, so nothing is breaking on boot anymore.

And thank you for the very detailed response!

apparatus · July 7, 2024, 4:15am

You can get the failed boot log like this if your previous boot failed:

journalctl -b-1 > journal.log

Here -1 is a previous boot log, -2 is the one before that etc.

You can also try to disable sys-usb autostart then boot without qubes.skip_autostart.
Then try to enable sys-usb autostart and disable sys-net/sys-firewall/sys-whonix autostart and boot without qubes.skip_autostart.

nic-hartley · July 7, 2024, 6:19pm

I’m… gradually realizing that Qubes is more like a normal Linux system than I thought. Thanks again.

So:

In lspci -vv it looks like the WiFi card is reporting that it supports FLReset+, but then FLR isn’t working.
Disabling sys-usb, autostart works. Enabling it, I get the reboot loop. (And no pciback wait, weirdly?)
I can then start sys-usb… which sends the laptop into a reboot. Looking at the logs for the last boot, there’s an error message at the very end (14.1 KB)

I have no idea how to read that error message, tbh. The only thing that looks meaningful out of the stack trace is screensaver_event_cb (and apparently it’s xss-lock breaking? which… odd.) It’s possible that’s just a coincidence/symptom, but I don’t see any other logs that look like things are erroring. If you’ve got anywhere you think I should look, let me know.

apparatus · July 7, 2024, 6:37pm

So with sys-net autostart enabled and sys-usb autostart disabled you can boot successfully?
Then try to enable the sys-usb autostart, remove all PCI devices from sys-usb and then try to attach the PCI USB controllers to sys-usb one by one and boot with them to figure out if this issue is caused by some specific USB controller.
Also maybe some internal USB device should be present in dom0 during boot for it to work properly.

Seems to be some issue with your GPU, but I’m not sure if it’s related to sys-usb or not.
Do you have the same error message in the normal boot log?
Maybe it’s an issue with your GPU firmware:

github.com/QubesOS/qubes-issues

[amdgpu] GUI freezes and doesn't recover

opened 03:03PM - 23 Jul 22 UTC

Rot127

T: bug C: other P: major hardware support diagnosed waiting for upstream affects-4.1

### Qubes OS release r4.1 CPU: `AMD Ryzen 7 4800H with Radeon Graphics` … ### Brief summary Since the last update (stable branch) the GUI froze three times in one day. The first two times the GUI recovered (I've got logged out after a few seconds and could log in again). The third time it didn't and I had to reset the laptop. I still could move my mouse pointer (USB mouse via default sys-usb) although everything else was frozen. The error logs of `journalctl -x -p 3 -r`: <details> <summary>First freeze</summary> ``` Jul 23 05:48:01 dom0 anacron[9407]: Can't find sendmail at /usr/sbin/sendmail, not mailing output Jul 23 05:38:40 dom0 systemd-coredump[13021]: Process 9589 (xss-lock) of user 1000 dumped core. Stack trace of thread 9589: #0 0x00007b151ce16fc2 g_logv (libglib-2.0.so.0 + 0x59fc2) #1 0x00007b151ce17233 g_log (libglib-2.0.so.0 + 0x5a233) #2 0x000062ee2b144be5 screensaver_event_cb (xss-lock + 0x4be5) #3 0x000062ee2b144cdf xcb_event_dispatch (xss-lock + 0x4cdf) #4 0x00007b151ce0f78f g_main_context_dispatch (libglib-2.0.so.0 + 0x5278f) #5 0x00007b151ce0fb18 g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0x52b18) #6 0x00007b151ce0fe33 g_main_loop_run (libglib-2.0.so.0 + 0x52e33) #7 0x000062ee2b143ecd main (xss-lock + 0x3ecd) #8 0x00007b151cbdd082 __libc_start_main (libc.so.6 + 0x27082) #9 0x000062ee2b14402e _start (xss-lock + 0x402e) Stack trace of thread 9596: #0 0x00007b151ccac86f __poll (libc.so.6 + 0xf686f) #1 0x00007b151ce0faae g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0x52aae) #2 0x00007b151ce0fbe3 g_main_context_iteration (libglib-2.0.so.0 + 0x52be3) #3 0x00007b151ce0fc31 glib_worker_main (libglib-2.0.so.0 + 0x52c31) #4 0x00007b151ce397f2 g_thread_proxy (libglib-2.0.so.0 + 0x7c7f2) #5 0x00007b151cad4432 start_thread (libpthread.so.0 + 0x9432) #6 0x00007b151ccb76d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9599: #0 0x00007b151ccac86f __poll (libc.so.6 + 0xf686f) #1 0x00007b151ce0faae g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0x52aae) #2 0x00007b151ce0fe33 g_main_loop_run (libglib-2.0.so.0 + 0x52e33) #3 0x00007b151d0636aa gdbus_shared_thread_func (libgio-2.0.so.0 + 0x1226aa) #4 0x00007b151ce397f2 g_thread_proxy (libglib-2.0.so.0 + 0x7c7f2) #5 0x00007b151cad4432 start_thread (libpthread.so.0 + 0x9432) #6 0x00007b151ccb76d3 __clone (libc.so.6 + 0x1016d3) -- Subject: Process 9589 (xss-lock) dumped core -- Defined-By: systemd -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel -- Documentation: man:core(5) -- -- Process 9589 (xss-lock) crashed and dumped core. -- -- This usually indicates a programming error in the crashing program and -- should be reported to its vendor as a bug. Jul 23 05:38:39 dom0 systemd-coredump[13007]: Process 9214 (Xorg) of user 0 dumped core. Stack trace of thread 9214: #0 0x0000715796b097d5 raise (libc.so.6 + 0x3c7d5) #1 0x0000715796af2895 abort (libc.so.6 + 0x25895) #2 0x00006098138900e0 OsAbort (Xorg + 0x1c60e0) #3 0x0000609813895959 AbortServer (Xorg + 0x1cb959) #4 0x00006098138966aa FatalError (Xorg + 0x1cc6aa) #5 0x000060981388d450 OsSigHandler (Xorg + 0x1c3450) #6 0x0000715796caba90 __restore_rt (libpthread.so.0 + 0x14a90) #7 0x000071579528fd88 si_emit_cp_dma.isra.0 (radeonsi_dri.so + 0x88fd88) #8 0x000071579528ff19 si_cp_dma_wait_for_idle (radeonsi_dri.so + 0x88ff19) #9 0x000071579529fde0 si_flush_gfx_cs (radeonsi_dri.so + 0x89fde0) #10 0x000071579528fb35 si_cp_dma_prepare (radeonsi_dri.so + 0x88fb35) #11 0x0000715795290074 si_cp_dma_clear_buffer (radeonsi_dri.so + 0x890074) #12 0x000071579528dd1b si_clear_buffer (radeonsi_dri.so + 0x88dd1b) #13 0x000071579528e145 si_pipe_clear_buffer (radeonsi_dri.so + 0x88e145) #14 0x000071579529c537 si_sdma_clear_buffer (radeonsi_dri.so + 0x89c537) #15 0x000071579529cd73 si_screen_clear_buffer (radeonsi_dri.so + 0x89cd73) #16 0x000071579526f025 si_texture_create_object (radeonsi_dri.so + 0x86f025) #17 0x000071579526f9e8 si_texture_create (radeonsi_dri.so + 0x86f9e8) #18 0x0000715794b32959 st_texture_create (radeonsi_dri.so + 0x132959) #19 0x0000715794b2451c guess_and_alloc_texture (radeonsi_dri.so + 0x12451c) #20 0x0000715794b2488c st_AllocTextureImageBuffer (radeonsi_dri.so + 0x12488c) #21 0x0000715794b28c2e st_TexImage (radeonsi_dri.so + 0x128c2e) #22 0x0000715794ca0c0f teximage_err (radeonsi_dri.so + 0x2a0c0f) #23 0x0000715794ca2ad2 _mesa_TexImage2D (radeonsi_dri.so + 0x2a2ad2) #24 0x00007157962644f2 _glamor_create_tex (libglamoregl.so + 0x254f2) #25 0x00007157962648f4 glamor_create_fbo (libglamoregl.so + 0x258f4) #26 0x0000715796249d46 glamor_create_pixmap (libglamoregl.so + 0xad46) #27 0x0000609813722b6e ProcCreatePixmap (Xorg + 0x58b6e) #28 0x0000609813727a2b Dispatch (Xorg + 0x5da2b) #29 0x000060981372bb04 dix_main (Xorg + 0x61b04) #30 0x0000715796af4082 __libc_start_main (libc.so.6 + 0x27082) #31 0x0000609813714e6e _start (Xorg + 0x4ae6e) Stack trace of thread 9237: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9239: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9238: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9236: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9235: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9240: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9246: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9244: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9247: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9249: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9242: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9253: #0 0x0000715796bcea1e epoll_wait (libc.so.6 + 0x101a1e) #1 0x000060981388dd9b ospoll_wait (Xorg + 0x1c3d9b) #2 0x000060981388b4c1 InputThreadDoWork (Xorg + 0x1c14c1) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9245: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9243: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9241: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9248: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) Stack trace of thread 9250: #0 0x0000715796ca6e92 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xfe92) #1 0x0000715794e9d20b util_queue_thread_func (radeonsi_dri.so + 0x49d20b) #2 0x0000715794e9cccb impl_thrd_routine (radeonsi_dri.so + 0x49cccb) #3 0x0000715796ca0432 start_thread (libpthread.so.0 + 0x9432) #4 0x0000715796bce6d3 __clone (libc.so.6 + 0x1016d3) -- Subject: Process 9214 (Xorg) dumped core -- Defined-By: systemd -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel -- Documentation: man:core(5) -- -- Process 9214 (Xorg) crashed and dumped core. -- -- This usually indicates a programming error in the crashing program and -- should be reported to its vendor as a bug. Jul 23 05:38:38 dom0 kernel: [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring gfx timeout, but soft recovered Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x000080010406a000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104068000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104069000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104068000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x000080010406a000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104069000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104068000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104065000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104064000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x3 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x1 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: TCP (0x8) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00101031 Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104063000 from IH client 0x1b (UTCL2) Jul 23 05:38:38 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:38 dom0 kernel: [drm:amdgpu_dm_commit_planes [amdgpu]] *ERROR* Waiting for fences timed out! Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104042000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104045000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104041000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104042000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104040000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104043000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104041000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104044000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CB (0x0) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00000000 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104040000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x3 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x1 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: TCP (0x8) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00101031 Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800104042000 from IH client 0x1b (UTCL2) Jul 23 05:38:28 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:24 vmid:1 pasid:32770, for process Xorg pid 9214 thread X:cs0 pid 9235) ``` </details> Second one in comment below. <details> <summary>Third and final</summary> ``` -- Reboot -- Jul 23 10:14:37 dom0 kernel: [drm:amdgpu_job_timedout [amdgpu]] *ERROR* Process information: process Xorg pid 21089 thread X:cs0 pid 21100 Jul 23 10:14:37 dom0 kernel: [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring gfx timeout, signaled seq=1837320, emitted seq=1837320 Jul 23 10:14:27 dom0 kernel: [drm:amdgpu_job_timedout [amdgpu]] *ERROR* GPU Recovery Failed: -110 Jul 23 10:14:27 dom0 kernel: [drm:amdgpu_device_ip_resume_phase2 [amdgpu]] *ERROR* resume of IP block <sdma_v4_0> failed -110 Jul 23 10:14:27 dom0 kernel: amdgpu 0000:04:00.0: [drm:amdgpu_ring_test_helper [amdgpu]] *ERROR* ring sdma0 test failed (-110) Jul 23 10:14:27 dom0 kernel: [drm:amdgpu_job_timedout [amdgpu]] *ERROR* Process information: process Xorg pid 21089 thread X:cs0 pid 21100 Jul 23 10:14:27 dom0 kernel: [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring gfx timeout, signaled seq=1837318, emitted seq=1837320 Jul 23 10:14:27 dom0 kernel: [drm:amdgpu_dm_commit_planes [amdgpu]] *ERROR* Waiting for fences timed out! Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x1 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x0 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0x5 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x0 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CPG (0x6) Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00740C50 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000800100ad1000 from IH client 0x1b (UTCL2) Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:40 vmid:7 pasid:32769, for process Xorg pid 21089 thread X:cs0 pid 21100) Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: RW: 0x0 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MAPPING_ERROR: 0x1 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: PERMISSION_FAULTS: 0xb Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: WALKER_ERROR: 0x6 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: MORE_FAULTS: 0x0 Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: Faulty UTCL2 client ID: CPF (0x4) Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: VM_L2_PROTECTION_FAULT_STATUS:0x00B009BC Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: in page starting at address 0x0000be000000c000 from IH client 0x1b (UTCL2) Jul 23 10:14:16 dom0 kernel: amdgpu 0000:04:00.0: amdgpu: [gfxhub0] no-retry page fault (src_id:0 ring:222 vmid:11 pasid:0, for process pid 0 thread pid 0) ``` </details> ### Steps to reproduce I am not aware of any particular things which let to this. ### Expected behavior No freezing. ### Actual behavior It did.

nic-hartley · July 7, 2024, 7:26pm

Yep, with sys-net autostarting, things work fine. With sys-usb autostarting, reboot. When I manually start sys-usb, reboot. I’ve just gone through the PCIe devices in sys-usb, and…

I have six PCIe devices, c1:00.{3,4} and c3:00.{3,4,5,6}. They’re all USB controllers; 5 and 6 are also labelled as “Thunderbolt”. After some experimentation, it seems like c3:00.4 is the issue – if it’s attached to sys-usb and I boot the VM, my laptop reboots. If not, sys-usb boots fine and adds devices as expected. if only c3:00.4 is attached, reboot.

As far as I can tell, c3:00.4 is exactly identical to c3:00.3. So this could be a case of faulty hardware, but for now I can just remove c3:00.4 from sys-usb and keep on with getting Qubes fully functional (notably, fixing poweroffs and the pciback stuff.) At some point I’ll test whether this is the expansion slot itself or the card plugged into it, but for now I’ve got other weekend plans.

Thanks again for your help! I’ll probably be back to it next weekend.

apparatus · July 8, 2024, 4:00am

You can boot in dom0 without sys-usb autostart so your USB controllers are attached to dom0 and check if there are any USB devices connected to c3:00.4 USB controller. Maybe some USB device is related to this issue.
Maybe you’ll need to disable USBGuard in dom0 as well to check this USB device. For this you’ll need remove usbcore.authorized_default=0 from GRUB kernel command line options during boot.

IVY · August 27, 2024, 8:48am

Hi @nic-hartley,

I’ve been encountering the same issue as you and I’ve been wondering if you managed to find out the issue which is causing the crash.

In my attempts to find out what is causing the issue I could also determine that it’s coming from sys-usb and removing the PCIe device c3:00.4 allowed me to start sys-usb.

I also removed all adapters with all PCIs devices attached to see if it was a ‘broken’ adapter, which was not the case.

One thing that was weird is that everything work when the .4 device is removed and the laptop is standalone and nothing is plugged in. When adding a DELL docking station with .4 still removed the issue with qubes crashing appears.

Here were my last log lines with only sys-usb attached, all PCI devices attached, plymouth.enable=0, and quiet removed:

xhci_hcd 0000:c3:00.4: remove, state 4
usb usb8: USB disconnect, device number 1
xhci_hcd 0000:c3:00.4: USB bus 8 deregistered
xhci_hcd 0000:c3:00.4: remove, state 4
usb usb7: USB disconnect, device number 1
xhci_hcd 0000:c3:00.4: USB bus 7 deregistered
xhci_hcd 0000:c3:00.4: xen_pciback: seizing device
Already setup the GSI :47

apparatus · August 27, 2024, 11:15am

Maybe it’s this issue:

github.com/QubesOS/qubes-issues

Hardware reset during installation and boot of R4.2 on Ryzen 9 7950X

opened 07:30PM - 04 Jul 23 UTC

Eric678

T: bug C: installer P: major hardware support C: usb proxy needs diagnosis affects-4.2

[How to file a helpful issue](https://www.qubes-os.org/doc/issue-tracking/) #…## Qubes OS release R4.2.0-rc1 + Ryzen 9 7950X + Gigabyte X670E motherboard ### Brief summary Installation proceeds normally till just after "Configure networking" when hardware resets. Further system boots reset just after entering disk password. ### Steps to reproduce Run a default installation of R4.2.0-rc1. ### Expected behavior No hardware resets. ### Actual behavior As noted. Problem appears to be caused by a single USB controller being mapped into sys-usb. There are 5 USB controllers on the CPU and 670 chipset, only one causes a problem. It is the last one in the devices list, address 37:00.0. Workaround is to add qubes.skip_autostart option to the linux kernel boot parameters at any boot after installation, then unmap this controller from sys-usb once system is up. I suspect that it is the on CPU controller that is used for the mouse and keyboard as others on different VM systems on the same CPU have a problem mapping running USB devices causing a hardware reset.