QubesOS freeze, crash and reboots

:worried:

Once again, during update ...

Nov 21 09:37:39 dom0 kernel: Linux version 5.15.76-1.fc32.qubes.x86_64 (mockbuild@2b0cec5c8b1143fc8fd66539678daa>
– Reboot –
Nov 21 09:35:42 dom0 qrexec-policy-daemon[29902]: 2022-11-21 09:35:42.471 qrexec-client[29902]: process_io.c:187>
Nov 21 09:35:40 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.NotifyUpdates+: debian-11-net → @adminvm: allowe>
Nov 21 09:35:38 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.UpdatesProxy+: debian-11-net → @default: allowed>
Nov 21 09:35:37 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.NotifyUpdates+: debian-11-net → @adminvm: allowe>
Nov 21 09:35:35 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.UpdatesProxy+: debian-11-net → @default: allowed>
Nov 21 09:35:33 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.UpdatesProxy+: debian-11-net → @default: allowed>
Nov 21 09:35:31 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.UpdatesProxy+: debian-11-net → @default: allowed>
Nov 21 09:35:27 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.UpdatesProxy+: debian-11-net → @default: allowed>
Nov 21 09:35:23 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.VMRootShell+: disp-mgmt-debian-11-net → debian-1>
Nov 21 09:35:23 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.VMRootShell+: disp-mgmt-debian-11-net → debian-1>
Nov 21 09:35:22 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.GetDate+: debian-11-net → @default: allowed to d>
Nov 21 09:35:19 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.VMRootShell+: disp-mgmt-debian-11-net → debian-1>
Nov 21 09:35:19 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.WindowIconUpdater+: debian-11-net → @adminvm: al>
Nov 21 09:35:18 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.VMRootShell+: disp-mgmt-debian-11-net → debian-1>
Nov 21 09:35:18 dom0 qrexec-policy-daemon[1502]: qrexec: qubes.VMRootShell+: disp-mgmt-debian-11-net → debian-1>

1 Like

Quick side note for others who might in this context see a notification that “domain disp-mgmt-bla already exists with uuid …”. @marmarek posted in a qubes-issue comment how to fix this:

virsh -c xen:/// undefine disp-mgmt-bla

Had my first freeze today, after a debian template update finished and the template was about to shutdown.

1 Like

I’m tracking kernel-latest, as the stable kernel did not solve issues
for me.
I had disappointing response to my request for information from users.

I have reverted my system to vanilla out of the box settings, and scaled
down my qube use.
I still see hard crash, and freezes (sometimes recovered), in normal
use, particularly during updates or qube start/stop.
I also see arbitrary reaping of running qubes.

There are two obvious areas which could impact - I/O and memory. I
suspect memory reallocations between qubes.
(As I have said before, I have swapped in 4.0 on an identical SSD, and
see none of these issues with aggressive memory allocation to qubes.)

1 Like

FWIW, after spending quite a bit of time trying to solve graphical issues I had, I found that there were quite a few issues/posts scattered around reporting crashes with intel /i915 that in hindisght seem closely related or even downright duplicates.
So far: issues # 4782, 7507, 7664, 7902, 7894. Forum post. Blog post (+ I remember reading another two but haven’t written down their urls).

The issue is that there’s tearing/glitches/artifacts/corruption with the fb driver when there’s no compositing (which i3wm doesn’t provide - probably explaining the high proportion of reports from i3wm users). People then switch to the intel driver because that’s the solution mentioned in posts to fix those glitches - but for some (yet to be found) reason intel is unstable for some people - from oopses to hard freeze/reboot.

In my case I’d get a few random reboots a day. I eventually made the connection and reverted to fb and so far haven’t encountered a single crash. In hindsight, given all the problems reported in this thread, I’m wondering which proportion is due to people using the intel driver instead of fb and being hit by bugs in intel

4 Likes

Seems to happen for me on Qubes Updates; never during a Dom0 update – had hoped those recent updates were addressing this. Oddly, and I’m not certain this is related, the crashes only occur for me when the laptop (Librem 14) is running on battery.

Thank you @taradiddles, very interesting.

I’ve never changed anything regarding the graphics and do see occasional tearing. Mostly when using the laptop screen at 60Hz and almost never when using the external 4K screen at 30Hz. And mostly in XTerm windows. In any case I do see freezes and from your summary I conclude that I am using ‘fb’ (default).

Of course that’s just one data point.

1 Like

Most likely. One way to find out is grep "Load.*intel" /var/log/Xorg.0.log: with `intel, it would return:

(II) LoadModule: "intel"
(II) Loading /usr/lib64/xorg/modules/drivers/intel_drv.so

Another way is to use sudo lsof +D /usr/lib64/xorg/modules/drivers/ ; with intel:

Xorg    [...] /usr/lib64/xorg/modules/drivers/intel_drv.so

with fb:

Xorg    [...] /usr/lib64/xorg/modules/drivers/modesetting_drv.so
2 Likes

Ok @taradiddles this has been interesting. I’m one of the i3wm users and I can force an instant crash/reboot just by converting a window to floating and move it between my monitors. I have also been using the “intel” fix to sort out some screen artifacts for months now. So I removed that fix and restarted, then attempted my crash and burn experiment… and whaddya know… no crash, no reboot!

You’ve certainly made things look up for me, thanks mate. I’ll continue to monitor this for now.

2 Likes

Goot to read that ! You’re back to tearing issues then (better than crashes :slight_smile: ). Installing a compositing manager should fix them, until someone finds why fbdev has those issues without compositing.

i3wm's faq mentions using compton as a compositing manager. (it’s an old faq, maybe there’s something more recent). Do you think you could give it a try it if you have some time and report back ? (no worries if you can’t). I’m updating the intel gfx troubleshooting community doc and this would definitely help other i3wm users.

Happy I could help !

@taradiddles I’ve not been a fan of installing software in dom0 so have avoided throwing a compositing package in there. Is there a stable, trustworthy version of compton even out there? I know it’s been forked several times, so you know… :thinking:

@qub411 Ah, I thought it would as simple as qubes-dom0-update compton :frowning:

Given that i3wm are usually Qubes OS power users maybe someone in the Qubes team would volunteer to build/provide a package.
(Pinging @Demi to see if there’s such interest).

[edit - I see that fedora32 packages picom and compton - I now understand that you prefer not to install additional packages - it’s not about building then installing in dom0 like I originally thought]

Otherwise we’d have to wait until the issue with fbdev is fixed but I have no idea how high it is in the devs’ priorities.

I have never had intel “fix” so probably that’s why I had so few crashes/freezes/logouts/reboots…
Thanks for the insight.

As per @taradiddles suggestion, reinstating the intel driver fix with the following option has also eliminated the crash/reboots for me so far:

Section “Device”
Identifier “Intel Graphics”
Driver “intel”
Option “AccelMethod” “UXA”
EndSection

2 Likes

I had that option enabled yesterday too and didn’t encounter a crash for 8+ hours - but it could just be luck. However my laptop then froze on suspend and I had to hard reset it - something I never encountered with fbdev.
If anyone does more tests and/or encounters issues please write (or PM) so that I can update the intel gfx troubleshooting doc accordingly…

I am one of those i3wm users who have experienced freezes in the past months, which resulted in a downscaling of my QubesOS usage.

Do you think using AwesomeWM or DWM with QubesOS help with this? Do these window managers require the (now commonplace) fix: I3 not working properly video/graphic issues - #4 by bungali

Because if I am understanding you correctly, you suggest that this fix put more pressure on intel CPU with graphics processing, which the old thinkpad x220 CPU has troubles with and freezes.

So, I wonder if using Awesome or DWM wouldn’t require that “fix” and thus would result in no-freezes.

This is nice to hear, but without that “intel” fix, you should be seeing graphical artefacts and tears on the screen. Do you see those? If so, do you think using AwesomeWM or DWM instead of i3wm result in no-tears/graphical glithces, but yet still a low-resource tiling manager use with Qubes?

AFAIK neither AwesomeWM nor DWM provide compositing so like i3wm you’d need to install a compositing manager. But more importantly, I don’t think AwesomeWM / DWM are supported by Qubes OS - eg. you won’t see the qubes’ window colors/labels which is a no-go for most security-conscious users.

Do these window managers require the (now commonplace) fix: I3 not working properly video/graphic issues - #4 by bungali

If you don’t install a compositing manager and you experience tearing, then yes (but then there’s a chance you’ll be hit by bugs/crashes in intel). If you have compositing then fbdev is OK.

To summarize, those are the two workarounds:

  • fbdev (which is Qubes’ default) with compositing. i3wm users will have to install compton or picom (both are packaged in f32 so can be installed in dom0 as any other package). This should fix tearing (I have yet to read from a i3wm user with this fix - if you give it a try please send a PM or reply here).

  • intel with “UXA” acceleration method instead of the the default, newer “SNA”. “UXA” fixes some crashes (like the one @qub411 had) but introduces others (eg. when suspending my T450s laptop). YMMV.

Because if I am understanding you correctly, you suggest that this fix put more pressure on intel CPU with graphics processing, which the old thinkpad x220 CPU has troubles with and freezes.

It’s a software bug - overloading your x220 (CPU and/or GFX) should only make your laptop sluggish, not trigger a crash. FWIW I’d expect the intel driver to be more efficient / less resource intensive than fbdev.

Thanks for the overall reply.

One point I would like to ask: I don’t think I understand your point around “compositing.” When I installed i3wm, I don’t remember setting up a “compositor.” Does it get installed by itself, as a dependency to i3wm?

AwesomeWM is supported by Qubes1. DWM is supported thanks to @3o14r473 2

i3wm doesn’t provide a compositor like xfce/gnome/kde do (to enable fancy effects), so you’ll have to install a standalone compositing manager. An old i3wm faq mentions compton as compositing manager but there’s also picom (the last fork in a long chain). It seems it’s only a matter of running the manager alongside i3wm (or awesomewm or dwm for that matter).
Sorry, I’m not a i3wm user so I can’t be of more help - I got hit by the same bugs as i3wm users because I had disabled compositing in xfce to decrease bloat a long time ago before an update introduced tearing.

Oh, I stand corrected ; I’ll add those WMs to the troubleshooting doc.

1 Like