Screencasting from Qubes OS

Edit: For a solution, see below.

I use qubes-video-companion to get my dom0 screen into the app VM running OBS and have my Nvidia GPU in there as well to do the encoding (following this guide).

Big issue for me, running this on a not-so-beefy CPU, is that the rendering (not the encoding) runs off the CPU entirely while this could be done by my GPU as well quite effortlessly. According to Claude, the rendering is done by OBS via OpenGL and thus handed off to llvmpipe on Qubes OS. It becomes an issue for me as the system-wide CPU usage hits 100% and the fps are still too low for my use case.

Any way to offload the rendering to my GPU?

Note: This is a testing machine, so security tradeoffs would be accepted.


A solution: I got this working; my CPU now sits nicely at around 10% in OBS, GPU mclk and wattage went up, by which I assume it does more stuff, and fps are fine now. Here is what I did on Debian 13 (Claude was used):

  1. Download .deb file from VirtualGL releases page and install using sudo dpkg -i filenamehere.deb
  2. sudo vglserver_config, choose the EGL option, answer “no” to the framebuffer access question
  3. echo 'export VGL_DISPLAY=egl' | sudo tee -a /etc/profile
  4. Restart VM.
  5. Run OBS (I installed it from the repo and not via Flatpak) with vglrun obs

Note on security: These five steps should not weaken security.