Seamless GPU passthrough on Qubes OS with VirtualGL

last time I tried it worked out of the box, not sure if xorg.conf is required. I can’t try again on Qubes OS as I can’t get the nvidia driver to load

Maybe you installed the driver, and the offloading was done to the CPU using LLVM, which will run a test like vkcube just fine.

I tried it this weekend, that was what happened when I just installed the driver, without making any changes to xorg.

When I tried using the nvidia driver with the dummyqbs driver, xorg just crashed while trying to initialize.

The way the documentation says you need to configure xorg, doesn’t seem to work with Qubes OS, at least the cookie-cutter setup doesn’t seem to work.

https://download.nvidia.com/XFree86/Linux-x86_64/440.44/README/primerenderoffload.html

https://wiki.archlinux.org/title/PRIME

Nope, I’m sure I’ve been using PRIME when it worked (on 4.1).

I wrote two blog posts about this topic on other distros:

Indeed, I forgot you need an xorg.conf file with X, but on wayland it’s automatically detected without configuration, and PRIME is used in both cases when the variables are set

I can’t launch the appvm, I’ve restricted nvidia gpu in the grub (rd.qubes.hide_pci=01:00:0) in the grub. In the template (debian-12-minimal) I tried different drivers:

  • NVIDIA-Linux-x86_64-535*.run :" You appear to be running an X server; please exit X before installing. For further details, please see the section INSTALLING THE NVIDIA DRIVER in the README available on the Linux driver download page at www.nvidia.com"
  • The nvidia-driver-515 driver as in the website: There, everything fits perfectly
  • nvidia-kernel-open-dkm…

In the bios, i tried discrete card, hybrid card and auto
Anyway, I think I’ve tried everything, I’ve followed all the instructions, but, when I want to launch my appvm, I always have: “cannot connect to qrexec agent for 60 seconds”.

my card is an nvidia Quadro M2200… :cry: I’m tearing my hair out!
PS: avec sys-gui-gpu, everything is working…
PS2: For each new driver installation, I create a new template to be sure there is no conflict

edit: it seems to work in debian-xfce with nvidia-kernel-open-dkm…

1 Like

I think this may be working.

I didn’t do anything with xorg and just was able to run stuff when attaching the nvidia GPU to the template.

i’m not sure if the gpu is accessible however and don’t really know how to test this.

the strange part is I’m not using virtualgl at all.

no wait it’s not

i don’t know what i’m doing

$ nvidia-smi
No devices were found

this leads to:

  NVIDIA Accelerated Graphics Driver for Linux-x86_64 (550.54.14)

You appear to be running an X server. Installing the NVIDIA driver while X
is running is not recommended, as doing so may prevent the installer from
detecting some potential installation problems, and it may not be possible
to start new graphics applications after a new driver is installed. If you
choose to continue installation, it is highly recommended that you reboot
your computer after installation to use the newly installed driver.

             Continue installation      Abort installation                 

What is an X server? Why am I here and what am I doing?

it gave me a lot of messages like

“this makes no sense”

and “stuff is already loaded. are you sure you want to continue? seems dumb!”

but i continued

then it said i need to uninstall the debian packages.

I don’t know what’s going on. It says there’s no nvidia driver loaded. How do I find out if it’s loaded? I’m attaching the GPU and it’s not shutting down like before? I don’t know if that has to do with Prime or what that even is?

I am also seeing “denied get dates + nanoseconds @default” or something like that?

@renehoj or @bamboodabpen does anyone know how i can test this to see if it’s working? i’m trying to run LLMs and I don’t see GPU as an option. nvidia-smi isn’t finding devices.

am i still doing this wrong? if anyone could take pity on me and help it would be nice

IT WORKS! PRAISE JOANNA IT WORKS! I DON’T KNOW WHAT I TYPED OR WHAT HAPPENED! I THINK IT’S BECAUSE OF THE UPDATES?

2 Likes

I got this to work for the GPU, just tested opening firefox and seeing the process in nvidia-smi. How do you suggest getting sound to work? I assume the sound is being ported to the dedicated hdmi on my laptop. I assume I’ll just have to find a device that has hdmi and speakers and hook it to that if I want sound?

Maybe you can open Volume Control (run pavucontrol) in your qube and change the output device from HDMI to Qubes Virtual Audio Sink?

When I open that the virtual audio is already selected.

:rofl:

You have a way with words, that made the feeling of your experience come to life. I needed a good laugh. Thanks

Also, make a backup of your entire system, before something breaks it ! :slight_smile:

1 Like

Is it possible to do this with sys-gui-gpu so I can achieve GPU passthrough that works for multiple VMs of different types, such as Windows, Fedora, Debian, and Nobara, without having to switch monitor input from integrated (Motherboard) to GPU cable?

No, sys-gui-gpu is not about providing 3D acceleration to multiple other qubes, it’s to move GPU and GUI away from dom0:

1 Like

Im not sure where to post this. I did try a request for help but didn’t get any replies

Is it accurate to say there isn’t a known way to share GPU passthrough seamlessly among multiple Guests?

I need the ability to use a Windows VM for Video Editing with my discreet GPU, switch to playing Games with a GPU in Nobara, and train an AI in a Debian VM with my GPU. I don’t want to have to switch monitor inputs, a KVM switch, etc.

I have been trying for six months and have hired multiple developers, but I just can’t get it to work. If I can’t, I think I’m going to have to go back to Windows or at least dual boot.

1 Like

Yes. At least not simultaneously.

1 Like

I don’t need to use them simultaneously. I need them to be seamless So I don’t have to change my monitor input. Am I guessing installing NVIDIA drivers for my 4090 still won’t help with 3d Acceleration? Since I am looking at returning to Windows, I am comfortable with a bit of security risk if it means I can use qubes.

I could use this guide to do it seamlessly, but then it would require setting virtual GL on each guest, and I’m guessing Windows still isn’t possible.

You mean installing NVIDIA drivers in dom0 to get 3d Acceleration in the qubes? No.

I don’t know if it’s possible for Windows.
Maybe it’s possible to implement this feature in new QWT so it’ll output the image rendered by GPU to the Xen virtual display. You can try asking this here:

Did anyone tried with this device ADLINK Pocket AI Pocket AI | Portable GPU | NVIDIA RTX A500 | ADLINK? I looks like a perfect solution to fill the gap not supported GPU for blender and similar.

this is just an external GPU through thunderbolt. I had such setup using a Razer Core X external case and pass through was possible on Qubes OS.

3 Likes