GPU Passthrough

[anon42499160]

Hey,

I am relatively new to Qubes and haven’t found a explanation that suits my knowledge.

So, from my understanding no Qube uses the GPU besides the gui-vm? This mean’s I can’t take advantage of my GPU in my Qubes.

But because of an interesting project of mine (I want to run my own ml models), I need a good GPU to run certain things. I am using the Nvidia RTX 3070 & noveau driver.

So, to achive that I need to do the following:

1. Check if IOMMU is activated in Bios for pci passthrough
2. Create a HVM qube
3. Find pci-id of my GPU with lspci | grep -i nvidia
4. qvm-pci attach dom0:
5. install nvidia-drivers in the qubes with apt install nvidia-driver firmware-misc-nonfree or sudo dnf install akmod-nvidia xorg-x11-drv-nvidia

Is this all? Did I forget / oversee things? and while writing I recognized is it probably not a good idea to have noveau in dom0 and the properitary driver in the qubes, so i will have to stick to one of them, right?

thanks in advance!

[apparatus]

Running local LLMs with or without GPU acceleration

This guide explains how to install text-generation-webui (oobabooga) on Qubes OS 4.2.0, it can be used with nvidia, amd, and intel arc GPUs, and/or CPU. I will only cover nvidia GPU and CPU, but the steps should be similar for the remaining GPU types. The GPU used is the nvidia 4060, it might not be exactly the same for nvidia GPUs that use the legacy driver. Not having a GPU is going to greatly limit the size of model you can use, and even small models are going to take relatively long to exe…

@anon42499160

and while writing I recognized is it probably not a good idea to have noveau in dom0 and the properitary driver in the qubes, so i will have to stick to one of them, right?

It doesn’t matter since dom0 won’t use this GPU because it’s passthrough to another qube.

[anon42499160]

@apparatus

It doesn’t matter since dom0 won’t use this GPU because it’s passthrough to another qube.

So this means only one qube can use the gpu at a time?

Thanks for the guide, i will definitely read into it.

[apparatus]

@anon42499160

So this means only one qube can use the gpu at a time?

Yes.