I’m not here discussing if AMD or Nvidia is better, I want to know which is better for Qubes at the higher end. It’s no surprise to the Qubes devs that if you own a newer Nvidia GPU and you own a high refresh monitor, you are stuck at 60hz in qubes unless you install proprietary drivers which could break qubes and the driver should be reinstalled everytime there is a kernel change in dom0, this is really annoying. It seems like this isn’t the case with AMD GPUs. What about general other performance, is AMD always better than Nvidia?
I’m building a new system specifically for qubes and I was wondering what GPU to get.
if they made their drivers upstream in mesa and kernel, it would works out of the box, that’s the case for AMD and Intel. They have an open source driver now for more recent models, I don’t know if it works great, but current dom0 version is too old to have them unfortunately.
For dom0, this does not really matter because it does almost nothing except show windows that are rendered on CPUs within qubes.
If you were to use a second GPU in qubes with passthrough, both works.
The card itself “just works” with any Linux distro. As for Qubes and PCI passthrough, it’s about the motherboard + GPU combination. I know of no good way to “check” if it’s going to work with Qubes without just trying it.
I’ve tried GPU PCI passthrough with 2 motherboard/GPU combos and they’ve always worked pretty nicely, but you can never be fully sure.
TLDR: Nvidia may be better for Passthrough, but I’d love to hear other people’s experience/views.
I have long preferred to buy AMD, because of the 2 problems with Nvidia:
Poor support of open source by the manufacturer.
Deliberately disabled passthrough in all but the most expensive cards.
AMD seemed to be a model citizen.
After a lot of problems trying to get various AMD GPUs to work with passthrough, I came across the following blog posts, one about the “AMD reset bug”:
The Proxmox forums are full of discussion of the AMD reset bug. It may be a good place to search for information about your specific card.
It seems to me that Nvidia GPUs should just support PT.
For AMD, it depends (I think) on the details of the specific GPU chipset and of the card manufacturer’s implementation - and apparently it’s a hardware problem, although it can be worked around with a out-of-tree module. It did get fixed in some generations, only to reappear in later ones.
Most annoying is that AMD do not seem to wish to provide a fix themselves, which leaves me feeling fairly disgusted at my small pile of AMD graphics cards, and looking to test the switch to Nvidia.
I have read about that being a good combination… but I have no actual experience of trying Nvidia. It does allow to blacklist the drivers for only one of the cards, although the stub drivers help with that.
If you’re interested in passthrough, I have also discovered that:
It’s important to check the iommu groupings of the motherboard*. Difficult information to find, and there are tales of bios updates changing the groupings. Not helpful!
It’s very useful to be able to select which GPU is used as the primary one - depends on the BIOS/Firmware. Some do not allow it.
It would be really nice to have this information in the HCL.
* on my most recent MB, a (quite old) MSI B550-chipset one (AMD AM4), the second GPU slot is mixed up with a lot of the peripheral devices, so unusable for PT. It requires a lot of effort just to get the main Dom0 gui on it, to free up the primary - which I can pass through but not (yet) use, due to reset problems, I think.
those links (the content) are badly outdated… so don’t waste your time
And as others mentioned, and I can also confirm:
any recent hardware needs testing… Even just to say if it would work or not. And nobody will try your desired setup… unless you lucky enough to be able to buy the exact same parts (Mobo, CPU, VGA) that somebody already tried, and shared it’s result. - but this is very rare in practice.
I’m also currently building a new (gamer) PC, which primary goal will be gaming, but I will try to build it as in preference:
a Qubes PC, with VGA passtrough to achieve a gaming VM
a KVM based virtual gaming platform - which would still acceptable compromise
a pure windows gaming rig - only as a last resort
This is my preference order, and I will need to try it myself… after I bought all the parts.
My setup is:
ASRock B850 PRO-A
Ryzen 7 9700X
ASUS RTX 5070 Dual OC (don’t have it yet)
So my plan is to use the iGPU for dom0 and Qubes in general, and passing the RTX 5070 to a windows VM, (using a separate display) for gaming. - at least this is the plan.
I only have the Mobo + CPU + RAM so far, so I just started to find out the IOMMU mappings (which is promising), solving the USB keyboard + mouse ‘issue’ (in progress), trying out basic PCI passtrought with the Ethernet card (seems to be working OOB)
As soon as I got the VGA, and have some result, I’m planning to share the details… - but again it is only helpful for others if they have the same hardware components. Without such matching it might be just another ‘works for me’ post.
Honestly, I’m still annoyed at myself that didn’t find that Community Guide, and I have anyway a selection of hardware that is itself
… and I reckon I would be very happy to see “just another ‘works for me’ post” in the HCL, with the most detailed description of what works for passthrough, would be very welcome.
Reminds me that I must do a “don’t buy this” one for my cheap and nasty MSI B550 Gaming Gen3.
