Windows HVM’s for gaming suffers from many performance issues, this thread discusses optimizations I have investigated to approach near bare metal performance.
Warning: below optimizations may increase attack surface which could make your system more vulnerable to attackers. Always consider your threat model.
Apologies in advance, had to remove some links because I’m limited as a new poster to only two links here, I kept the important ones.
GPU and USB Hub Passthrough (#1)
Requirements
- Second USB hub controller for mouse and keyboard, your motherboard may already have one. Otherwise you can buy a PCIe USB Hub and attach it to your motherboard.
- Second dedicated gaming GPU
- A KVM switch (recommended) or a second monitor, mouse, keyboard, audio device.
Without GPU passthrough modern gaming is not possible. This has been discussed already and there’s existing tutorials so I will just skim over what needs to be done and won’t discuss in depth. This tutorial https://forum.qubes-os.org/t/create-a-gaming-hvm/19000 is a good reference you don’t need to configure max-ram-below-4g anymore. The below is sufficient:
- Hide secondary GPU and USB hub from dom0 by modifying GRUB configuration
- In qubes settings for the HVM under devices tab, attach GPU and USB hub devices.
Disk IO Bottleneck (#2)
A major bottleneck I encountered is with Disk IO. My system benchmarks showed extremely high p99 latency, very poor random read, poor sequential read, poor sequential write, and extremely poor p50 latency. Games were very choppy and large amounts of time was spent in system interrupts. This made games almost unplayable for me.
QWT optionally installs Xen PV Drivers for you, but it is not recommended for the below reasons:
- QWT is buggy and there’s a few active issues being investigated, including system crashes. It was not a feasible solution for me.
- QWT bundled xen drivers are potentially compromised (see QSB-091). This would require a 0day to exploit, and it is unknown if they even are backdoored in the first place.
I recommend one of the below solutions, either Option 1 or Option 2:
Option 1: Install Xen PV Drivers
I went with this solution and from my benchmark my disk performance improved by a factor of 5-10x (HDD performance) to 30x (SSD performance) on pretty much all metrics mentioned above. Latency and disk reads dramatically improved, making gaming feasible for me. The downside here is the drivers provided by Xen are not signed so you must enable test signing on Windows.
All below steps are performed in Windows host
- Open elevated CMD prompt in Windows host
- Disable hibernation to avoid incomplete shutdown which may lead to VM disk corruption:
powercfg -H off - Turn on test signing mode:
bcdedit /set testsigning onHopefully Xen project signs new Windows PV drivers one day, then this step won’t be required. - Restart PC
- Download and install 7zip or another program to extract tarballs on Windows
- From xenbits download the following tarballs:
xenbus.tar(Core),xeniface.tar(Core),xennet.tar(Networking),xenvif.tar(Networking),xenvbd.tar(Block devices). Note we also install network PV drivers here. Networking is poor according to Xen wiki, but I didn’t benchmark it myself. - It is recommended but not required to validate the hashes of these tarballs. See this security notice as well as respective hashes for tarballs at time of writing.
- Extract them and install by going into x64 folder (for x64 architecture) and run dpinst.exe to install each one. I installed them following order: xenbus, xeniface, xennet, xenvif, xenvbd
- Reboot.
Option 2: NVMe Passthrough
Get a dedicated gaming SSD and pass it through to your host. The main downside is it requires a second SSD and you can’t use qubes to manage your volumes, no software raid, no LUKS, no LVM.
CPU Pinning
Provided no performance improvement for me, so I won’t discuss. It may be different for the reader and may change depending on workload, although from research it seems other people have my experience and don’t benefit much from it.
File sharing
Sometimes you want to share files between windows and linux. Without QWT, you can setup something like below: vsftpd on linux as FTP server and install filezilla on windows. I won’t provide instructions, refer to qubes firewall doc or other tutorials here for how to set this up.
Other Xen Tuning and Configuration (Not Recommended)
Provided negligible performance improvement for me, so I won’t discuss.
End
If anyone has anything else that helped improve performance, please share.