Thank you. A bit of a rhetorical question, hoping that I won’t waste the whole day.
I documented some of the procedure. But I believe some key points might be missing:
Are you using i915 or xe gpu driver for this?
I was able to successfully enable sr-iov with i915 on a 12th gen CPU, but starting the HVM with one of the GPUs attached never worked.
Thanks for doing the write-up and sharing!
I will be incredibly happy if I can finally get GPU acceleration working with SR-IOV.
Make sure to share your github when its ready and add a link so I can donate a beer or ten to you.
From my experience, blocking the xe module in dom0 SR-IOV does not work. I have not yet been able to determine why. It will be easier if I record a screen from dom0. There is also a problem with Qubes 4.2 and the xorg version with the x11 intel driver (no acceleration for MeteorLake, only LLVM). I think that by moving everything to 4.3 and xen 4.19 it will work without a problem.
VIDEO: https://youtu.be/C-7a5dgdd64
Can you share how you got the i915 driver working? I might be able to carry you through the last mile.
In next two weeks i write some how to do in 4.2 and 4.3 (begin tested)
The way I installed the sr-iov patches is via dkms:
- install dkms kernel-devel kernel-latest-devel in dom0 and gpu-hvm template
- download latest release zip from GitHub - strongtz/i915-sriov-dkms: dkms module of Linux i915 driver with SR-IOV support
- copy into dom0
qvm-run --pass-io <src-vm> 'cat /path/to/i915-sriov-dkms-2025.02.03' > /path/to/i915-sriov-dkms-2025.02.03
and to gpu-hvm templateqvm-copy i915-sriov-dkms-2025.02.03.zip
- unzip release zip
unzip i915-sriov-dkms-2025.02.03.zip
- add to dkms via
dkms add ./i915-sriov-dkms-2025.02.03
- install module via
dkms install i915-sriov-dkms/2025.02.03
- kernel commandline needs to be adjusted:
vim /etc/default/grub
and changeGRUB_CMDLINE_LINUX_DEFAULT
tointel_iommu=on i915.enable_guc=3 i915.max_vfs=7 module_blacklist=xe
, or add to it if you have other arguments there already. - update grub config
grub2-mkconfig -o /boot/efi/EFI/qubes/grub.cfg
(dom0) - rebuild initramfs
dracut --regenerate-all --force
- after rebuild I can enable the virtual devices via
echo 7 > /sys/devices/pci0000\:00/0000\:00\:02.0/sriov_numvfs
I did the same on the VM template in my case fedora-40 only leaving out the i915.max_vfs=7
kernel parameter.
After setting everything up and making sure all kernel options are in place I tried passing through one of the GPUs to the new VM but I get the libxenlight failed to create a domain
when trying to start it with GPU attached. Permissive / no-strict-reset for the gpu didn’t fix it.
I’ll try to create a debian-13 template analog to 1x3n1x’s setup and will see if it changes anything. If not I guess I can wait another 2 weeks for his howto after many years without gpu acceleration on Qubes
This is a strongtz modules way via dkms and i dont like this. I dont use this way. But if you do not blacklist xe module in kernel may work. If not try create modprobe.conf.d witch
options pciback hide=(0000:00:02.1)(0000:00:02.2)(0000:00:02.3)(0000:00:02.4)(0000:00:02.5)(0000:00:02.6)(0000:00:02.7)
and check dmesg | grep i915 and GuC/HuC firmware versions. You must see like this:
i915 0000:00:02.5: [drm] GT0: GUC: interface version 0.1.17.0
i915 0000:00:02.5: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
Thanks for the hints! Will make sure to check it out.
So you are not using strongtz patches but are building a complete kernel from linux-intel-lts?
With some patches by me for MeteorLake i made some changes for sleep etc. Module i915 from strongtz github repository should work out of the box but sleep states frozen hardware and intel ipex with pytroch not working well
No luck with strongtz modules on Tiger Lake either. Same issue as tze with libxenlight failed to start domain.
The libxl_create error is basically that the device model did not start: -9
.
I tried so hard, and got so far, but in the end it didn’t even matter
someone was nice enough to open a github issue: SR-IOV slice passthrough: libxenlight reports failed to create domain, device model not starting, error -9 · Issue #9853 · QubesOS/qubes-issues · GitHub
I tried hiding the sr-iov gpu devices with pciback like suggested by 1x3n1x but it did not solve the issue. The GuC/HuC firmware version were correct and were successfully loaded.
I guess the strongz implementation is lacking something.
I hope it’s not a hardware generation difference, 1x3n1x is running meteor lake, i’ve only tried on alder lake and raptor lake-s
I have a Meteor Lake PC that I can test with later this week. I’ll report the results after attempting with the Strongtz modules again.
I was able to get past the libxenlight error on my Tigerlake PC by using the fix posted by @marmarek at SR-IOV slice passthrough: libxenlight reports failed to create domain, device model not starting, error -9 · Issue #9853 · QubesOS/qubes-issues · GitHub
I’m able to launch Qubes with an attached iGPU. However, the i915 driver does not seem be in use by iGPU while inside of the Guest OS.
From Dom0:
<user>@dom0 ~]$ sudo dkms status
i915-sriov-dkms/2025.03.27 6.12.18-1.qubes.fc37.x86_64, x86_64: installed (original_module exists)
<user>@dom0 ~]$ uname -a
Linux dom0 6.12.18-1.qubes.fc37.x86_64 #1 SMP PREMPT_DYNAMIC Tue Mar 11 05:05:34 GMT 2025 x86_64 x86_64 x86_64 GNU/Linux
From the Guest OS:
user@personal:~$ uname -a
Linux personal 6.12.18-1.qubes.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 11 05:50:34 GMT 2025 x86_64 GNU/Linux
user@personal:~$ sudo modprobe i915
modprobe: ERROR: could not insert 'i915': No such device
user@personal:~$ sudo dkms status
i915-sriov-dkms/2025.03.27, 6.12.18-1.qubes.fc37.x86_64, x86_64: installed
user@personal:~$ sudo lspci -nnkk -s 007
00:07.0 VGA compatible controller [0300]: Intel Corporation TigerLake-LP GT2 [Iris Xe Graphics] [8086:9a49] (rev 01)
Subsystem: Lenovo TigerLake-LP GT2 [Iris Xe Graphics] [17aa:22c9]
Kernel modules: xe, i915
Normally the output should have a line that says kernel driver in use: i915
, but this is missing from my Qube.
That being said, I still haven’t gotten around to testing on Meteor Lake yet.
I was also able to get it working on my alder lake laptop and finally have working gpu acceleration
Will test on my Raptor Lake-S workstation soon.
Since a lot of people run intel systems i’ll try to combine all the information and write a “complete” howto.
@Qubernetes I had to change the Kernel of the AppVM to “provided by qube” and install the strongtz into the AppVM. You’d also want to force recreate your initramfs images to include the patched kernel module. For fedora that’s dracut --regenerate-all --force
. Make sure the /boot/grub2/grub.cfg in the AppVM includes i915.enable_guc=3 module_blacklist=xe
otherwise edit /etc/default/grub
and regenerate via grub2-mkconfig -o /boot/grub2/grub.cfg
After the gpu is successfully passed through and i915 module is loaded for it you have to remove the variable LIBGL_ALWAYS_SOFTWARE=1
from the ENV of the AppVM / Template. I chose the hammer method and removed /etc/profile.d/qubes.sh & qubes.csh
from my template. I’m sure there is a more elegant way to do it, but i’m unsure how to disable the software-rendering property:
bash-5.2# cat qubes-gui.sh
export DISPLAY=:0 _JAVA_AWT_WM_NONREPARENTING=1
if test -f /var/run/qubes-service/software-rendering; then
export GSK_RENDERER="cairo" \
GDK_DISABLE="gl vulkan" \
GDK_DEBUG="gl-disable vulkan-disable" \
LIBGL_ALWAYS_SOFTWARE=1
fi
Thanks, this was the last bit that I was missing! My previous attempts failed with dom0 providing the AppVM kernel and with debian-12-xfce as my AppVM template. But after switching over to fedora-40-xfce and setting the AppVM kernel to (provided by qube), everything is now working.
Hardware acceleration is awesome!
So far SR-IOV vGPUs seem to be working pretty well on TigerLake.
On the other hand, I had a chance to test the strongtz modules with AlderLake MeteorLake and unfortunately I can’t make it past booting into dom0 when the i915 driver is loaded. After entering the encryption passphrase, I’m greeted with a blinking cursor. It seems like it’s stalling somewhere between normal startup and loading Xorg. @1x3n1x, did you experience anything like this prior to patching the intel-lts module version?