@enmus is nice, my post only guided him, the real solution is in his answer.
I can’t really help you diagnose the xl dmesg output in bold, as my buffer ring exploded, I’m not using an Intel CPU, and those architectural-related things are chinese to me, still learning what they mean exactly !
This may tell me you’re trying to assign devices from the same IOMMU to different domains.
I found this old github issue about RMRR, didn’t read through the end but stopped at this comment : fail to assign pci device at start of usbvm · Issue #1544 · QubesOS/qubes-issues · GitHub, maybe that’ll help to pinpoint the issue.
Another advice, maybe useless on recent versions
Try the PCI “permissive” option, but IIRC that must be avoided for security reasons. Plus I have no idea how to set that up from Qubes (I only just discovered how to manually edit individualVM config files). On vanilla Xen, in the domU config file, you would have to set the permissive option on the pci assignments lines for individual devices, or using a pci_permissive=1 option for applying it globally to all PCI-PT devices of the domU.
But check below before trying that, or try using it via the manual way at the end of the post.
Again, don’t know how it’s handled by Qubes, but considering vanilla Xen and my usage/experience, one of the lines below seems in error to me.
For a device that would be PT’ed, the kernel driver should be pciback, or the module xen_pciback, so like this :
Kernel driver in use: pciback
Kernel modules: snd_hda_intel
OR
Kernel driver in use: snd_hda_intel
Kernel modules: xen_pciback
Maybe check the Xen PCI passtrough guide for known PCI-PT errors, and how it’s done, and you could compare with how it’s done on Qubes ? There are static or dynamic assignments, using the kernel driver or the module. I’ll gladly read your findings about the differences btw, that would enhance my Qubes knowledge ! So much to learn ^^
Another strange thing :
Dunno what that means, but maybe that’s an “important” device after all, on Intel platforms ? Here, nothing like that.
Anyways, good luck and thanks, you make me learn a lot with your problem ^^
Manual assignments
Last thing, but I don’t know if it’s recommended AT ALL on Qubes, here are the Xen commands useful to manage PCI-PT devices, use at your own discretion ! Except the 1st command which is totally harmless. Don’t worry though, nothing survives a dom0 reboot.
If trying this, first remove the device from the Qubes manager before launching the domU, as Qubes scripts handle that usually.
# list all devices available for PT
xl pci-assignable-list
# 1. make a device assignable
xl pci-assignable-add [OPTION] DEVICE_BDF
# 2. start the domU via Qubes manager
# 3. hotplug the device to the domU
xl pci-attach [OPTION] DOMID DEVICE_BDF
# or if wanna try permissive mode
xl pci-attach [OPTION] DOMID 'DEVICE_BDF,permissive=1'
# hotUNplug the device from the domU (care with -f, your domU may crash)
xl pci-detach [OPTION] [-f] DOMID DEVICE_BDF
# 4. remove assignable status (care with -r, it reassigns the device to dom0)
xl pci-assignable-remove [OPTION] [-r] DEVICE_BDF
Square brackets used for optional switches, caps used for replacements.
To get a DOMID, do xl list or xl domid DOMAIN_NAME
[OPTION] means you can use xl global options like -v for verbose, and -N for a dry-run (dont do anything, just output what would be done, if possible).
You shouldn’t have to use -f and -r, especially on Qubes for -r, but then you may have to reboot to do more tests. If you want more info, man xl explains what the switches do, and more about the above commands.
Offtopic
That’s freaking good to know, thanks !!! It’s also a nice pointer to help me solve the remarks you quoted ! More work to do ^^ In fact, that may be of GREAT help to make nested Qubes working, as I just discovered about virtual IOMMUs, but as you said, off topic 