Thanks for this great resource
Because I recently spent a lot of time on an obscure issue, I would like to add it here if appropriate
If you have USB controllers that are put into a bad state at boot via FLR, it is likely caused by sysfs operations in the initramfs script which supports usb-sys configurations
There’s a Qubes-specific workaround inspired by this proxmox post
The issue manifests as:
- FLR waiting and timeout of USB controllers at boot
- If the system eventually boots, you will see invalid and missing data for the device in
lspci -v - You may receive more FLR timeouts and a fail to boot sys-usb after a couple of minutes, - You may receive an “invalid PCI header” message from libvirt when starting sys-usb.
In both cases of the sys-usb start cases, the error is fatal and sys-usb won’t actually boot
My Qubes forum post on this is here
For convenience, the workaround is to modify /usr/share/dracut/modules.d/90qubes-pciback/qubes-pciback.sh (unmodified source is [here])(qubes-core-admin-linux/dracut/modules.d/90qubes-pciback/qubes-pciback.sh at 3f0afb7030276f7afc0212bbaec187abdab72860 · QubesOS/qubes-core-admin-linux · GitHub)
Add the following at the top of the file, one per problematic BDF (any line before the sysfs operations occur)
BDF1=f4:00.0
BDF2=8d:00.0
# Fool the kernel into thinking there are
# no reset mechanisms available for
# the specified BDFs, to prevent implicit
# FLR requests from being sent during
# bind/unbind to/from a driver via sysfs
echo "" > /sys/bus/pci/devices/0000:$BDF1/reset_method
echo "" > /sys/bus/pci/devices/0000:$BDF2/reset_method
That allows the controllers that “break” when issued FLR to successfully be assigned to pciback without going into a bad state.
They can then be attached to a sys-usb (you may at that point need the well-documented no_strict_reset=True)
This issue (the precise trigger, and the solution) is unique to USB controllers and separate from FLR issues with GPUs and other devices, because there is special logic to handle USB controllers when you have sys-usb and a USB keyboard
I’ve asked in the linked post for a better way to resolve the issue, but the above method works