I have run into an issue where I recently rebooted a long running Qubes R4.2.2 box, so updates to the VM’s got utilized, and now a USB device has become unstable and unusable. I am not sure if this a Dom0 issue or perhaps an issue in the AppVM(s) involved. This has been working fine for months, and the only change (on my part) was rebooting.
I have an AppVM using the Debian-12-xfce template that has a USB card passed through to it. The card has a couple of ethernet dongles on it. One dongle gets used directly by the AppVM. This works fine using it locally on the AppVM.
The second dongle gets mapped to another AppVM using a Debian-12-xfce based template updated with ZFS. This is where the problem has started. Previously, the mapping worked perfectly, and the dongle was usable on the AppVM. Now the dongle keeps disappearing with the following error message appearing:
Error
Attaching device 0bda:8153_Realtek_USB_10_100_1000_LAN_DF{mac address}
to {VM name} failed. Error: QubesException - Device attach failed: Timed out for waiting the udev queue being empty.
I have tried replacing the dongle with a different one, same issue. I have tried moving it to a different port on the USB card, same issue.
Has a patch caused a bug perhaps? If there is some place I should be looking for more debugging info, please point me to it and I would be happy to collect and post.
Dennis
Did you try to attach some simpler USB device e.g. USB flash disk?
I’d try to install the original debian-12-xfce template without your modifications to it and try to use it for both “sys-usb” qube and the test qube to which you’re trying to attach the USB device from sys-usb.
Thanks for the reply.
I have used the same setup in the past to mount USB flash drives between these same qubes, though I am not sure if it is impacted as well currently. I passed a flash drive between them a few days ago without incident, but I am unsure if that was before or after I rebooted the system. So it may either be unaffected, or just had not manifested yet.
I will setup a test to see if the issue affects a vanilla qube without ZFS support. For the affected qube, ZFS is mandatory as it hosts a large drive array for file serving.
The other possibility for the cause, may be some sort of race condition caused by resource contention. Perhaps due to increases in traffic, the ZFS qube is no longer able to service the USB device fast enough. The ZFS qube has 20GB and 6 VCPUs assigned currently. I will try and up the VCPUs to see if that helps. For memory, I am tapped out as the box only has 64 GB, and other qubes eat up the rest. The box in question has 16 physical cores, so I have extra CPU power to spare.
My plan to workaround this for the moment is to add another USB controller that will be passed through to the ZFS qube, so that the dongle will be a local device and will no longer be passed through by Qubes USB support.
I have a larger box ready that I was planning to upgrade to (24 cores, 128 GB), but it has almost no ability for adding discrete USB cards, so this issue is a potential blocker for that plan as USB support has to work for the new box to be usable.
I will let you know if the vanilla qube or increase to resources (VCPUs) changes the outcome.
A bit of feedback.
The increase to the VCPUs was ineffective.
I noticed that when assigning the dongle on to the ZFS qube, when setting up the IP address, the udev-worker thread blocks (D), as does any command accessing the interface. Then the assigned interface disappears, and the commands complete/fail.
However if I wait a bit before trying to assign an address, the assignment succeeds, but the previously posted error message appears as soon as I put the interface under stress (E.g accessing a large video file over it), and the interface drops off.
Assigning it to a vanilla debian-12-xfce works fine. The interface was usable and accepted the address right away. I did not go past that as this was a noticeable difference from the ZFS qube already.
Assigning USB drives and copying from them works OK. I am copying 75 GB over using the assigned drive right now.
That being said, I do recall having a few copy failures at least once in the past few months. When it occurred I suspected the DJT camera the content was coming from was the culprit, but it may be that this issue can affect USB drives, just that it is a much rarer case.
How much memory did you assign to the qube with attached PCI USB controller?
Did you disable memory balancing for it?
I have 20 GB of memory assigned and memory balancing is disabled on the VM that dongle is being passed to, and 4 GB (memory balancing disabled) on the VM it is being passed from.
I can try more memory on the VM hosting the USB device.
Knock on wood, adding 2 GB of memory to the hosting VM (6 GB total now), may have helped. I have been able to stream a video off the VM housing the ZFS array. Time will tell if this is a fix as I have got this far before only to have it fail in short order.
Hopefully this is the fix. The USB card I added turned out to use a NEC chipset and I have been stuck in firmware hell trying to get it usable under the VM to no avail.