I am using a HP G2 thunderbolt dock and the displayport’s work. The only issue with those is I have to reconfigure the display layout every time I connect the dock. My main issue is that I cannot use any of the USB-A or USB-C ports on the dock. I have looked around the documentation and the forums. I haven’t been able to figure anything out and I was wondering if someone can point me to what I need to do.
The only issue with those is I have to reconfigure the display layout every time I connect the dock.
That’s by design. I’ll be the first to say that Qubes doesn’t do multiple displays very well (…yet, but it’s only a matter of time).
My main issue is that I cannot use any of the USB-A or USB-C ports on the dock.
Have you managed to find which PCI device the ports are located on?
If you can, check the output of dom0
lsusb -t. That will tell you which devices to pass through.
Post the output too, if you feel comfortable doing so, but feel free to redact any info
Update: this is probably a bad idea, I just learned that, if you do this, then the USB qube will only run if you boot the computer with the dock connected. I’ll keep the original post so you don’t have to repeat my mistake.
I haven’t used it enough to call this a solution, but early results seem promising if I remove the USB qube and then re-create it with my Caltech dock connected. The the very least, the dock’s USB ports work the same way as the computer’s built-in ports. The next step is to try various tests around both hot and cold-plugging the dock. I also need to try out its audio, video, and network ports.
I followed the instructions at USB Qubes | Removing a USB qube and then the command to create one from the top of that document:
sudo qubesctl state.sls qvm.sys-usb
Maybe this might help?
udev would be the place to start to “reset” your ports when you plug in and remove the dock. That way, you wouldn’t need to cold-boot (unless of course, the BIOS took care of it, and I’d be horrified if it did…),
Thank you, @alzer89. I’ll have to review your article a bit more carefully. It could be part of a solution for Qubes. Am I correct in understanding this would let me hot-plug the dock into dom0? After that, I’ll need to figure out how to handle the USB controllers that are only present when the dock is connected.
Does Qubes support multiple USB qubes? I’m wondering if it would help to have a sys-usb-dock that I would only run if the dock is connected. I’m thinking of adding the dock’s USB controllers to this new qube, while the laptop’s built-in controllers would remain with sys-usb.
Alternately, can we programmatically add and remove devices from existing qubes?
Well, almost everything goes into dom0 at some point, because dom0 has the passthrough kernel modules and drivers.
The reason I brought this up is that I find it weird (and incredibly un-Linuxy) that the dock only works from a cold boot. That would imply that the dom0 kernel (or potentially the BIOS, but hopefully not) is essentially telling the kernel the system tree (with the dock either connected or unconnected).
(I say BIOS because I recently discovered how proprietary the rackmount server industry is, and how big a role BIOSes actually play in server hardware, when I got a ProLiant DL380p Gen8; so I honestly wouldn’t be surprised if HP’s BIOSes “micromanage” your machine, instead of the OS…)
But all that information should be able to be modified with
udev rules and hooks. For example, you could set up scripts for your machine to run when ports have things plugged into them, etc.
That system tree can be modified and remapped without a reboot. Pretty much, the only thing in -nix OSes that actually requires a reboot is the kernel itself (and even then, live patching is possible, albeit it so painfully annoying to do that it’s often easier to just reboot).
The issue that we’ll have to solve is how to update your machine’s system tree to what it looks like from cold boot with the dock connected. My guess is that
udev will be the answer.
Check to see what your system tree looks like with the dock connected (and functional) and without the dock connected. Once we know that information, we can then figure out a way to remap it whenever the dock is connected/disconnected.
See that’s the thing. Am I right in assuming that this dock contains things other than USB ports? Like an Ethernet port, a DisplayPort, HDMI port, etc.?
If they do, and you pass the entire dock into a VM, then pretty much only that VM will be able to use the dock. I’m guessing that’s not what you want… That’s why it’s important to take a “snapshot” of your system functioning with it and without it.
You can as many VMs with as many PCI devices passed through to them as you like, as long as each PCI device is only passed through to one VM at a time (they don’t like sharing…).
If you’ve got multiple USB buses on your machine, you could definitely create a separate VM for each bus, if you wanted.
I have laptops with an xbox 360 gamepad built into the chassis, on its own USB bus, and I pass that bus into a VM for playing Nintendo 64 ROMs occasionally (I will play GameCube ROMs as soon as integrated graphics performance in Qubes OS gets better), so it’s definitely possible.
(bear in mind that you’re passing through the entire bus, and not individual ports; so be aware of what else is on that bus. You might just lose your internal keyboard/mouse to a VM )
The way you’ve described it to me, this wouldn’t fix your “cold boot” problem. It seems like your machine tells the kernel “I’ve got this many USB ports, this many HDMI ports, this many Ethernet ports, etc.” when the kernel is loaded.
Sure, you could create a separate qube for your dock, but if it wasn’t attached when you cold-booted your machine, I don’t think it would make any difference…
You can do this too. It can cause kernel panic inside VMs sometimes depending on what devices you do it to, and how they’re done.
(If a real machine suddenly got a GPU shoved into one of its PCI slots, it probably wouldn’t react very well…)
Basically, what you need is a solution that, when you plug in this dock, will:
- Tell dom0 that a dock is plugged into one of your thunderbolt ports
- Tell dom0 to treat the individual ports/buses on the dock as separate buses (so you can have the multiple monitors in
sys-gui, the Ethernet port in
sys-net, etc.), and remap your system tree.
- Pass through the appropriate ports/buses to the appropriate VMs (similar to the way
sys-usbwill pass through a USB keyboard/mouse to dom0, but in reverse).
I have a USB-C “dock” that came with my PinePhone that has a similar layout to your dock, but I haven’t been able to achieve what I’m describing yet. If I find a solution, I will post it. (Sorry!)
Thank you for the detailed answer! That will help a great deal. The way I see it, I have two, mostly independent problems.
Let’s set hot-plugging aside for the moment (that’s problem #2 for me).
The first problem is that the dock presents itself as several PCI devices to the computer:
|02:00.0||PCI bridge||Intel Corporation||JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]|
|03:00.0||PCI bridge||Intel Corporation||JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]|
|03:01.0||PCI bridge||Intel Corporation||JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]|
|03:02.0||PCI bridge||Intel Corporation||JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]|
|03:03.0||PCI bridge||Intel Corporation||JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]|
|03:04.0||PCI bridge||Intel Corporation||JHL6540 Thunderbolt 3 Bridge (C step) [Alpine Ridge 4C 2016]|
|04:00.0||USB controller||Fresco Logic||FL1100 USB 3.0 Host Controller|
|05:00.0||USB controller||Fresco Logic||FL1100 USB 3.0 Host Controller|
|06:00.0||USB controller||ASMedia Technology Inc.||ASM1142 USB 3.1 Host Controller|
|07:00.0||Network controller||Intel Corporation||I210 Gigabit Network Connection|
Several of these devices need to be mapped to various qubes to be useful. USB controllers to sys-usb and network controller to sys-net in this case. However, if I map a device to a certain qube, and the device isn’t present, such as when the dock isn’t connected, then that qube cannot boot.
That’s why I asked about multiple sys-usb qubes. If it works as I hope, then sys-usb will always be bootable because the laptop’s built-in USB ports will always be there, and sys-usb-dock will be bootable only if the dock is present.
I still need to figure out what, if anything, can be done for the network controller. I can certainly make a sys-net-dock qube for that controller, but how can I integrate that with the rest of the system? I’m kinda thinking I’ll need some way to add and remove the Ethernet controller to sys-net on bootup.
As for hotplugging, that’s another can of worms that I should probably save for after I get cold-plugging working. Basically, it sounds like I need to set up udev to detect when the dock is plugged or unplugged and then rescan the PCI bus and make whatever changes it needs to make in response:
- Start or stop qubes.
- Restart qubes while slipping in changes to their device configuration (hopefully without needing to restart other qubes, like how stopping sys-net requires that sys-firewall stop, which in turn requires that all network-connected appvms also stop).
- Adjust screen layout.
- Adjust audio outputs.
By the way, if it makes any difference, there’s no HP in my system. I have a Framework laptop with a CalDigit TS3Plus+ dock. I originally hoped to contribute to the original question before seeing for myself how big a dumpster fire Thunderbolt docks are.
That’s not a problem. That’s by design. That’s how Thunderbolt works
Yeah. That makes sense a lot more now.
This is an example of a command to attach PCI devices to VMs in dom0:
sudo xl pci-attach sys-net '03:00.0'
Maybe that would work when linked to a
It might also help you avoid separate qubes entirely
Oh, fun! I just finished testing the separate qube for the dock’s USB ports (still cold-plugging only). It was a partial success. Everything works fine if I start the laptop with the dock disconnected or with the dock connected to the same Thunderbolt port as when I configured it. However, if I plug the dock into a different Thunderbolt port, then the dock’s PCI identifiers all change.
I guess, in my case, I can just make a rule for myself that the dock must be connected to a specific port. That’s probably not true in a general case though.