How to figure out which USB port goes to which USB controller?

User Support General
21

When you request for help, please reply to ALL information requests.
Where are your logs and command outputs ?
If you need a metaphor, what you’re telling us is : “my car makes a noise, what’s the exact problem ?”.

If you’re afraid of disclosing HW information (it’s totally acceptable), just change the numbers in outputs, or give the outputs to someone you trust who can do it for you. The digits/numbers in my command output example are totally made up, I ran the command and then changed digits randomly before posting, in a text editor.
After getting answers, you will just have to “map” made up information to the real one.

I don’t think he did miss that part, at least I didn’t. It’s not because 4 controllers are listed as PCI devices that they are not connected somehow, or not handled as one.
For example, look a (desktop) GPU, it has a PCI device listed as VGA controller, and a PCI device listed as an audio controller. So in your mind, is it one or two devices ? Things may have changed, but last I checked you can NOT pass the GPU part WITHOUT the audio part.
This is WHY we need the logs, as it seems you misinterpret BDF (Bus-Device-Function). In Xen, some multi-Functions devices must be PT’ed as a whole.

lspci launched on dom0 will report all PCI devices of the computer, with real (HW) BDF, passtrough-ed to other domains or not, as this is one of dom0’s jobs : handle+share the hardware.

lspci launched on whatever qube will report mostly virtual PCI devices, and PT’ed adapters, all with virtual BDFs. So, AFAIK and as frank told you, a PT’ed device does not get the same PCI BDF than on dom0. I’m not using sys-usb, but I imagine lspci is available in this template ?

As we already told you, it’s not the purpose of lspci. As its name implies, it lists PCI devices. An USB controller is a PCI device, an USB device is not.
lsusb, as its name also implies, lists USB devices (ok, with the bonus of the USB controllers on which they are attached). Devices can be mapped to ports.
It’s not because you can’t link controllers to ports and devices than no one can !

I repeat, you’re telling us : “my car makes a noise, what’s the exact problem ?”.
And what we are replying is : “let us have a look” !
So, post your logs and command outputs !

Edit: minor changes

22

I’m really not knowledgeable about laptops, especially new ones, but on some the integrated keyboard and trackpad/mouse are cabled to a PS/2 connector/controller internally, leaving the USB controller fully available for passthrough. Although I think I’ve read on this very forum that nowadays, it tends to disappear.

Also, there are laptops on which you can plug ExpressCards as hotpluggable PCI devices, but I don’t know if some still have that. I guess USB took precedence.

23

With lsusb, look for the bus number of the device you’re interested in then look for the root hub on that same bus. With lsusb -v look for the iSerial entry of the root hub on that bus. That will give you the BDF number of the USB controller that will match what you see is lspci.

Well, iSerial will have 4 leading zeros, so a root hub with an iSerial of 0000:00:03 will look like 00:03 in lspci.

You may have USB controllers listed in lspci that don’t correspond to any external ports.

24

I have one like that - Thinkpad T16, it was 2022-2023 released. Even if I throw all USB controllers to some random qube - keyboard and touchpad still works.

But even in this case USB controllers are not free, they manage Bluetooth (but not wifi in my case) and a couple less useful internal devices.

So, to find laptop that has spare USB ports that are on separated USB Controller is quite a challenge. And even if you found one, and even if it supports S3 - the laptop probably still will not be able to suspend properly, because Qubes OS is very bad and unstable with sleeping (suspending and resuming) on almost all possible devices, unfortunately.

25

For what it’s worth, I’m talking about a desktop system.

I didn’t post logs and outputs in response to this morning’s (my time) posts here…because I am not anywhere near the system (and I had some trouble getting the info off of sys-usb in any case). I’ll give it another go tonight (roughly six hours from now).

26

IIRC Joanna (rootkowska) was very vocal about that nonsense as it opens a lot of security holes (while being unpractical for Qubes). I hope the laptop vendors will change that, esp. the open hardware ones. But multiply 1$ chips by the number of laptops sold …

I dunno but even if true, that’s really not fair to consider Qubes itself being the reason. I’ve also never managed to get my Debian dom0s (so w/o Qubes extra machinery) to sleep either.
BTW, I don’t know if KVM or VMware are doing any better considering Sx features, but anyways they are not suited for Qubes.

Slightly offtopic explanation

Qubes is based on Xen, which is enterprise-focused, like running up to massive 24/7/365 clouds, not small laptops you start then sleep|hibernate then restart. And you can’t compare big corps manpower (like Amazon or Citrix) with Qubes manpower.
As an example, I read a recent patch reducing CPU init time at boot from … ~120ms to ~10ms ! Yeah, totally useless for us, but really important for them. But I also see patches by Marek so hope is not lost ! ^^
Maybe other Xen usages will help developping Sx features in the future (automotive/embedded) ?
– end off topic –

That’s good, as there is a chance you have at least 2 USB controllers : 1 provided by the CPU, 1 by the chipset. dom0’s lspci will tell us, but to be sure you can check the technical specs in the MoBo manual, or if there’s a block diagram.

Already gave you 2 possible workarounds, and instead of posting here, you should have created another post asking for help Re your secure copy/paste problem :stuck_out_tongue: (I’m just messing with you / joking).

27

OK. dom0 result of lspci. I only kept 00:0d.X and 00:14.x. Note that 14.0 is in the same grouping as RAM memory and Network controller–I think I did see what looked like wifi whilst playing with dmesg yesterday.

00:0d.0 USB controller: Intel Corporation Tiger Lake-LP Thunderbolt 4 USB Controller (rev 01)
00:0d.2 USB controller: Intel Corporation Tiger Lake-LP Thunderbolt 4 NHI #0 (rev 01)
00:0d.3 USB controller: Intel Corporation Tiger Lake-LP Thunderbolt 4 NHI #1 (rev 01)
00:14.0 USB controller: Intel Corporation Tiger Lake-LP USB 3.2 Gen 2x1 xHCI Host Controller (rev 20)
00:14.2 RAM memory: Intel Corporation Tiger Lake-LP Shared SRAM (rev 20)
00:14.3 Network controller: Intel Corporation Wi-Fi 6 AX201 (rev 20)

(OK so that worked and I don’t need to open another thread about copy/paste not working. :stuck_out_tongue: )

As I understand it those should be all of the actual PCI devices on the computer; when I set up sys-usb I do a qvm-pci attach --persistent no-strict-reset=true for every single one of those four devices. d.0, d.2, d.3 and 14.0.

OK, so now on to lsusb -tv. I’ve lost track of whether you wanted dom0 or sys-usb or both, so here they both are. dom0:

/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
    ID 1d6b:0003 Linux Foundation 3.0 root hub
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub

and sys-usb (freshly restarted) [ > to a file, the file qvm-copied, cat-ed on this VM and mouse selected and pasted here. Yeesh, all because I don’t get a right mouse popup on sys-usb. Probably something I didn’t do with the minimal template it’s based on.]

/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
    ID 1d6b:0003 Linux Foundation 3.0 root hub
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
        ID 152d:2561 JMicron Technology Corp. / JMicron USA Technology Corp. CEB-2235S-U3 external RAID box
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/12p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    |__ Port 3: Dev 6, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
        ID 1c4f:0034 SiGma Micro XM102K Optical Wheel Mouse
    |__ Port 5: Dev 3, If 4, Class=Vendor Specific Class, Driver=, 480M
        ID 03f0:0154 HP, Inc 
    |__ Port 5: Dev 3, If 2, Class=Vendor Specific Class, Driver=, 480M
        ID 03f0:0154 HP, Inc 
    |__ Port 5: Dev 3, If 0, Class=Vendor Specific Class, Driver=, 480M
        ID 03f0:0154 HP, Inc 
    |__ Port 5: Dev 3, If 3, Class=Vendor Specific Class, Driver=, 480M
        ID 03f0:0154 HP, Inc 
    |__ Port 5: Dev 3, If 1, Class=Printer, Driver=usblp, 480M
        ID 03f0:0154 HP, Inc 
    |__ Port 7: Dev 4, If 1, Class=Human Interface Device, Driver=usbhid, 12M
        ID 046d:c33b Logitech, Inc. 
    |__ Port 7: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 12M
        ID 046d:c33b Logitech, Inc. 
    |__ Port 10: Dev 5, If 0, Class=Wireless, Driver=btusb, 12M
        ID 8087:0026 Intel Corp. 
    |__ Port 10: Dev 5, If 1, Class=Wireless, Driver=btusb, 12M
        ID 8087:0026 Intel Corp. 
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/6p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    |__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 480M
        ID 0627:0001 Adomax Technology Co., Ltd

And the frank_sullivan lsusb -v 2>/dev/null | grep 'root hub\|iSerial output, for dom0

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
  idProduct          0x0003 3.0 root hub
  iSerial                 1 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
  idProduct          0x0002 2.0 root hub
  iSerial                 1

And on sys-usb:

  iSerial                 3 [dvdrom serial number--I think.  USB to SATA bridge in qui-devices]
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
  idProduct          0x0003 3.0 root hub
  iSerial                 1 0000:00:06.0
  iSerial                 3 [redacted, it's my keyboard serial number]
  iSerial                 3 [printer serial number]
  iSerial                 0 
  iSerial                 0 
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
  idProduct          0x0002 2.0 root hub
  iSerial                 1 0000:00:06.0
  iSerial                10 42
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
  idProduct          0x0002 2.0 root hub
  iSerial                 1 0000:00:05.0

(As you can see I am determined to avoid having to open another thread for copy./paste issues :stuck_out_tongue: )

My mouse has a serial number, but it doesn’t appear. Possibly one of the two zero serial numbers for Bus 003 Device 001: ID 1d6b:0003
The hardware box has two USB 3.0 “A” plugs on the front, and the DVD rom is plugged into that. This is encouraging as it implies based on this last capture that the front two plugs might be at least somewhat different than the four that are on the back. On the back there are four plugs. The two USB “A” plugs have the keyboard and mouse, the printer is on one of the other two plugs which are USB-C. There are also two HDMI ports (probably irrelevant but just in case…) and one ethernet plug.

28

This really requires manual testing, but you can automate it to some
extent.
Create 4 new USB qubes, each one allocated a different controller.
Create a script in dom0:

qvm-shutdown sys-usb
sleep 10
qvm-start USB_A
sleep 10
qvm-shutdown USB_A
sleep 10
qvm-start USB_B
...
...
qvm-start  sys-usb

Insert a USB device to the port(s) you are investigating.
Run the script.
(You may need to configure strict reset.)
Watch the popup windows identifying the associated devices.
At end, you return to current state.

If you insert a different type of USB device to each port, you can map
individual ports…

It’s quite possible that only one controller governs all your USB
ports: that’s hardware design.

3 Likes
29

Presently with my sys-usb no-strict-reset is set to true.

I don’t want to be confused by multiple negatives. Are you suggesting that I might need to set no-strict-reset=false (i.e., turning strict reset on) for these test sys-usb-A-D qubes? Or that I should?

[Side note of possible interest to people who speak English as a second, third, fourth, or sixteenth language: English is peculiar in having double negatives cancel each other, at least in the educated version of the language, in most languages they emphasize each other, and that’s also true in most casual dialects of English, generations of teachers’ efforts to beat double negatives out of their pupils notwithstanding. You’re not supposed to say things like “ain’t got no computer” but people do when they’re not being formal.] [And “ain’t” is something else they try to beat out of people, it basically means “isn’t”, “hasn’t” or just about any negation of any form of “to be”.]

30

That was a useful report, but a few things are missing though :
(now that your copy/paste -almost- works :stuck_out_tongue:).

  • the dmesg output (dom0 and sys-usb) : dmesg | grep -i usb | grep -i /devices/pci
  • the lspci from sys-usb, to check if the 2 detected controllers are “real” ones. You have one on 00:05.0 and one on 00:06.0

Strange thing, the iSerial stuff in dom0 is empty. Did you remove/redact stuff or is that expected on Qubes ?

I may be wrong, but it seems you have 2 USB controllers : one on 0d and one on 14, and as you guessed, they may be both PT to sys-usb, BUT teh output raises more questions ^^

  • you need to know if you can PT the controller without the RAM and wifi stuff. Are you using Wifi ?
  • if you don’t care sharing this info, can you post the brand of your MoBo, or better, a link to its manual ? I’m asking that info because depending on your MoBo, the plugs on the front can be wired to different controllers. Same for the back plugs.

As an example, on my MoBo, the “back top” plugs are on one controller, the “back bottom” ones are on the other, and the front plugs are wired on the same controller than the “back top” ones. I have a “free/unused” connector on the MoBo backplate, which is wired to the same controller than the “back bottom” ones. If I want to link the front plugs to the other controller, I move the backplate plug to the free one.
I hope that makes sense ^^
PS: I checked a random Tiger Lake MoBo (MSI with Z690 chipset), and there are two controllers, one from the CPU, one additional.

31

Then you should be able to shutdown sys-usb and start another qube using
an allocated controller without a problem.
Make sure you have no-strict-reset set to True on each of the USB_X
qubes.
If you do not, then you may not be able to start sys-usb at the end of
the script, and that is the whole point.
You may also need to sleep for longer, depending on your system.

digression on English

On your side note, “ain’t” here means “have not” - so here “I have
not got no computer”.
There, the double negative strictly cancels, but any English speaker
would understand the informal, “I do not have a computer”.
Only a pedant would claim not to understand, (or a confused non English
speaker).

Double negatives are not common in standard English, but very common in
some dialects - sometimes triple negatives.
There is one form which is acceptable in standard English - double
negation with adjectives - there the double negative shows a hint of
caution in the affirmative. This is not unusual in formal usage.

English - more complicated than Qubes.
Or perhaps: Qubes - less complicated than English.

1 Like
32

This is my machine (I filed the HCL on it):

All it came with was a folded, double-sided sheet of paper, which I’ve long since mislaid.

I am again not near that machine, and thus must wait to do the dmesg and lspci you requested tonight. However, if I recall the lspci command is unknown to sys-usb. That could mean one of two things: 1) You’re asking for something, having forgotten it’s not part of the standard templates or 2) (and much more likely) since I’m using a minimal template, it’s one of the many things not installed on it and I’ve got to install it. (Certainly it makes sense to have that particular command on sys-usb fer cryin’ out loud, so I’ll likely make that a permanent addition to my salt file, etc.)

33

Oh my it’s a NUC, so not a real desktop ! NUCs are disguised laptops ^^
Found this on ark : Intel NUC 11 Pro Kit NUC11TNHi7 Product Specifications, will check later.

For lspci missing I dunno, I’ll have to check, but I’m using an old version of Qubes (4.1 something).
Even though lspci is really useful for some debugs (like PT stuff), it’s completely useless for “normal/user” usage.
For example, you can get a lot of information about the PCI bus from the kernel sysfs on /sys. Note that IIRC the translation between PCI ids and manufacturers (for friendly names) needs a DB, the pciids package or something like that.

34

Front: 2x USB 3.2
Rear: 2x USB 4 (type C), 1x USB 3.2, 1x USB 2.0
Internal: 1x USB 3.2 on m.2 22x42 (pins), 2x USB 2.0 (headers)

I found this on the spec page…it may help.

I came into Qubes with some Linux knowledge, but more as a user (even though I develop GUI apps with GTK). I’ve learned a lot more about the software end of things but my idiocy/ignorance level increases the closer one gets to the hardware, still. (Some might claim it’s not much above total ignorance even on the software end. :stuck_out_tongue: )

35

sudo dmesg | grep -i usb | grep -i /devices/pci on dom0 produced no output whatsoever.

On sys-usb:

[    3.244087] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:05.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input6
[    3.253769] input: SIGMACHIP Usb Mouse as /devices/pci0000:00/0000:00:06.0/usb2/2-3/2-3:1.0/0003:1C4F:0034.0002/input/input7
[    4.217788] input: Logitech K840 Mechanical Corded Keyboard as /devices/pci0000:00/0000:00:06.0/usb2/2-7/2-7:1.0/0003:046D:C33B.0003/input/input8
[    4.274436] input: Logitech K840 Mechanical Corded Keyboard as /devices/pci0000:00/0000:00:06.0/usb2/2-7/2-7:1.1/0003:046D:C33B.0004/input/input9

Lspci not only won’t work on sys-usb, when I try to install it, it says there’s no such package. (Debian-11-minimal qube, btw.)

36

I’ve tried other Technical sheets, and it’s a mess ^^
On the block diagram in “https://www.intel.com/content/dam/support/us/en/documents/intel-nuc/NUC11TN_TechProdSpec.pdf”, there’s not much info.
The “https://cdrdv2-public.intel.com/631119/631119-007.pdf”, section “32.4.2” mentions stuff about “dual-role xHCI/xDCI”.
On “https://cdrdv2-public.intel.com/631121/631121_011.pdf”, section “6 USB-C* Sub System”, there’s some info too, but the “USB-C* Sub-system Block Diagram” does not tell us much.
Then, there’s “https://cdrdv2-public.intel.com/631122/631122_003.pdf”, section “11 Type C Subsystem (TCSS)”, the 2nd § shows DMA for only the 0d.2 and 0d.3.

And … I dunno what to think of it ^^ Also I’m a Thunderbolt noob, and couldn’t find a section about Thunderbolt specifically. Thunderbolt on your host provides USB (0d.0), PCIe lane(s) and DisplayPort (0d.2 and 0d.3, didn’t check which is which). Maybe someone with deep HW knowledge can make sense of it.

Anyways, datasheets being useless to me, let’s get back on what we know ^^
In sys-usb, you only have one “real” (ie PT) controller on 0:06, and this is the 14.0 in dom0.
I guess you have nothing plugged in the USB-C connector, which is linked to the Thunderbolt USB (00:0d.0) ?
Does the qubes manager allow you to PT this device to a qube ?

Now, to the original question, linking USB ports to controllers.

"lsusb -vt" shows (removed useless info) :

/:  Bus 03
    ID 1d6b:0003 Linux Foundation 3.0 root hub
    |__ Port 1: Dev 2
        ID 152d:2561 JMicron Technology Corp (...)

Now on sys-usb do the command below, you will get something like :

$ find /sys -iname "*152d:2561*"
/sys/kernel/debug/hid/0014:152d:2561.0002
/sys/devices/pci0000:00/0000:00:0x.x/0000:14:00.0/usb1/3-1/3-1:1.0/0014:152d:2561.0002
/sys/bus/hid/devices/0014:152d:2561.0002
/sys/bus/hid/drivers/hid-generic/0014:152d:2561.0002
# I adapted my output to your system, but only for some stuff, you'll get a slightly different output

For other devices, just change the device ID in the command.
I let you find the correlations ! :stuck_out_tongue:

Now that you’re totally bored by this maze, let’s use the easy way :

  • do “ls -l /sys/bus/usb/devices” in sys-usb. The -l is important, as you need the timestamps
  • unplug a device (w/e), wait a few seconds, then re-plug it, wait a few seconds.
  • arrow-up, Enter (ie. redo the ls).
  • compare timestamps
  • do that for every USB plug you have

Banzai ^^

37

PS: edited stuff, mostly because there was a missing quote in the find cmd. Sorry for mail users …
So, the find command should look like :
find /sys -iname "*152d:2561*"
The other edits are not important.

38

Here you go :

$ apt-file search "bin/lspci"
pciutils: /usr/bin/lspci                  
$ apt show pciutils
Package: pciutils
[...]
Depends: libc6 (>= 2.14), libkmod2 (>= 5~), libpci3 (= 1:3.7.0-5)
[...]
APT-Manual-Installed: yes
[...]
Description: PCI utilities
 This package contains various utilities for inspecting and setting of
 devices connected to the PCI bus.
39

Create 4 sys-usb’s, assign them only one controller per qube via GUI (from Qube manager’s right click on a sys-usb’s Settings-Devices, for example) and name them accordingly:
sys-usb-d.0
sys-usb-d.2
sys-usb-d.3
sys-usb-14

Set them all to autostart. Disable autostart for the current, original sys-usb. Include all sys-usb names to etc/qubes-rpc/policy/qubes.InputKeyboard and qubes.InputMouse.
Restart qubesd ($ sudo systemctl restart qubesd), just in case.

Reboot.

Check device widget to which sys-usb of these keyboard and mouse are attached. Remove unnecessary sys-usbs from policies and restart qubesd. You may reboot again, but absolutely unnecessary.

Get an USB flash, put it in each port and keep checking device widget, putting USB stick from port to port.

I can’t believe this topic had 39 posts only to discover this triviality.

40

You’d have read them, you’d have seen your solution was already given by unman. And his is more failsafe. But it’s a bit cumbersome.

Plus when testing stuff like that, you should NEVER use autostart, moreover for all of the qubes tested, that’s very bad for debugging, and a recipe for failure.
What if you make a mistake and block your system ? You’d better know “qubes.skip_autostart” beforehand (Autostart troubleshooting | Qubes OS).
Better go step-by-step !