Using USB blocks in an usb-installed QubesOS

User Support
1

Hi folks,

I’m a noob and all I’ve searched about my problem didn’t help me till here. If ever you have some links about already answered similar problem, please tell me.

I’ve got a problem using usb memory sticks in my usb-installed QubesOS.

I can see my USB stick in the list of devices, can attach them to Personal Qube for example. But it doesn’t show up in Thunar FileManager.

Also, I noticed USB stick seems to be directly “linked” to dom0, and not to sys-usb qube. Listing the usb devices in dom0 doesn’t show my stick, wether it is attached to some qube or not (but still visible in the device list in the desktop upper right bar)
I can find sys-usb nowhere (qubemanager etc…). I tried to add a sys-usb qube with but i’m afraid it ruined my install (i’m currently reinstalling everything from scratch).

I’ve got the feeling it is due to an installation on USB medium, so that I can’t have a sys-usb isolating the USB key from the QubeOS itself.

So, how could I do to have an trustful access to usbkeys without threatening dom0 ? Should I add another usb Qube ? If yes, could you explain me the steps to follow (as I’m a very newbie in linux and CLI… :wink: )

I’m running QubesOs R4.2.2 on a sandisk 64GB stick ; computer are HP 250G7 Notebook 64 ; or Victus Laptop (probably Victus 15 or 16 ; it’s running, if needed I’ll tell you). Laptop’s keyboard never shows up when listing usb devices (lsusb ; but I’m unsure about how to list PS2 devices in CLI and check that keyboard is PS2).

Many thanks,
all the best,

2

Check if you have multiple USB controllers and they are available on the physical USB ports. If you have them then you can dedicate one of the USB controllers to dom0 and connect your USB disk to it and then use other USB controllers to create sys-usb.
Run this command in dom0 to see if there are multiple USB controllers available:

lspci | grep USB

Then check if there are different USB controllers connected to the different USB ports:

You can create sys-usb like this:

But in step 4 use rd.qubes.hide_pci=<BDF1>,<BDF2>,<BDFx> option to hide the USB controllers that you want to attach to sys-usb from dom0 instead of usbcore.authorized_default=0 and rd.qubes.dom0_usb=<BDF>.

And use grub2-mkconfig -o /boot/grub2/grub.cfg command in 6 step in either case.

3

Hi Apparatus,
Many thanks for taking time for answering my request, much appreciated.

I rode the fist link you gave me (but not sure I understood everything). What I noticed was that I didn’t have sys-usb:sda2 but only dom0:sda2. I guessed it was a problem, as every usb stick would be directly linked to dom0.

I runned the two next commands lsusb and lsbdf, here are the outputs

[tanka@dom0 Bureau]$ lspci | grep USB
08:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
08:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne USB 3.1
[tanka@dom0 Bureau]$

[tanka@dom0 Bureau]$ bash lsbdf.sh
BDF: 08:00.4 Bus 004 Device 002: ID 0781:5591 SanDisk Corp. Ultra Flair
BDF: 08:00.4 Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
BDF: 08:00.4 Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
BDF: 08:00.3 Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
BDF: 08:00.3 Bus 001 Device 004: ID 13d3:3567 IMC Networks Wireless_Device
BDF: 08:00.3 Bus 001 Device 003: ID 0408:5482 Quanta Computer, Inc. HP Wide Vision HD Camera
BDF: 08:00.3 Bus 001 Device 002: ID 046d:c019 Logitech, Inc. Optical Tilt Wheel Mouse
BDF: 08:00.3 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
[tanka@dom0 Bureau]$

Can you confirm me everything is ok with the different USB ports/controllers (as i’m not sure to understand well what it’s about ; I’ll dig into these notions too) ?

I will proceed with the next steps as soon as I read you back,

I’m wondering too, if I attach one usb port to dom0, and others remaining hidden from dom0, does that mean that I will have to connect the usb stick always at the same port of my computer (i.e. always on the left-side usb port ; and never to the right one ? as I have 3 usb ports, 1 “standard” on the left where I generally put the usb stick ; 1 “standard” on the right, and a USB-C on the right too) ?
Also, would I be able to use the USB stick on other computers ? My main goal with having Qubes on a USB is to test it, see how I can manage with this distro on a daily use ; but if I could it would be nice for me to be able to use it on other computers (my personnal and my pro one for ex).
Finally, should I do all these steps with a fresh install ? I already plugged a USD disk, when I tried to attach and mount it. But that means it has been in contact with dom0 : is there any risk ? should I re-install it from zero ?

many thanks for help,

All the best,

4

You have two USB controllers: 08:00.3 and 08:00.4.
Your USB disk is connected to 08:00.4 USB controller.
Your USB mouse is connected to 08:00.3 USB controller.
So by the look of it you have physical USB ports available for both USB controllers so you can create sys-usb for your 08:00.3 controller and leave 08:00.4 USB controller in dom0.

Yes, you can only connect the USB disk to the port corresponding to the 08:00.3 USB controller.
You need to check which USB port is connected to which USB controller.
If you run bash lsbdf.sh with your USB disk connected to the “standard” USB port on the left:

  • 1 “standard” on the left where I generally put the usb stick - connected to the 08:00.3 USB controller
  • 1 “standard” on the right (if you’ve connected your mouse to it) - connected to the 08:00.4 USB controller
  • USB-C on the right - unknown, connect some USB-C device to it and run bash lsbdf.sh to see to which USB controller is it connected to

Qubes OS is attaching PCI devices to the qubes (e.g. sys-net/sys-usb) based on their BDF and they may be different on another computers. And the available devices may be different as well.
For example, you have PCI Network Controller with BDF 03:00.0 attached to your sys-net in your computer, but on another computer the device with BDF 03:00.0 could be the GPU and the GPU will be attached to sys-net instead of dom0 so your Qubes OS will boot with black screen or won’t boot at all.
Or there could be only one USB controller on another computer so you won’t be able to use sys-usb there.

You can boot the Qubes OS without qubes autostart like this:

And also remove the rd.qubes.hide_pci from the kernel command line options in GRUB so it won’t hide e.g. GPU from dom0.
Then you can configure the PCI devices attached to your qubes (sys-net, sys-usb) and specify correct USB controller to hide from dom0 using rd.qubes.hide_pci option.
Then you can use this Qubes OS on another PC.
But Qubes OS is not designed to be plug and play to work on any computer you connect it to right away.

5

Hi Apparatus and everybody here,
Many thanks for the instructions, there were very usefull. I partly successed in setting up the sys-usb as intended

I will describe how I did so that probably it could be usefull for others, and to get some answers, as not everything seems to be OK in the way I did, and as I’m unsure of what exactly I did.

steps

\USB Qubes “4.1”
→ with USB qubes | Qubes OS as a reference, but modified this way :

  • steps 1 to 3
  • step 4 : rd.qubes.hide_pic=08:00.3
  • sudo grub2-mkconfig -o /boot/grub2/grub.cfg (it didn’t worked without the sudo)
  • restart

NEW sys-usb Qube
→ “MANUAL CREATION” here USB qubes | Qubes OS

  • Qube Manager, New Qube
  • name “sys-usb”, AppVM, Debian12, networking set to “none” (as I imagined there no need to)
  • clicked on OK, BUT : “Error devices tab : can’t attach PCI device to VM in PVH mode”
  • re-edited this new sys-usb qube and modified the mode from PVGH to HVM
  • devices now appear in sys-usb qubes’ settings, and there I added the 08:00.3 device to it.

Errors appeared about /etc/qubes.rpc/policy/qubes.InputMouse (not mouse working on its port)

  • then I did sudo qubesctl state.sls qvm.sys-usb (it didn’t worked without the sudo)

Then : everything was ok : I got the USB mouse working, and when replacing it by a USB stick, it worked too. Till here no more problem !

… but when I reboot, nothing worked.
I was blocked out at the LUKS decrypting step : unable to enter it, and after a very looooong time, a screen appeared telling me nothing was ok about dom0, LUKS etc (picture on my phone, I can send it if needed)

I looked around for details in topics I already saw “broken my install after creating sys-usb” “locked out of entering LUKS password…” and so on,
and then I checked :

  • Solene told to edit the grub entry at starting with “e”, so did I do
  • I saw a rd.qubes.hide_all_usb somewhere in the output
  • So, I removed it, and pressed F10 to boot with this mod

OK, I COULD BOOT INTO QUBES.

  • Then I edited /etc/default/grub.cfg in dom0 nano,
  • and what was my surprise to see the good parts “rd.qubes.hide_pci=08:00.3” in the GRUB_CMDLINE_LINUX ; but too, I saw another GRUB_CMDLINE_LINUX at the end of the grub stating “$GRUB_CMDLINE_LINUX rd.qubes.hide_all_usb” !

I try to remove this line, and

  • Run the command grub2-mkconfig -o /boot/grub2/grub.cfg in dom0, then reboot to see

Can you tell me where does it come from, when did I do a mistake, if I executed somehow wrong manager/commands…?

(I actually notice that in USB qubes | Qubes OS it is said that as soon as you use the command qubesctl, “USB controllers are automatically hidden from dom0” ?? But then, how to get a mouse working without using this qubesctl command ?)

I think I’m going to install qubes onboard, it will be easier haha

All the best, thanks for the help !

6

The sudo qubesctl state.sls qvm.sys-usb is adding rd.qubes.hide_all_usb to hide all USB controllers from dom0, that’s default configuration.

Not sure which error did you see, but you can allow the mouse in Qubes Global Config → USB Devices tab for manually created sys-usb.

7

hi,

After having experimented a little thanks to the advices here and on the linked topics, I finally got Qubes installed as my daily OS.
I successfully modified the default card reader affection from dom0 to sys-usb. I did it as for the USB devices in my previous install :

  • I edited the file /etc/default/grub in dom0.
  • In the line that begins with GRUB_CMDLINE_LINUX, I added rd.qubes.hide_pci=<BDF> where <BDF> is the PCI controller identifier.
  • I saved and closed the file.
  • I ran the command sudo grub2-mkconfig -o /boot/grub2/grub.cfg (legacy boot) in dom0.
  • in sys-usb qube settings, I addded the good <BDF> pci controller to the sys-usb device list

Everything seems to be ok. After rebooting, no problem appeared : I have my card reader attached to sys-usb by default.

I felt that it was better to link nothing to dom0, so did I do this mod. I didn’t read anything stating it was not a good idea to detach the card reader from dom0 ; but I rode that some people here did the samed as me. If I’m wrong, please tell me.

(I’ve got another question, probably not requesting a new topic : is it possible that my computer runs with EFI, but work with the legacy boo command /boot/grub2/grub.cfg ? I thought I have EFI, but this command is working. Or am I completely confusing ?)

All the best,

8

Qubes OS is now using /boot/grub2/grub.cfg for both legacy and UEFI boot mode:

1 Like