Sys-USB causing immidiate shutdown once it is manually initiated

linux-enthusiast · May 6, 2024, 10:54pm

I was reading this article, How to use block storage device, which was quite informative because I wanted to mount a thumb drive.

Let me start by saying my keyboard and mouse are both USB connection, and have worked flawlessly from the initial install.

Now when I insert the thumb drive. Nothing, literally nothing happens. So I search for explanations and found said article.

So I open dom0 Terminal

$ lsusb
Bus 009 Device 002: ID 26ce:01a2 ASRock LED Controller
Bus 009 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 010 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 008 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 007 Device 002: ID 046b:c52b Logitech, Inc. Unifying Receiver  <-- Mouse
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 005 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 002: ID 413c:2105 Dell Computer Corp. Model L100 Keyboard  <-- Keyboars
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
bus 001 Device 002: ID 05dc:a81d Lexar Media, Inc. LJDTT16G [JumpDrive 16GB]  <-- Trying to mount
Bus 001 Device 003: ID  0e8d:0616 MediaTek Inc. Wireless_Device
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

So the system sees the USB drive. So I proceed:

$ lsblk
< I will spare you all the qube vm pools and loopbacks>
<Point is there was no "/dev/sda" to  mount>

I open my qube manager, and discover the my Sys-USB is not running. I highlight Sys-USB and hit “Start/Resume” in the ribbon bar at the top. Instantly, the system SHUTS DOWN. No prompt, no warning, no “Qubes Status…” in the top right corner. Instant blackout, board and all. The kind of sudden thing that you are left saying, “What just happened!”

So of course I had to do it again, to make sure is was not something else, and sure enough, same thing! Ok Sys-USB clearly has an ISSUE.

I go into System Setting, and that is when I realize my Keyboard and Mouse is working, and they are USB.

So the system sees, and initiates the USB for the keyboard and mouse, but fails completely otherwise. The system see the Lexar JumpDrive but fails to initiate a udev and the dev/sda allocation.

So I need some assistance diagnosing this one. I read a few post, by a few I mean a lot, but none seem to jump out at me similar to this particular problem. Mouse and Keyboard if not lagging, It might be causing my system to boot-loop but not sure. See my only other post. Which was fixed by adding “qubes.skip_autostart” to grub. It is not “malfunctioning” as some others have said, it is straight up not starting and crashes the system.

apparatus · May 7, 2024, 3:42am

By default only input USB devices are initialized in dom0:

You should also add the usbcore.authorized_default=0 option, which prevents the initialization of non-input devices. (Qubes ships with a USBGuard configuration that allows only input devices when usbcore.authorized_default=0 is set.)

USB qubes | Qubes OS

Check the PCI devices connected to sys-usb in its Settings-> Devices tab.
Make sure that only USB controllers are present in the right column.
Maybe some other critical PCI device is selected in the right column like GPU and your system reboots once it’s trying to remove it from dom0 and passthrough it to sys-usb.
Or try to remove all devices from the right column and then add them back one by one until you find the device that is causing the issue.

linux-enthusiast · June 2, 2024, 7:23pm

I am sorry, I had to step away. I was working on this issues, and found another issue, that lead to several more issues. I won’t lie, I was discouraged, and came close to throwing in the towel and just installing another distro. I hadn’t turned on this system since and it has been weeks. Today, I turned is back on and said I am going to get this done. So here I am.

Ok, so from the screen shot attached you can see what I think you asking. @apparatus.

apparatus · June 2, 2024, 7:38pm

Did install Qubes OS on USB disk?
Try my previous suggestion:

Add the USB controllers to sys-usb one by one and try to start sys-usb until you find which USB controller is causing the system shutdown.

linux-enthusiast · June 2, 2024, 11:31pm

@apparatus
As to be expected, you were right on the money.

I removed all of them. Started sys-usb no issue. of course, but set a baseline.

Afterwards, one by one, I added one and removed the previous one. Start/Stopping sys-usb each time.

All was well until I got to: “17:00.4 USB controller…” and got this:
photo_2024-06-02_18-10-59

Obviously I lost access to my keyboard and mouse. So I could not tell if the system crashed as there was no input of any kind.

Did a hard reset, and decided to the last one for good measure.
Proceeded to remove “17:00.4 USB controller…”, and added “18:00.0 USB controller…” and went to started it like I had done 4 times before, and instantly crashed.

!! Found the culprit !!

I added the previous three, leaving “17:00.4 USB controller…” and “18:00.0 USB controller…” out started sys-usb. No crash, and still had access to my keyboard and mouse, so I took the chance to stick my usb key into the top of my case, and I just had a pop-up on the side that said “Lexar…” is available. I attached it to a VM and I can access the contents.

So not to say I am not grateful for all the assistance, or that I have access to what I originally could not and the reason for the post. BUT HOW???

HOW do I diagnose this? What is the reason for this?
HOW do I avoid this in the future?..

Don’t get me wrong, I am not expecting to be a Qubes guru. It took me twenty year of working on Linux to get to the point that I am now, that I install and maintain systems for other people and actually teach them to use and enjoy Linux. With Qubes I feel less than a toddler, struggling to crawl. I can honestly say “I don’t like it!”

Eric · June 3, 2024, 2:50am

Noting you are on AMD, this looks like the issue I reported last July:
https://github.com/QubesOS/qubes-issues/issues/8322 that is still open.
Seems to be caused by the controller for the mouse and keyboard being added to sys-usb. Waiting for something to trickle down from upstream…

linux-enthusiast · June 3, 2024, 4:01am

@Eric
What you seem to be saying it that my two issues, so far, seem to be related.

Eric · June 3, 2024, 6:06am

@linux-enthusiast Yes that it the same problem. I note you had the power off symptom as well - in my case I think it was the hardware reset causing a transient that the PSU picked up and it did a soft shutdown. Seems that was because my case is very well ventilated, without pressure differential in to out, and the PSU is fanless and getting a bit too warm. Adding a fan to the PSU fixed the power offs. YMMV.

I stopped updating my issue above as it seemed to be being ignored. AMD is not that well supported yet. 4.2.1 install gets stuck on the console after Anaconda completes with skip_autostart. Stability is getting better, for first 8 months MTBF was about a week, last run was 90 days till the spurious logout hit, and made a bit of a mess, needing a restart.