Sys-usb crash when printing

[scallyob]

I don’t believe this is a duplicate of my previous post here:

Whonix qubes randomly shutdown, especially gateway

I’ve had qubes randomly shutting down on me lately. So far it’s been just qubes based on whonix templates and usually it is a whonix gateway. Usually the whonix workstation will not run without the gateway booting first, but what has been happening is a service goes offline,the workstation is still running but the gateway has crashed. I had to make a change to my Xen configuration to get my machine to work with recent versions: No Qubes/VMs boot after latest updates - #32 by scallyob It’s poss…

This has a very clear cause, happens when sending print jobs to USB printer. Doesn’t happen every time. But has started happening much more regularly. I have many examples of the below if helpful to post more examples.

from /var/log/xen/console/guest-sys-usb.log

[2024-06-08 19:56:06] [114115.500799] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:07] [114116.511893] usbip-host 2-1: usb_set_interface done: inf 0 alt 0^M 
[2024-06-08 19:56:28] [114137.676786] usbip-host 2-1: usb_set_interface done: inf 0 alt 1^M 
[2024-06-08 19:56:28] [114137.697300] usbip-host 2-1: usb_set_interface done: inf 0 alt 0^M 
[2024-06-08 19:56:28] [114137.717116] usbip-host 2-1: usb_set_interface done: inf 0 alt 1^M 
[2024-06-08 19:56:28] [114137.758366] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:30] [114139.830236] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:31] [114140.886311] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:32] [114141.916918] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:33] [114142.937803] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:34] [114143.967149] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:35] [114144.993924] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:36] [114146.013400] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:37] [114147.037783] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:38] [114148.073627] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:40] [114149.105408] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:41] [114150.129504] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:42] [114151.153467] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:43] [114152.178916] usbip-host 2-1: unlinked by a call to usb_unlink_urb()^M 
[2024-06-08 19:56:44] [114153.203288] general protection fault, probably for non-canonical address 0xf4ec3d094082861: 0000 [#1] PREEMPT SMP NOPTI^M
[2024-06-08 19:56:44] [114153.205198] CPU: 1 PID: 2647 Comm: stub_rx Not tainted 6.6.25-1.qubes.fc37.x86_64 #1^M
[2024-06-08 19:56:44] [114153.207221] Hardware name: Xen HVM domU, BIOS 4.17.4 04/26/2024^M
[2024-06-08 19:56:44] [114153.209229] RIP: 0010:stub_rx_pdu+0x22c/0x270 [usbip_host]^M
[2024-06-08 19:56:44] [114153.211095] Code: c3 01 48 8b 3c e8 e8 03 59 d8 c5 41 89 c1 83 f8 8d 74 2b 49 8b 44 24 20 41 8b 4c 24 30 89 da 48 c7 c6 40 16 87 c0 4d 8b 04 24 <48> 8b 04 e8 48 8b 78 40 48 81 c7 a8 00 00 00 e8 10 20 ca c5 41 39^M
[2024-06-08 19:56:44] [114153.214201] RSP: 0018:ffffbad3007f7e80 EFLAGS: 00010216^M
[2024-06-08 19:56:44] [114153.214893] RAX: 0f4ec3d094082861 RBX: 0000000000000001 RCX: 0000000000000000^M
[2024-06-08 19:56:44] [114153.215927] RDX: 0000000000000001 RSI: ffffffffc0871640 RDI: ffffffff885f2670^M
[2024-06-08 19:56:44] [114153.216969] RBP: 0000000000000000 R08: 0000000000000000 R09: 00000000ffffffd5^M
[2024-06-08 19:56:44] [114153.217993] R10: 0000000000000000 R11: 0000000000000038 R12: ffff97a04c17cf00^M
[2024-06-08 19:56:44] [114153.219018] R13: ffff97a06edb3f00 R14: ffff97a06edb3f94 R15: ffffffffc0868450^M
[2024-06-08 19:56:44] [114153.220364] FS:  0000000000000000(0000) GS:ffff97a0b9700000(0000) knlGS:0000000000000000^M
[2024-06-08 19:56:44] [114153.221390] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M
[2024-06-08 19:56:44] [114153.222255] CR2: 000076adf7589100 CR3: 000000003deb6000 CR4: 00000000000406e0^M
[2024-06-08 19:56:44] [114153.223442] Call Trace:^M
[2024-06-08 19:56:44] [114153.223791]  <TASK>^M
[2024-06-08 19:56:44] [114153.224141]  ? die_addr+0x36/0x90^M
[2024-06-08 19:56:44] [114153.224669]  ? exc_general_protection+0x1c5/0x430^M
[2024-06-08 19:56:44] [114153.225369]  ? asm_exc_general_protection+0x26/0x30^M
[2024-06-08 19:56:44] [114153.226080]  ? __pfx_stub_rx_loop+0x10/0x10 [usbip_host]^M
[2024-06-08 19:56:44] [114153.226785]  ? stub_rx_pdu+0x22c/0x270 [usbip_host]^M
[2024-06-08 19:56:44] [114153.227489]  stub_rx_loop+0x23/0x40 [usbip_host]^M
[2024-06-08 19:56:44] [114153.228190]  kthread+0xe8/0x120^M
[2024-06-08 19:56:44] [114153.228716]  ? __pfx_kthread+0x10/0x10^M
[2024-06-08 19:56:44] [114153.229239]  ret_from_fork+0x34/0x50^M 
[2024-06-08 19:56:44] [114153.229761]  ? __pfx_kthread+0x10/0x10^M 
[2024-06-08 19:56:44] [114153.230284]  ret_from_fork_asm+0x1b/0x30^M 
[2024-06-08 19:56:44] [114153.230809]  </TASK>^M 
[2024-06-08 19:56:44] [114153.231156] Modules linked in: usbip_host usbip_core usblp uas usb_storage nft_ct ipt_REJECT nf_reject_ipv4 xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables nfnetlink joydev crct10dif_pclmul crc32_pclmul crc32c_intel polyval_clmulni polyval_generic ghash_clmulni_intel sha512_ssse3 sha256_ssse3 xhci_pci sha1_ssse3 pcspkr ohci_pci floppy xhci_pci_renesas drm_vram_helper ata_generic drm_ttm_helper xhci_hcd ttm ohci_hcd pata_acpi i2c_piix4 ehci_pci ehci_hcd serio_raw xen_scsiback target_core_mod xen_netback xen_privcmd xen_gntdev xen_gntalloc xen_blkback xen_evtchn fuse loop overlay xen_blkfront^M 
[2024-06-08 19:56:44] [114153.237460] ---[ end trace 0000000000000000 ]---^M 
[2024-06-08 19:56:44] [114153.237817] RIP: 0010:stub_rx_pdu+0x22c/0x270 [usbip_host]^M 
[2024-06-08 19:56:44] [114153.407101] Code: c3 01 48 8b 3c e8 e8 03 59 d8 c5 41 89 c1 83 f8 8d 74 2b 49 8b 44 24 20 41 8b 4c 24 30 89 da 48 c7 c6 40 16 87 c0 4d 8b 04 24 <48> 8b 04 e8 48 8b 78 40 48 81 c7 a8 00 00 00 e8 10 20 ca c5 41 39^M 
[2024-06-08 19:56:44] [114153.408760] RSP: 0018:ffffbad3007f7e80 EFLAGS: 00010216^M 
[2024-06-08 19:56:44] [114153.409118] RAX: 0f4ec3d094082861 RBX: 0000000000000001 RCX: 0000000000000000^M 
[2024-06-08 19:56:44] [114153.410086] RDX: 0000000000000001 RSI: ffffffffc0871640 RDI: ffffffff885f2670^M 
[2024-06-08 19:56:44] [114153.411227] RBP: 0000000000000000 R08: 0000000000000000 R09: 00000000ffffffd5^M 
[2024-06-08 19:56:44] [114153.412163] R10: 0000000000000000 R11: 0000000000000038 R12: ffff97a04c17cf00^M 
[2024-06-08 19:56:44] [114153.413061] R13: ffff97a06edb3f00 R14: ffff97a06edb3f94 R15: ffffffffc0868450^M 
[2024-06-08 19:56:44] [114153.414034] FS:  0000000000000000(0000) GS:ffff97a0b9700000(0000) knlGS:0000000000000000^M 
[2024-06-08 19:56:44] [114153.414937] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M 
[2024-06-08 19:56:44] [114153.415943] CR2: 000076adf7589100 CR3: 000000003deb6000 CR4: 00000000000406e0^M 
[2024-06-08 19:56:44] [114153.416871] Kernel panic - not syncing: Fatal exception^M 
[2024-06-08 19:56:44] [114153.423006] Kernel Offset: 0x5800000 from 0xffffffff80200000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)^M

[apparatus]

Did you try increasing amount of memory assigned to your sys-usb?
You can also try to use latest kernel version from current-testing repository for you qubes:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing kernel-latest-qubes-vm

[scallyob]

@apparatus

Did you try increasing amount of memory assigned to your sys-usb?

I think I only increased the initial memory. Max was at 2GB, so increased to 4GB and will report back. Though I never saw memory use go high, I have seen CPU usage go high in sys-usb before a crash sometimes. (i have 2 VCPUS)

re: Kernel

I noticed I had it set to 6.6.25 for sys-usb, rather than default of 6.6.31. Changed that as well. Will report back.

[scallyob]

4GB Max memory and default kernel of 6.6.31 did not help, still crashing regularly on print

[apparatus]

You can try kernel-latest-qubes-vm from stable dom0 repository, the kernel there is 6.9.2.
Or maybe you can try the kernel installed in VM instead of kernel provided by dom0 if this issue is caused by the kernel provided by dom0:

Managing qube kernels

[scallyob]

6.9.2 kernel still getting regular crashing now (though at first it seemed to not be crashing, today i’m having to restart sys-usb and reconnect everything after most print jobs)