Environment:
Up to date R4.2, kernel 6.6.63, fedora-40 for most templates.
Hardware is Dell Latitude 7490, docking station is a Thunderbolt 4 Lenovo 40B0 series. sys-usb is disposable, has only laptop’s USB controller passed.
I cloned sys-usb to sys-usb-dock, set it to not be started on boot, and removed laptop’s USB controller, and added dock’s USB controller.
Issue:
I start the laptop with docking station plugged in. I need to start the sys-usb-dock manually (without it the keyboard and mouse seem to be available in dom0 directly), and need to attach the ethernet from dock to sys-net since it’s presented as a USB device. All of that works, and so do the USB devices connected to the dock, and the two monitors.
Problems start without the dock. Even with ethernet unshared from sys-net, and with sys-dock-usb shut down, disconnecting the dock results in system completely freezing after seemingly random amount of time. What’s more confusing, when booting up without the dock connected, system also freezes after a random amount of time - sometimes before the login screen comes up, sometimes a while after logging in and starting some VMs.
For the first part I would start asking about recompiling the kernel with enabled hotplug, but the second part completely baffles me. Any ideas, suggestions, ways to debug this?
This doesn’t sound like a kernel issue directly. It sounds more like Xen not passing things through correctly…
This sort of system freeze occurs when Xen tries to do something with PCI devices, and they don’t “play nice” (don’t get reset properly, don’t like being reset, get only partially passed through, etc.)
What is the output of lspci in dom0 when you have the dock connected, and when it’s not connected?
Do the outputs of dmesg say anything interesting when your system freezes?
I assume you mean from a system booted without the dock connected? I’ll see if I’m able to get that.
My understanding is that dmesg is available only during the “current” boot, thus when system freezes and I need to powercycle it, I won’t be able to read it. I’m likely to be wrong, but I don’t know how to access such previous dmesg, any suggestions?
Oh, I see, journalctl -o short-iso-precise -k -b <how many boots ago>.
From a boot that hanged during startup (I think I have the right one), I don’t see anything too interesting. Tail of it looks like this, do you want the whole 1k lines of it?
2024-12-19T15:57:26.381433+0100 dom0 kernel: [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
2024-12-19T15:57:26.381509+0100 dom0 kernel: ACPI: video: Video Device [GFX0] (multi-head: yes rom: no post: no)
2024-12-19T15:57:26.381578+0100 dom0 kernel: input: DELL081C:00 044E:121F Mouse as /devices/pci0000:00/0000:00:15.1/i2c_designware.1/i2c-1/i2c-DELL081C:00/0018:044E:121F.0001/input/input6
2024-12-19T15:57:26.381646+0100 dom0 kernel: input: DELL081C:00 044E:121F Touchpad as /devices/pci0000:00/0000:00:15.1/i2c_designware.1/i2c-1/i2c-DELL081C:00/0018:044E:121F.0001/input/input7
2024-12-19T15:57:26.381714+0100 dom0 kernel: input: DELL081C:00 044E:121F as /devices/pci0000:00/0000:00:15.1/i2c_designware.1/i2c-1/i2c-DELL081C:00/0018:044E:121F.0001/input/input8
2024-12-19T15:57:26.381813+0100 dom0 kernel: hid-generic 0018:044E:121F.0001: input,hidraw0: I2C HID v1.00 Mouse [DELL081C:00 044E:121F] on i2c-DELL081C:00
2024-12-19T15:57:26.532202+0100 dom0 kernel: input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00/input/input5
2024-12-19T15:57:26.532225+0100 dom0 kernel: fbcon: i915drmfb (fb0) is primary device
2024-12-19T15:57:26.532243+0100 dom0 kernel: fbcon: Deferring console take-over
2024-12-19T15:57:26.532260+0100 dom0 kernel: i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
2024-12-19T15:57:26.532519+0100 dom0 kernel: usb 1-7: new full-speed USB device number 3 using xhci_hcd
2024-12-19T15:57:26.532561+0100 dom0 kernel: input: DELL081C:00 044E:121F Mouse as /devices/pci0000:00/0000:00:15.1/i2c_designware.1/i2c-1/i2c-DELL081C:00/0018:044E:121F.0001/input/input9
2024-12-19T15:57:26.532579+0100 dom0 kernel: input: DELL081C:00 044E:121F Touchpad as /devices/pci0000:00/0000:00:15.1/i2c_designware.1/i2c-1/i2c-DELL081C:00/0018:044E:121F.0001/input/input10
2024-12-19T15:57:26.532607+0100 dom0 kernel: input: DELL081C:00 044E:121F UNKNOWN as /devices/pci0000:00/0000:00:15.1/i2c_designware.1/i2c-1/i2c-DELL081C:00/0018:044E:121F.0001/input/input11
2024-12-19T15:57:26.532625+0100 dom0 kernel: hid-multitouch 0018:044E:121F.0001: input,hidraw0: I2C HID v1.00 Mouse [DELL081C:00 044E:121F] on i2c-DELL081C:00
2024-12-19T15:57:26.559838+0100 dom0 kernel: usb 1-7: New USB device found, idVendor=8087, idProduct=0a2b, bcdDevice= 0.10
2024-12-19T15:57:26.560165+0100 dom0 kernel: usb 1-7: New USB device strings: Mfr=0, Product=0, SerialNumber=0
2024-12-19T15:57:26.560415+0100 dom0 kernel: usb 1-7: Device is not authorized for usage
2024-12-19T15:57:26.673835+0100 dom0 kernel: usb 1-9: new full-speed USB device number 4 using xhci_hcd
2024-12-19T15:57:26.801837+0100 dom0 kernel: usb 1-9: New USB device found, idVendor=1050, idProduct=0407, bcdDevice= 4.27
2024-12-19T15:57:26.802130+0100 dom0 kernel: usb 1-9: New USB device strings: Mfr=1, Product=2, SerialNumber=0
2024-12-19T15:57:26.802382+0100 dom0 kernel: usb 1-9: Product: Yubikey 4 OTP+U2F+CCID
2024-12-19T15:57:26.802627+0100 dom0 kernel: usb 1-9: Manufacturer: Yubico
2024-12-19T15:57:26.802887+0100 dom0 kernel: usb 1-9: Device is not authorized for usage
2024-12-19T15:57:27.567881+0100 dom0 kernel: pcieport 0000:04:00.0: Unable to change power state from D3hot to D0, device inaccessible
2024-12-19T15:57:37.295029+0100 dom0 kernel: fbcon: Taking over console
2024-12-19T15:57:37.317849+0100 dom0 kernel: Console: switching to colour frame buffer device 240x67
2024-12-19T15:57:50.303855+0100 dom0 kernel: kauditd_printk_skb: 43 callbacks suppressed
2024-12-19T15:57:50.303878+0100 dom0 kernel: audit: type=1338 audit(1734620270.302:54): module=crypt op=ctr ppid=1 pid=540 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="systemd-cryptse" exe="/usr/lib/systemd/systemd-cryptsetup" dev=253:0 error_msg='success' res=1
2024-12-19T15:57:50.303897+0100 dom0 kernel: audit: type=1300 audit(1734620270.302:54): arch=c000003e syscall=16 success=yes exit=0 a0=4 a1=c138fd09 a2=5a5ecaacda50 a3=0 items=6 ppid=1 pid=540 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="systemd-cryptse" exe="/usr/lib/systemd/systemd-cryptsetup" key=(null)
2024-12-19T15:57:50.303915+0100 dom0 kernel: audit: type=1307 audit(1734620270.302:54): cwd="/"
2024-12-19T15:57:50.303932+0100 dom0 kernel: audit: type=1302 audit(1734620270.302:54): item=0 name=(null) inode=2049 dev=00:07 mode=040755 ouid=0 ogid=0 rdev=00:00 nametype=PARENT cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
2024-12-19T15:57:50.303950+0100 dom0 kernel: audit: type=1302 audit(1734620270.302:54): item=1 name=(null) inode=4862 dev=00:07 mode=040755 ouid=0 ogid=0 rdev=00:00 nametype=CREATE cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
2024-12-19T15:57:50.303967+0100 dom0 kernel: audit: type=1302 audit(1734620270.302:54): item=2 name=(null) inode=23 dev=00:07 mode=040755 ouid=0 ogid=0 rdev=00:00 nametype=PARENT cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
2024-12-19T15:57:50.303990+0100 dom0 kernel: audit: type=1302 audit(1734620270.302:54): item=3 name=(null) inode=4863 dev=00:07 mode=040755 ouid=0 ogid=0 rdev=00:00 nametype=CREATE cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
2024-12-19T15:57:50.304007+0100 dom0 kernel: audit: type=1302 audit(1734620270.302:54): item=4 name=(null) inode=4863 dev=00:07 mode=040755 ouid=0 ogid=0 rdev=00:00 nametype=PARENT cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
2024-12-19T15:57:50.304025+0100 dom0 kernel: audit: type=1302 audit(1734620270.302:54): item=5 name=(null) inode=4864 dev=00:07 mode=0100444 ouid=0 ogid=0 rdev=00:00 nametype=CREATE cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0
2024-12-19T15:57:50.304042+0100 dom0 kernel: audit: type=1327 audit(1734620270.302:54): proctitle=2F7573722F6C69622F73797374656D642F73797374656D642D6372797074736574757000617474616368006C756B732D66363066316338332D663033652D346439302D623763642D363133323962303732383737002F6465762F6469736B2F62792D757569642F66363066316338332D663033652D346439302D623763642D36
2024-12-19T15:57:51.347843+0100 dom0 kernel: EXT4-fs (dm-4): mounted filesystem af7b177f-f71b-4288-a125-b0d67f86cd54 ro with ordered data mode. Quota mode: none.
2024-12-19T14:57:52.672020+0100 dom0 systemd-journald[248]: Received SIGTERM from PID 1 (systemd).
2024-12-19T14:57:52.672060+0100 dom0 systemd[1]: RTC configured in localtime, applying delta of 60 minutes to system time.
2024-12-19T14:57:52.672079+0100 dom0 systemd[1]: systemd 251.19-1.fc37 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
2024-12-19T14:57:52.672094+0100 dom0 systemd[1]: Detected virtualization xen.
2024-12-19T14:57:52.672110+0100 dom0 systemd[1]: Detected architecture x86-64.
2024-12-19T14:57:52.672127+0100 dom0 systemd[1]: bpf-lsm: BPF LSM hook not enabled in the kernel, BPF LSM not supported
2024-12-19T14:57:52.672148+0100 dom0 systemd[1]: /usr/lib/systemd/system/qubesd.service:8: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
2024-12-19T14:57:52.672164+0100 dom0 systemd[1]: /usr/lib/systemd/system/qubes-qmemman.service:7: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
2024-12-19T14:57:52.672179+0100 dom0 systemd[1]: /usr/lib/systemd/system/qubes-db-dom0.service:9: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
2024-12-19T14:57:52.672194+0100 dom0 systemd[1]: /usr/lib/systemd/system/qubes-core.service:8: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
2024-12-19T14:57:52.672209+0100 dom0 systemd[1]: systemd-ask-password-plymouth.service: Deactivated successfully.
2024-12-19T14:57:52.672223+0100 dom0 systemd[1]: initrd-switch-root.service: Deactivated successfully.
2024-12-19T14:57:52.672238+0100 dom0 systemd[1]: Stopped initrd-switch-root.service - Switch Root.
2024-12-19T14:57:52.672258+0100 dom0 systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.
2024-12-19T14:57:52.672274+0100 dom0 systemd[1]: Created slice system-getty.slice - Slice /system/getty.
2024-12-19T14:57:52.672289+0100 dom0 systemd[1]: Created slice system-qubes\x2dvm.slice - Slice /system/qubes-vm.
2024-12-19T14:57:52.672304+0100 dom0 systemd[1]: Created slice system-serial\x2dgetty.slice - Slice /system/serial-getty.
2024-12-19T14:57:52.672319+0100 dom0 systemd[1]: Created slice system-systemd\x2dfsck.slice - Slice /system/systemd-fsck.
2024-12-19T14:57:52.672339+0100 dom0 systemd[1]: Created slice user.slice - User and Session Slice.
2024-12-19T14:57:52.672354+0100 dom0 systemd[1]: systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch was skipped because of a failed condition check (ConditionPathExists=!/run/plymouth/pid).
2024-12-19T14:57:52.672369+0100 dom0 systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
2024-12-19T14:57:52.672385+0100 dom0 systemd[1]: Reached target blockdev@dev-mapper-luks\x2df60f1c83\x2df03e\x2d4d90\x2db7cd\x2d61329b072877.target - Block Device Preparation for /dev/mapper/luks-f60f1c83-f03e-4d90-b7cd-61329b072877.
2024-12-19T14:57:52.672399+0100 dom0 systemd[1]: Stopped target initrd-switch-root.target - Switch Root.
2024-12-19T14:57:52.672414+0100 dom0 systemd[1]: Stopped target initrd-fs.target - Initrd File Systems.
2024-12-19T14:57:52.672429+0100 dom0 systemd[1]: Stopped target initrd-root-fs.target - Initrd Root File System.
2024-12-19T14:57:52.672449+0100 dom0 systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
2024-12-19T14:57:52.672463+0100 dom0 systemd[1]: Reached target remote-cryptsetup.target - Remote Encrypted Volumes.
2024-12-19T14:57:52.672478+0100 dom0 systemd[1]: Reached target remote-fs.target - Remote File Systems.
2024-12-19T14:57:52.672493+0100 dom0 systemd[1]: Reached target slices.target - Slice Units.
2024-12-19T14:57:52.672507+0100 dom0 systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
2024-12-19T14:57:52.672522+0100 dom0 systemd[1]: Listening on dm-event.socket - Device-mapper event daemon FIFOs.
2024-12-19T14:57:52.672537+0100 dom0 systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
2024-12-19T14:57:52.672551+0100 dom0 systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
2024-12-19T14:57:52.672566+0100 dom0 systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
2024-12-19T14:57:52.672586+0100 dom0 systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
2024-12-19T14:57:52.672601+0100 dom0 systemd[1]: Listening on systemd-userdbd.socket - User Database Manager Socket.
2024-12-19T14:57:52.672616+0100 dom0 systemd[1]: Activating swap dev-mapper-qubes_dom0\x2dswap.swap - /dev/mapper/qubes_dom0-swap...
2024-12-19T14:57:52.672631+0100 dom0 systemd[1]: dev-hugepages.mount - Huge Pages File System was skipped because of a failed condition check (ConditionPathExists=/sys/kernel/mm/hugepages).
2024-12-19T14:57:52.672646+0100 dom0 systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
2024-12-19T14:57:52.672661+0100 dom0 systemd[1]: Mounting proc-xen.mount - Mount /proc/xen files...
2024-12-19T14:57:52.672681+0100 dom0 kernel: Adding 4116476k swap on /dev/mapper/qubes_dom0-swap. Priority:-2 extents:1 across:4116476k SS
2024-12-19T14:57:52.672697+0100 dom0 systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
2024-12-19T14:57:52.672717+0100 dom0 systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
2024-12-19T14:57:52.672733+0100 dom0 systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
2024-12-19T14:57:52.672748+0100 dom0 systemd[1]: Starting lvm2-monitor.service - Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
2024-12-19T14:57:52.672763+0100 dom0 systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
2024-12-19T14:57:52.672778+0100 dom0 systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
2024-12-19T14:57:52.672798+0100 dom0 systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
2024-12-19T14:57:52.672813+0100 dom0 systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
2024-12-19T14:57:52.672849+0100 dom0 systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
2024-12-19T14:57:52.672863+0100 dom0 systemd[1]: plymouth-switch-root.service: Deactivated successfully.
2024-12-19T14:57:52.672891+0100 dom0 systemd[1]: Stopped plymouth-switch-root.service - Plymouth switch root service.
2024-12-19T14:57:52.672907+0100 dom0 systemd[1]: Stopped systemd-journald.service - Journal Service.
2024-12-19T14:57:52.672923+0100 dom0 systemd[1]: Starting systemd-journald.service - Journal Service...
2024-12-19T14:57:52.672938+0100 dom0 kernel: fuse: init (API version 7.39)
2024-12-19T14:57:52.672952+0100 dom0 kernel: loop: module loaded
2024-12-19T14:57:52.672973+0100 dom0 systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules...
2024-12-19T14:57:52.672988+0100 dom0 systemd[1]: Starting systemd-network-generator.service - Generate network units from Kernel command line...
2024-12-19T14:57:52.673003+0100 dom0 systemd[1]: Starting systemd-remount-fs.service - Remount Root and Kernel File Systems...
2024-12-19T14:57:52.673019+0100 dom0 systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
2024-12-19T14:57:52.673034+0100 dom0 systemd[1]: Activated swap dev-mapper-qubes_dom0\x2dswap.swap - /dev/mapper/qubes_dom0-swap.
2024-12-19T14:57:52.673049+0100 dom0 systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
2024-12-19T14:57:52.673109+0100 dom0 systemd-journald[1103]: File /run/log/journal/85747d862aa746e19b8f3ded65dc4b6f/system.journal corrupted or uncleanly shut down, renaming and replacing.
2024-12-19T14:57:52.673140+0100 dom0 systemd[1]: Mounted proc-xen.mount - Mount /proc/xen files.
2024-12-19T14:57:52.673155+0100 dom0 systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
2024-12-19T14:57:52.673171+0100 dom0 systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
2024-12-19T14:57:52.673186+0100 dom0 systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
2024-12-19T14:57:52.673202+0100 dom0 systemd[1]: modprobe@configfs.service: Deactivated successfully.
2024-12-19T14:57:52.673218+0100 dom0 systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
2024-12-19T14:57:52.673238+0100 dom0 systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
2024-12-19T14:57:52.673253+0100 dom0 systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
2024-12-19T14:57:52.673269+0100 dom0 systemd[1]: modprobe@drm.service: Deactivated successfully.
2024-12-19T14:57:52.673285+0100 dom0 systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
2024-12-19T14:57:52.673300+0100 dom0 systemd[1]: modprobe@fuse.service: Deactivated successfully.
2024-12-19T14:57:52.673315+0100 dom0 systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
2024-12-19T14:57:52.673331+0100 dom0 systemd[1]: modprobe@loop.service: Deactivated successfully.
2024-12-19T14:57:52.673346+0100 dom0 systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
2024-12-19T14:57:52.673826+0100 dom0 systemd[1]: Started systemd-journald.service - Journal Service.
2024-12-19T14:57:52.681909+0100 dom0 kernel: EXT4-fs (dm-4): re-mounted af7b177f-f71b-4288-a125-b0d67f86cd54 r/w. Quota mode: none.