Slow VM boot time

User Support General
1

After posting, previous post, I reinstalled firmware of the router, firmware of NovaCustom NV41 12th Gen laptop and reinstalled Qubes OS. There is no change in the behavior. So I thought I’ll post again and see if anyone has inputs.

Basically, the Qubes OS gets very slow when I connect to my router directly using wired connection. There is no issue when I connect to the router using a switch.

Below are the some of the relevant logs showing startup times of an offline tmp-usb qube.

Scenario 1 - Wired connection to the router - through switch.

dom0 journalctl logs 
Jul 02 20:30:45 dom0 qubesd[2225]: vm.tmp-qube: Starting tmp-qube
Jul 02 20:30:45 dom0 lvm[1636]: No longer monitoring thin pool qubes_dom0-vm--pool-tpool.
Jul 02 20:30:45 dom0 lvm[1636]: Monitoring thin pool qubes_dom0-vm--pool-tpool.
Jul 02 20:30:46 dom0 lvm[1636]: No longer monitoring thin pool qubes_dom0-vm--pool-tpool.
Jul 02 20:30:46 dom0 lvm[1636]: Monitoring thin pool qubes_dom0-vm--pool-tpool.
Jul 02 20:30:47 dom0 kernel: loop3: detected capacity change from 0 to 1399448
Jul 02 20:30:47 dom0 qubesd[2225]: vm.tmp-qube: Setting Qubes DB info for the VM
Jul 02 20:30:47 dom0 qubesd[2225]: vm.tmp-qube: Starting Qubes DB

Jul 02 20:30:47 dom0 qubesd[2225]: vm.tmp-qube: Activating the tmp-qube VM

/var/log/xen/console/guest-tmp-qube.log
xen-console-tmp-qube-switch.log (58.6 KB)

tmp-qube journalctl logs 
Jul 02 20:30:49 localhost kernel: Linux version 6.12.11-1.qubes.fc37.x86_64 (mockbuild@0d48d2e97ab5428ca4981acbfa2ff9e2) (gcc (GCC) 12.3.1 20230508 (Red Hat 12.3.1-1), GNU ld version 2.38-27.fc37) #1 SMP PREEMPT_DYNAMIC Tue Jan 28 01:03:48 GMT 2025
Jul 02 20:30:49 localhost kernel: Command line: root=/dev/mapper/dmroot ro nomodeset console=hvc0 rd_NO_PLYMOUTH rd.plymouth.enable=0 plymouth.enable=0 clocksource=tsc xen_scrub_pages=0 swiotlb=2048 apparmor=1 security=apparmor

Jul 02 20:30:50 tmp-qube systemd[1]: Startup finished in 1.082s (kernel) + 1.812s (userspace) = 2.895s.

Scenario 2 - Wired connection to the router - direct.

dom0 journalctl logs 
Jul 02 12:17:42 dom0 qubesd[2129]: vm.tmp-qube: Starting tmp-qube
Jul 02 12:17:43 dom0 lvm[1456]: No longer monitoring thin pool qubes_dom0-vm--pool-tpool.
Jul 02 12:17:43 dom0 lvm[1456]: Monitoring thin pool qubes_dom0-vm--pool-tpool.
Jul 02 12:17:43 dom0 lvm[1456]: No longer monitoring thin pool qubes_dom0-vm--pool-tpool.
Jul 02 12:17:44 dom0 lvm[1456]: Monitoring thin pool qubes_dom0-vm--pool-tpool.
Jul 02 12:17:56 dom0 kernel: loop3: detected capacity change from 0 to 1399448
Jul 02 12:17:56 dom0 qubesd[2129]: vm.tmp-qube: Setting Qubes DB info for the VM
Jul 02 12:17:56 dom0 qubesd[2129]: vm.tmp-qube: Starting Qubes DB

Jul 02 12:17:56 dom0 qubesd[2129]: vm.tmp-qube: Activating the tmp-qube VM

/var/log/xen/console/guest-tmp-qube.log
xen-console-tmp-qube-wired.log (60.9 KB)

tmp-qube journalctl logs 
Jul 02 12:18:02 localhost kernel: Linux version 6.12.11-1.qubes.fc37.x86_64 (mockbuild@0d48d2e97ab5428ca4981acbfa2ff9e2) (gcc (GCC) 12.3.1 20230508 (Red Hat 12.3.1-1), GNU ld version 2.38-27.fc37) #1 SMP PREEMPT_DYNAMIC Tue Jan 28 01:03:48 GMT 2025
Jul 02 12:18:02 localhost kernel: Command line: root=/dev/mapper/dmroot ro nomodeset console=hvc0 rd_NO_PLYMOUTH rd.plymouth.enable=0 plymouth.enable=0 clocksource=tsc xen_scrub_pages=0 swiotlb=2048 apparmor=1 security=apparmor


Jul 02 12:18:11 tmp-qube systemd[1]: Startup finished in 4.324s (kernel) + 10.240s (userspace) = 14.564s.

Below are some observatons

Description Wired connection using switch Wired connection direct
Time taken between the lines starting tmp-qube and Activating the tmp-qube VM in dom0 logs 2 seconds 14 seconds
Time taken between the lines Activating the tmp-qube VM in dom0 logs and first entry in tmp-qube logs 2 seconds 6 seconds
Time taken between the first entry and the line Startup finished in tmp-qube logs 2 seconds 9 seconds
Overall elapsed time between the lines starting tmp-qube in dom0 logs and Startup finished in tmp-qube logs 5 seconds 29 seconds

Has anyone encountered similar scenario? Is there any other information I can collect to determin the reason for slowness?

1 Like
2

switch

[    0.000000] Xen version 4.17.
[    0.068332] tsc: Fast TSC calibration failed

direct

[    0.000000] Xen version 4.17.
[    0.996707] tsc: Fast TSC calibration failed

and from now on direct line adds 10-100 milliseconds with any line but switch line adds only 1-10 milliseconds respectivelly

Does switch line means WiFi connection and wired means eth connection by usb->eth dongle?

On internet there’s only usb problems with TSC

1 Like
3

NovaCustom laptops comes with ethernet port. No need for usb dongle. Both scenarios are wired connections. By switch, I refer to network switch.

switch = laoptop eth connected to switch and switch eth connected to router.
wired / direct = laptop eth connected to router.

1 Like
4

There was also realtek ethernet PCI card problem, but it wasn’t described if router or router+switch connected. It was driver problem.

Does router have VLAN’s configured?
Does switch have VLAN’s configured?

Have ended all my ideas.

1 Like
5

After looking at your previous post…

Warning: many "maybe"s are here…

I am thinking that the scenario with "Captive portal " requiring login maybe does not start with any internet access. Can the extra delay come from services like VPN/mullvad which send packets and then wait for timeout? Every VM must normally wait for its NetVM before it can start… and maybe the VPN startup must finish before the previous VM declares it has started.

Maybe you can see more details with

journalctl -b -1

(from memory - I think it is the messages for the last boot of the VMs.)

There is also a systemctl or systemd command for extra information about delays during booting. Maybe it is this:

systemd-analyze blame

You can look in each VM, and look for the long delay(s).

[Edit: Great thanks to @FranklyFlawless for fixing the mess I seem to have left here ]

1 Like
6

Have an idea.
Do you have updated BIOS?
Do you disabled any unnecesary boot option in BIOS like CD\DVD boot, PXE boot, wake on eth?
Maybe some other strange ethernet BIOS options that interacts with router but switch cuts the crap out?

1 Like
7

@KitsuneNoBaka @phceac I no longer able to recreate the issue. Looks like it resolved itself.

1 Like
8

Thank you for the news…

…did you have any updates or restarts of Dom0 or templates? Or maybe it’s your lucky day!

Personally, I just hate those problems where “it just stopped/started/changed it’s way of working” - they make me feel very insecure.