Survey: CPU and VM boot time

As for Qubes VM start/stop, perhaps there are snapshot rotation steps that can be left in the background and considered non-critical. For example, if VM startup always checks for the situation where a volume has been left in its active unsettled state (which it already does), and that unsettled volume name has the newest timestamp, then you can recover that situation easily with zero risk. So there could be a benefit to treating VM shutdown rotations as an unimportant background process (also using ‘ionice’).

I added an entry using debian-11-minimal on 4.2-RC4, I hope it’s fine :smiley:

1 Like

Added my R4.2 results to the top. Times have returned back down to R4.0 levels.

Also, from here on assume that people reporting their times are using the latest available version of Debian for their release unless otherwise stated.

E.g. If someone posts for R4.1, then it’s assumed they’re using Debian 11. If for R4.2; Debian 12 until a template for Debian 13 is released.

I added my R4.2 result to the bottom of the list.
I was between 3.4 and 3.9, so I decided to go with 3.6 since most of my tests were close to it.

Update my result for v4.2 with Debian 12
(Specs: i9-13900K, DDR5 memory, Samsung evo 990 Pro NVMe drive)

Fastest boot time was 3.22
Fastest 10 run average was 3.38
Total average of 30 runs ~3.44

I tried doing the test with the Debian 11 template, and it’s a lot faster.

Fastest D11 boot time was 2.61
Total average of 20 runs ~2.93

2 Likes

Update my results for Qubes R4.2.0 with debian 12.

Fastest boot time was 3.390
10 run average was 3.496

R4.2 final release really has astonishing impact on individual qube boot time. Qmemman memory hotplug support since core-admin v4.2.18 (core-admin v4.2.18 (r4.2) · Issue #4121 · QubesOS/updates-status · GitHub) indeed boosted boot speed significantly, for qubes that have memory balancing on.

If we gotta figure out what makes debian-12 slower than debian-11, we may gain even further improvements.

I suggest grouping the big table in the original post by Qubes version, like three separate tables, for Qubes R4.0, R4.1, R4.2, respectively.

2 Likes

I took your suggestion and reformatted the entire post while I was at it. Thanks

Every release we get a step closer to the 1-second boot

2 Likes

I looked into this a little, it seems to be systemd-binfmt.service that is the main difference between 11 and 12.

D12 with systemd-binfmt.service disabled
Fastest boot 2.7s
20 run average ~2.95s

1 Like

I’m still around 17s, but that’s better than 4.1. ^^
Also I noticed that my CPU load is less in 4.2 than in 4.1 - apparently qubesd or Xen are performing way better.

Not sure why you guys seem to get so much better results. Probably much better hardware (T530 here)…

@tripleh Do you have HDD or SSD in your laptop? A SSD has huge advantages over a HDD.

Do you think this and other findings can be formulated into Quick QoL tips?

I double-checked all entries after @johnboy pointed out that I dropped his entry in the reformatting process and re-added @augsch 's other entry.

1 Like

I have no idea why it has been added or how it’s used, if used at all. It adds support for different executable formats, is it safe to disable/remove it?

I just masked the service in the tamplate I used to test, I believe you can just disable/mask it in the template and unmask/enable it from rc.local in the appvm.

If it isn’t needed, it should probably just be removed from the minimal template, and the user can manually install it if needed.

Is there a Linux guru who can help out? Lowering boot times with a quick tip will help a lot of people but I need more info lest this causes major issues.

I looked into it a bit more, I think systemd-binfmt is enabled as a side effect of upgrading systemd from v247 to v252, it adds After=local-fs.target to the default service file.

It didn’t need to autostart in D11, my gut feeling is that it also isn’t needed in D12, but I could be wrong.

I think the safe way to disable it, would be to copy the service file from /lib/systemd/system to /etc/systemd/system and remove the after line, it should allow the service to start if needed.

2 Likes

I’d rather not show off my lack of Linux skills, so would you be kind enough to rephrase this as a few commands so I can add it to the post?

This is what I have been doing

sudo cp /lib/systemd/system/systemd-binfmt.service /etc/systemd/system
sudo vim /etc/systemd/system/systemd-binfmt.service

In systemd-binfmt.service delete the line After=local-fs.target

On my system, binfmt still loads, but it boots faster.

2 Likes

Added the “Modifications” column. Only items in the modifications table can be added to that column.

To create a modification, post the instructions to implement it here, then create an entry and link your post to it.

1 Like

Can this ^^^ be understood to mean that this tweak vvv

has no effect except to speed up the debian 12 boot? (on your system)

Yes, I have not noticed any negative effects after making the change, I don’t think it prevents the service from being started if needed. I also don’t think the service is really used in Qubes OS, but I could be wrong.

It allows for non-native binary formats to be executed, e.g. if you mount a windows drive and tries to execute a .exe file it would be able to map the format to wine, if it’s installed, and it can do the same with emulators and such, but it probably also has other uses.

2 Likes