Mods: please merge this post with the mentioned thread.
@unman: continuation of Preloaded dispoables in 4.3
On 25-10-28 12:43:11, deeplow via Qubes OS Forum wrote:
[45_2] deeplow Volunteer Moderator
October 28
I don’t see this behavior. For me preloaded disposables are “starting” in under
5 seconds, which is faster than regular boots, if feels. But I haven’t exactly
benchmarked.
But I think @ben-grande showcased some benchmark during his summit talk this
year.
Previous Replies
[45] unman Qubes Team
October 28
I’ve had reports of issues with the preloaded disposables in 4.3, which
agree with my own experience.
This is my first time using the forum for a long message via e-mail, I
hope everything works. If this doesn’t work, we can try using the
qubes-users mailing list (which I am not subscribed to yet). And… it failed.
Can you please share the reports? So I could comment on that.
I thought this would be a great new feature, but something seems off.
I find that the time saving is marginal, and if I start a number of
disposables quickly, the load time is actually slower than under 4.2.
- Can you measure the time the qube takes too boot and the time it takes
to use it? I’ve had some templates taking a longer time to boot
because some services were delaying the boot.
- When you say quickly, is it concurrent or sequential calls?
- I think that comparing against 4.2 is not a good metric in this case,
you should compare against the same version and different modes, with
preloaded disposables and without preloaded disposables. Comparing
across versions may be impacted by the template being different, then
you would need to compare no preloaded disposables on 4.2 and 4.3 to
see the difference.
Measure startup time:
qvm-shutdown default-dvm
time qvm-run --no-gui --service -- default-dvm qubes.WaitForRunningSystem
qvm-shutdown default-dvm
If it takes too long to start:
systemd-analayze blame
systemd-analayze plot
Measure preload usage time:
qvm-features default-dvm preload-dispvm-max 1
disp="$(qvm-features default-dvm preload-dispvm)"
while ! qvm-check --paused -- "$disp"; do sleep 1; done
time qvm-run --no-gui --service -- default-dvm qubes.WaitForRunningSystem
Increase preload-dispvm-max and loop the qvm-run calls. I recommend
experimenting with different templates (Debian and Fedora).
Note that this test is barebones and checks total time of the qvm-run
call, not how fast the target replied. The performance benchmark gives
different results for different methods:
- Admin API (qvm-run): dom0 to disposable
- Admin API (raw): dom0 to disposable
- Qrexec (qrexec-client-vm): qube to disposable
The Admin API (raw) results shows enough information on stages:
- Execution
- Cleanup
- Total = Execution + Cleanup
But the raw method is a bit skewed because users normally use
qvm-run or qrexec-client-vm, which are a bit slower than the raw
variant. Anyway, it is the best method to see the per stage result.
The cleanup time can double on sequential calls because during this
process, other disposables are being preloaded.
I see that this is on GitHub as a possibility, but it’s a regular
feature for me and some other users. Does anyone else see this behavior?
I’d like to get a feeling for incidence before going to GitHub.
OpenQA reports preloaded disposables performance in
JSON
with some images generated from that results file. There is no alt text
for the images, but there is an idea to use matplotalt in the future.
If you have recommendations on how to render matplotlib graphs with alt
text, I’d be glad. Currently difficult to use matplotalt because it
requires pip, not available in distribution repositories. Note that
if you have the results files, you can generate the images from any
machine even outside of Qubes OS.
OpenQA @hw7 has a weak hardware with the following specs:
- RAM: 16226 MiB
- CPU 13th Gen Intel Core i3-1315U
And software:
- Qubes R4.3-rc2
- Kernel 6.12.47-1.fc41
- Xen 4.19.3
And it still results on a great increase. Raw API calls from dom0 to
debian-13-xfce preloaded disposables, on 12 sequential calls, on calls
that don’t require a GUI, qubes.WaitForRunningSystem (results
in seconds):
Execution:
- 0 preloads: start 6.2, lowest 5.8, highest 7.1, mean 6.3;
- 3 preloads: start 0.4, lowest 0.4, highest 5.4 (outlier), mean 1.6;
- 4 preloads: start 0.4, lowest 0.3, highest 5.6 (outlier), mean 1.1;
- 5 preloads: start 0.4, lowest 0.4, highest 2.6 (outlier), mean 0.9;
Total:
- 0 preloads: start 7.0, lowest 6.9, highest 8.0, mean 7.1;
- 3 preloads: start 2.6, lowest 2.2, highest 7.3 (outlier), mean 3.9;
- 4 preloads: start 2.4, lowest 1.0, highest 7.1 (outlier), mean 3.5;
- 5 preloads: start 2.6, lowest 1.8, highest 5.6 (outlier), mean 3.4;
Images:
If you benchmark with qvm-run, you will see the total, while the
results above uses raw Admin API. You can also benchmark on your own
machine, the script is available at
qubes-core-admin/tests/dispvm_perf.py. The reader can be run outside
of Qubes and is available in the same directory at
dispvm_perf_reader.py.
The debian-13-xfce template with qvm-run and GUI call
(qubes.WaitForSession) had some weird results recently, they took longer
to be used than their no preload counterpart, 23% slower actually. No
idea why at this moment but on the same test, fedora is 80% faster against it’s no preload counterpart, which would be expected for debian also.