Qubes Cpu Pinning: automatically pin CPUs for VMs

[Atrate]

Hey there. I’m releasing yet another of “my” scripts for everyone to use .

This time “my” is in quotation marks, as it’s just a small modification made to the scripts by @noskb and @renehoj in CPU Pinning Alder Lake.

Regardless, it has a small modification (allowing for soft pinning) so I thought I’d share. I also want to have a “recommended” way of CPU pinning for my Window Boost script: Qubes Window Boost: Dynamic, window focus-based CPU pinning

GitHub - Atrate/Qubes-CPU-Pinning: Automatically pin Qubes VMs to CPU cores based on...

[Atrate]

@Talkabout

@Atrate for your CPU pinning script, have you considered to add an option to pin to performance cores only when machine is ac powered? In case it is in battery mode it might not be expected to “waste” battery life by only using the “non-optimized” cores.
I will research today a little bit how this can be achieved and will report back.

@Talkabout Audio dropouts and stutter - #40 by Talkabout I presume you meant this script and not Window Boost.

It would be very easy to implement with psutil:

user@dom0 ~ % python3
Python 3.11.6 (main, Oct  3 2023, 00:00:00) [GCC 12.3.1 20230508 (Red Hat 12.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> psutil.sensors_battery().power_plugged
True

Personally, I’d still prefer to pin qubes to p-cores even on battery power, but I can see your use-case.

Implementing this in the vCPU pinning script would be very simple and I will gladly accept a PR (or do it myself, but I don’t think I’ll have time right now). It would not apply to aleady-started qubes, though, so that makes me think that it may be a good idea to also write something that would use udev triggers to re-pin all VMs to e-cores on battery power. Technically it could be integrated within this script, too (maybe via pyudev).

EDIT:

For what it’s worth, it may also be interesting to consider applying settings for xen_hwp, e.g. xenpm set-cpufreq-hwp powersave maximum:25 for power saving and xenpm set-cpufreq-hwp balance otherwise.

[solene]

I need some help to ensure my parameters are correct with regard to the CPU topology. I though it was 0-7 for P and 8-15 for E but it turns I was wrong? (or it changed?)

The output of xenpm get-cpu-topology is the following:

[user@dom0 ~]$ xenpm get-cpu-topology
CPU	core	socket	node
CPU0	 0	 0	 0
CPU1	 0	 0	 0
CPU2	 4	 0	 0
CPU3	 4	 0	 0
CPU4	 8	 0	 0
CPU5	 8	 0	 0
CPU6	 12	 0	 0
CPU7	 12	 0	 0
CPU8	 16	 0	 0
CPU9	 17	 0	 0
CPU10	 18	 0	 0
CPU11	 19	 0	 0
CPU12	 20	 0	 0
CPU13	 21	 0	 0
CPU14	 22	 0	 0
CPU15	 23	 0	 0

Does it mean this configuration is right for it? SMT should be on from what I have in grub.conf

P_CORES = '0,1,4,5,8,9,12,13'
E_CORES = '16,17,18,19,20,21,22,23'

[Atrate]

If your CPU only has SMT on P-cores then IMO this config should be correct:

P_CORES = '0-7'
E_CORES = '8-15'

[Atrate]

SMT should be on from what I have in grub.conf

Was there a patch for Xen recently that made it work with asymmetric CPUs + SMT securely? Last I checked it had some problems with that (setting scheduling granularity failed).

[solene]

SMT is still disabled by default because it’s a shitshow that can’t really be considered safe I guess. Is it what you mean?

[solene]

Are you sure soft affinity works by default?

xl man page mentions that soft affinity only works with the credit1 scheduler, from xl info it looks like credit2 is used (by default?).

[renehoj]

Affinity doesn’t work with credit2, you can set it, but Xen doesn’t respect it, I think it’s a bug.

AMD CPUs can use NUMA to keep cores on the same CCD, this also doesn’t work with credit2. It correctly sets the affinity for each CCD, but doesn’t respect it when assigning the cores.

It works as expected with credit(1)

[Atrate]

@solene

SMT is still disabled by default because it’s a shitshow that can’t really be considered safe I guess. Is it what you mean?

I was referring to this comment by @renehoj, wherein you could help make SMT (moderately) more secure by setting sched-gran=core, but it was not possible with asymmetric CPUs.

Is it safe to use hyper-threading (SMT) with Qubes OS if done the 'correct way'?

sched-gran doesn’t work on CPUs that are asymmetric, like the Intel E and P core CPUs. It also does solve all problems, transient HT exploits are still possible even if you change the scheduling granularity to core. If you want to use smt then sched-gran=core is better than cpu, if you can use it, but it doesn’t make smt safe. If smt is worth it depends on your threat model, and your work load, but if you need the highest level of security you shouldn’t enable smt, it’s never going to be safe…

@solene

Are you sure soft affinity works by default?

I am not fully sure about that, althought the Xen wiki page about CPU pinning does not mention that it should only work with credit(1). Later in the day I can do some testing based on the amount of fan noise/temperature rise when I perform soft pinning.

[solene]

I don’t think it’s a bug, it’s not implemented: https://wiki.xenproject.org/wiki/Credit2_Scheduler#Missing_Features

[Atrate]

Seems there used to be patches “in review” but I have no idea how far they got (abandoned?).

https://www.mail-archive.com/xen-devel@lists.xen.org/msg111357.html

Once I have a second, I’ll add a warning to the script that soft affinity will only work with credit(1).

[KitsuneNoBaka]

As to how check which core i P and which is E, turbo frequency analyze should be sufficient

xenpm get-cpufreq-para |awk 'BEGIN{i=1} /cpu id/{ID[i]=$4} /turbo/{FREQ[i]=$7; i++} END{for (x=1; x<i; x=x+2){print("CPU ID:",ID[x],"\tFreq:",FREQ[x])}}'