CPU frequency set to base speed in every qube

[IDontKnowWhatImDoing]

Hey, everyone! Hope you’re doing well.

I should preface that I’m very new to Qubes and Linux in general, so I might be missing something obvious. If I am, sorry.

I’m using a desktop machine with an i7 Ultra 265k and a Z890 AYW GAMING WIFI W motherboard. I have the latest stable installation of Qubes.

The problem:
I can’t get anything to use more CPU than the base speed of the CPU (~3.9 GHz). xenpm get-cpufreq-average and xenpm start are both able to use other clock speeds, but no qube that I’ve made can. The output of xenpm get-cpufreq-para

cpu id               : 0
affected_cpus        : 0
cpuinfo frequency    : base [3900000] max [5500000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [26]
                     : nominal [45] highest [64]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 1
affected_cpus        : 1
cpuinfo frequency    : base [3900000] max [5500000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [26]
                     : nominal [45] highest [64]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 2
affected_cpus        : 2
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 3
affected_cpus        : 3
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 4
affected_cpus        : 4
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 5
affected_cpus        : 5
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 6
affected_cpus        : 6
cpuinfo frequency    : base [3900000] max [5500000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [26]
                     : nominal [45] highest [63]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 7
affected_cpus        : 7
cpuinfo frequency    : base [3900000] max [5500000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [26]
                     : nominal [45] highest [63]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 8
affected_cpus        : 8
cpuinfo frequency    : base [3900000] max [5500000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [26]
                     : nominal [45] highest [63]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 9
affected_cpus        : 9
cpuinfo frequency    : base [3900000] max [5500000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [26]
                     : nominal [45] highest [63]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 10
affected_cpus        : 10
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 11
affected_cpus        : 11
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 12
affected_cpus        : 12
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 13
affected_cpus        : 13
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 14
affected_cpus        : 14
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 15
affected_cpus        : 15
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 16
affected_cpus        : 16
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 17
affected_cpus        : 17
cpuinfo frequency    : base [3900000] max [4600000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [15]
                     : nominal [33] highest [46]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 18
affected_cpus        : 18
cpuinfo frequency    : base [3900000] max [5500000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [26]
                     : nominal [45] highest [63]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

cpu id               : 19
affected_cpus        : 19
cpuinfo frequency    : base [3900000] max [5500000]
scaling_driver       : hwp
cppc variables       :
  hardware limits    : lowest [1] lowest nonlinear [26]
                     : nominal [45] highest [63]
  configured limits  : min [1] max [255] energy perf [128]
                     : activity_window [0 hardware selected]
                     : desired [0 hw autonomous]
turbo mode           : enabled

Shows the correct clock speeds, but only the base speed is used.
This problem seems similar to this post: Intel CPU Frequency Scaling Broken · Issue #4604 · QubesOS/qubes-issues · GitHub . Which concerns me, as this post started in 2018, and I’d hope a fix has been found since 2018.
Modifying the grub with cpufreq=xen:performance does change the scaling driver and governor, but it displays the max speed as the base speed (just like that post), and the set-scaling-maxfreq command does nothing. I have been unable to use pstate drivers, and sudo dmesg | grep pstate returns
[ 2.084148] intel_pstate: CPU model not supported
Interestingly, though presumably intentionally (I’m new), the number before inter_pstate changes seemingly every reboot.

I am worried this is potentially a hardware compatibility issue with the CPU and the Xen hypervisor, which is disheartening, but maybe not super surprising since there appears to be at least one other compatibility issue with the recent Intel CPUs.

I’d love any help. I’m hopeful there’s a way to get this working properly. Thanks for reading

[renehoj]

I think you should expect dom0 to say intel_pstate isn’t supported, but I could be wrong.

Dom0 is just a VM, it doesn’t have access to the physical CPU, only Xen can control the pstate. If you want to change the state of the CPU you need to use xenpm, but the actual change is done by Xen and happens outside dom0.

[corny]

The GitHub link is pretty old and I have that CPU gen (8th gen) as well as a couple of other AMD and Intel CPUs that all boost above base freq in qubes

I also have the pstate not supported message. Qubes often warns me about things that make no difference to my performance and stability

When I do a high CPU util task like compiling and run in dom0

Xenpm start 1

I can see avg freq rises above the base

If you do the same, do you see no increase ?

If so, it is probably a new gen CPU issue.

[Atrate]

In my experience on the Ultra 155H AppVMs can use turbo frequencies without a problem, it just doesn’t show inside the AppVMs properly. You can check that by executing xenpm disable-turbo-mode and xenpm enable-turbo-mode and comparing the temperature/noise output/performance metrics of your AppVM.

[IDontKnowWhatImDoing]

@renehoj That theoretically makes sense. I booted with cpufreq=dom0-kernal because that should make dom0 handle the CPU (right?), and got the same error.

@corny The original post is extremely old, yes, I mentioned it because it had essentially the exact same symptoms I had. Though either I completely missed something, or messing around in the grub/turning off turbo and turning it on again somehow fixed it. Because it seems like it’s working now.

@ Atrate As stated just above this, it seems like it’s just working now? xenpm get-cpufreq-para is still telling me that it doesn’t have the correct max, and I can’t change the max, but it seems like it’s benching properly now. Interestingly, turning off turbo when booted with the additional grub command to change the scaling driver does nothing. It still boosts to max, which is odd but not really problematic for me. Using the hwp driver does show a significant change, so changes to that work. I think it’s possible that turning it off and on again did actually fix it, so, I’m glad you suggested it lol. My Windows HVM still doesn’t get proper speeds, but I imagine that’s a Windows problem now, since everything else benches properly.
Thanks everyone

[corny]

That’s good you got it working. I’ve never found that changing most of those parameters or the credit scheduler nor grub settings, made any actual difference to frequency or performance. The only thing that boosted it was changing bios settings for boost index multiplier. Perhaps your generation of CPU is different to mine

This thread has a lot of info if you care to fiddle more (though I found most of the tweaking to be ineffectual)

CPU Pinning Alder Lake

If you have an Alder Lake or newer Intel CPU, you probably want the option to load a qube on the P or E cores, and this might help someone who is new to xen. https://dev.qubes-os.org/projects/core-admin/en/latest/libvirt.html https://libvirt.org/formatdomain.html#cpu-allocation Dom0 First, you need to add the following to GRUB_CMDLINE_XEN_DEFAULT dom0_max_vcpus=4 dom0_vcpus_pin If you want to use SMT read the notes at the end. The CPU I’m using is the 12900K, which has 16 physical cores an…

[renehoj]

As far as I understand, cpufreq=dom0-kernel makes dom0 function as the driver, with intel_pstate it’s the CPU that is the driver. It doesn’t change the fact that dom0 is a VM that only has access to the virtual CPU.

I don’t think the OS can use the intel_pstate driver unless you are running a bare-metal OS where the kernel has access to the physical CPU, and with Xen dom0 is always going to be a VM.

In the output you posted, you can see the scaling_driver is hwp. The hardware is the driver, which means Xen is using intel_pstate.

[IDontKnowWhatImDoing]

I agree, I’m very happy I got that working. Though it’d be nice if my Windows HVM decided to magically fix itself too. I’m not sure. There is a super small difference in performance peaks between the drivers, but none of the settings seem to really do anything with the none hwp drivers, and with the hwp drivers it seems I’m mostly limited to turning off and on boost. That’s an interesting guide, I think I’d rather set up a soft affinity (where, by my understanding, it will prioritize giving certain cores to specified VMs, but still allow the scheduler to decide what is appropriate) instead of pinning, and even then only if it actually made a large difference. Theoretically, though, I shouldn’t have to do any of that, right?

@renehoj Ahhh, okay. That does make sense. So hwp is the intel_pstate driver?

[corny]

@IDontKnowWhatImDoing

Theoretically, though, I shouldn’t have to do any of that, right?

Theoretically yes. Renehoj was trying to optimize I think. And he largely gave up on it with Intel CPU and moved to AMD (although mostly due to CPU damage issues on 13th gen).

I don’t think micromanaging P and E cores is worth it for a little gain unless you do a lot of high CPU util tasks. Qubes has a fair few handicaps on using hardware efficiently due to security tradeoffs