Security problem? Nitro-PC microcode-updates

yea the MMIO vulnerability listed in Dom0 terminal is what made me start looking around. Strangely is Dom0 it list the vulnerability but if you run terminal in a template VM it’s marked as mitigated. Super confusing why microcode mitigations would be different between VM’s. I would think the mitigations would be most important in dom0. Can you run “lscpu” in both a template VM terminal and a Dom0 terminal and let me know if the MMIO vulnerability is mitigated for you in both?

This part I did figure out, it list 20220510 package as both update available and updated which confused me into thinking 20220510 was the update that was installed and loaded. It actually isn’t, it’s the last update package that was loaded. what it is saying is that 20220510 did/does have an updated available in the first column and the 2nd column is confirming if that update was installed or not. Only the “loaded microcode version” column shows the updated info from the current microcode package. It would have been less confusing if the first column had 20220510 and if it had an updated available, and the second column with the new update package 20230808 and that it was installed along with showing the current microcode version.

<pre>CPU  F-M-S/PI     Loaded microcode  20220510 update  20220510 update
                       version           available        installed
      0  06-8e-0a/80   000000f4          yes              yes
      1  06-8e-0a/80   000000f4          yes              yes
      2  06-8e-0a/80   000000f4          yes              yes
      3  06-8e-0a/80   000000f4          yes              yes
 (cores) (signature)  (current)         (#=old package)  (#=old package)
</pre>

from all the research I’ve done I’m positive I have the current microcode now but for some weird reason Dom0 doesn’t show the mitigations for MMIO which the microcode updates did address in the past already, I still don’t know why, I put up a post and waiting on feedback.

yea the MMIO vulnerability listed in Dom0 terminal is what made me start looking around. Strangely is Dom0 it list the vulnerability but if you run terminal in a template VM it’s marked as mitigated.

Here it shows in dom0 and in domUs as mitigated.

from all the research I’ve done I’m positive I have the current microcode now but for some weird reason Dom0 doesn’t show the mitigations for MMIO which the microcode updates did address in the past already, I still don’t know why, I put up a post and waiting on feedback.

Maybe the available microcode still hasn’t reached the Fedora repos from which dom0 updates.

What is your CPU?

i5-8350U Kaby Lake-U refresh

Per this list INTEL-SA-00615 should be mitigated on this processor.

I don’t know what Nitro PC has such CPU.

In case your computer is different, I would suggest that you open a separate thread with title focused on your system specifics. Ideally, you could also open an issue on GitHub, if you are confident this is something that can be fixed by Qubes OS’s developers.

I’m not sure the brand/PC has any impact on this topic at all. It’s included in the title but other than that’s there’s little to no relevance to the topic. He’s asking about microcode mitigations which are not PC brand/model specific rather whole groups of Intel processor families and can be updated outside of the control of Nitro key at all. He also had questions about MMIO vulns showing, which is a question I have as well as both the processor he mentioned and mine DO have microcode updates to mitigate this issue and shouldn’t be showing on lscpu.

The i7-10510U cpu in his devices (CPUID 806EC) can be found in this chart previously linked. The MMIO vuln he had quesitons about is INTEL-SA-00615 by following that chart linked (ctrl+f 806EC) and scrolling over to the INTEL-SA-00615 column you can see this issue on his processor WAS mitigated with a microcode update. He should not be getting that vuln message in Qubes. You said your Nitro is a i7-10610U, which is also cpuid 806EC family abbreviation CML-U62. You reported a microcode version 000000f0this isn’t correct for your machine either you should have 000000f8 as your newest microcode.

1 Like

I’m not sure the brand/PC has any impact on this topic at all. It’s included in the title but other than that’s there’s little to no relevance to the topic.

I am not arguing or discussing how others put their questions. I have simply suggested that you open a separate thread/issue for your specific model for your own good, unless you expect that people will look for your specific questions in a thread with title which says something else. It’s up to you. Generally, staying on topic is helpful for present and future readers, and considered good netiquette.

You reported a microcode version 000000f0this isn’t correct for your machine either you should have 000000f8 as your newest microcode.

As of today, it is 000000f8.

Re. your earlier:

Super confusing why microcode mitigations would be different between VM’s.

In Qubes 4.2.0, dom0 uses Fedora 37. If your VM template is Fedora 38 it may include newer packages, hence the difference. Perhaps the proper way to address this issue is to report it upstream (to Red Hat), so they can make microcode updates for your CPU available in Fedora 37.

The Intel microcode is manually updated by the Qubes developers. Everything is in this repository:

Can you check the first line of xl dmesg in dom0 and see if you have something like this:

(XEN) Built-in command line: ept=exec-sp spec-ctrl=unpriv-mmio

The spec-ctrl=unpriv-mmio option should load the mitigation from Xen using the provided intel microcode.

In configurations where less privileged domains have MMIO access to
buggy endpoints, spec-ctrl=unpriv-mmio can be enabled which will cause
Xen to mitigate cross-domain fill buffer leakage, and extend SRBDS
protections to protect RNG data from leakage.

The unpriv-mmio= boolean indicates whether the system has (or will have) less than fully privileged domains granted access to MMIO devices. By default, this option is disabled. If enabled, Xen will use the FB_CLEAR and/or SRBDS_CTRL functionality available in the Intel May 2022 microcode release to mitigate cross-domain leakage of data via the MMIO Stale Data vulnerabilities.

https://xenbits.xenproject.org/xsa/advisory-404.html
https://xenbits.xenproject.org/docs/4.14-testing/misc/xen-command-line.html

Since my CPU is not affected, I can’t investigate further. You may be able to find more information in xl dmesg.

The microcode to mitigate this issue is available in both version, which is also installed, as the OP stated in a previous post this question is still unresolved for them. I’m not asking a new question I’m continuing the same discussion and providing my own context on my processor and Qubes which is identical to the issue he has and is applicable to this topic. Even though we have different processor we both have the same mitigation update for this issue for our respective processors/family. What isn’t applicable to the topic at all is the brand. I personally dont see an etiquette issue with adding additional context to the same topic issue. Not trying to be confrontational just don’t agree with the assessment.

He left this post thinking that the only way to fix this issue is for Intel to provide updates and therefore doesnt want to use the PC for higher threat needs and also left not trusting Nitro even though what they told him is correct that the microcode on their machines to mitigate that issue is installed. Intel did provided the microcode for this that is applicable to his processor and it is available in both version of Fedora so that should have no impact. I truthfully dont think it has anything to do with his microcode being out of date as being the reason dom0 is return vuln. Likely his microcode is fully up to date just like mine and there’s something weird going on still reporting vuln, Ive seen his post about it along with another which none of them finished with a clear answer on why Dom0 is returning these vuln even though the microcode to mitigate is up to date and installed. Like him I havent seen anything that should assure someone that the microcode vulns show when pulling cpu-microcode-info is not something to worry about even when the processors in question have a confirmed mitigation for it and you can confirm it’s installed. It could be benign but it would still be nice to have a solid answer on it. I’m glad yours doesnt return it but that doesn’t seem to universal. The fact that both OP and you have the exact same family processor therefore the exact same microcode mitigations and that his returns vuln and yours doesn’t would suggest it has nothing to do about a microcode being needed by either Intel or Redhat and more towards some kind of specific configuration issue within Qubes or some conflict that is returning untrue results out of lscpu. Like i said i’ve found at least 3 cases and I’m sure there could be more as people dont often check their microcode info. I’m trying to help answer his question as much as mine.

1 Like

requested pull below.

Isn’t it distributed by the kernal now? is that something they need to do manually also?

xl dmesg
(XEN) Built-in command line: ept=exec-sp spec-ctrl=unpriv-mmio
 Xen 4.17.2
(XEN) Xen version 4.17.2 (mockbuild@[unknown]) (gcc (GCC) 12.3.1 20230508 (Red Hat 12.3.1-1)) debug=n Wed Jan  3 00:00:00 UTC 2024
(XEN) Latest ChangeSet: 
(XEN) Bootloader: GRUB 2.06
(XEN) Command line: placeholder console=none dom0_mem=min:1024M dom0_mem=max:4096M ucode=scan smt=off gnttab_max_frames=2048 gnttab_max_maptrack_frames=4096 no-real-mode edd=off
(XEN) Xen image load base address: 0x7d800000
(XEN) Video information:
(XEN)  VGA is graphics mode 1920x1080, 32 bpp
(XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  [0000000000000000, 0000000000057fff] (usable)
(XEN)  [0000000000058000, 0000000000058fff] (reserved)
(XEN)  [0000000000059000, 000000000009cfff] (usable)
(XEN)  [000000000009d000, 00000000000fffff] (reserved)
(XEN)  [0000000000100000, 000000003fffffff] (usable)
(XEN)  [0000000040000000, 00000000403fffff] (reserved)
(XEN)  [0000000040400000, 0000000074f8bfff] (usable)
(XEN)  [0000000074f8c000, 0000000074f8cfff] (ACPI NVS)
(XEN)  [0000000074f8d000, 0000000074f8dfff] (reserved)
(XEN)  [0000000074f8e000, 000000007e8f3fff] (usable)
(XEN)  [000000007e8f4000, 000000007ff2bfff] (reserved)
(XEN)  [000000007ff2c000, 000000007ff99fff] (ACPI NVS)
(XEN)  [000000007ff9a000, 000000007fffefff] (ACPI data)
(XEN)  [000000007ffff000, 000000007fffffff] (usable)
(XEN)  [0000000080000000, 0000000087ffffff] (reserved)
(XEN)  [0000000088600000, 000000008c7fffff] (reserved)
(XEN)  [00000000f8000000, 00000000fbffffff] (reserved)
(XEN)  [00000000fe010000, 00000000fe010fff] (reserved)
(XEN)  [0000000100000000, 0000000a717fffff] (usable)
(XEN) ACPI: RSDP 7FFFE014, 0024 (r2 LENOVO)
(XEN) ACPI: XSDT 7FFAC188, 0124 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: FACP 7FFE1000, 00F4 (r5 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: DSDT 7FFBA000, 22090 (r2 LENOVO SKL             0 INTL 20160527)
(XEN) ACPI: FACS 7FF3E000, 0040
(XEN) ACPI: SSDT 7FFE9000, 1320E (r2 LENOVO DptfTabl     1000 INTL 20160527)
(XEN) ACPI: SSDT 7FFE8000, 03DB (r2 LENOVO Tpm2Tabl     1000 INTL 20160527)
(XEN) ACPI: TPM2 7FFE7000, 0034 (r3 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: UEFI 7FF53000, 0042 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: SSDT 7FFE3000, 30E4 (r2 LENOVO  SaSsdt      3000 INTL 20160527)
(XEN) ACPI: SSDT 7FFE2000, 05C6 (r2 LENOVO PerfTune     1000 INTL 20160527)
(XEN) ACPI: HPET 7FFE0000, 0038 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: APIC 7FFDF000, 012C (r3 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: MCFG 7FFDE000, 003C (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: ECDT 7FFDD000, 0053 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: SSDT 7FFB8000, 1CA3 (r2 LENOVO RVP7Rtd3     1000 INTL 20160527)
(XEN) ACPI: SSDT 7FFB6000, 163C (r2 LENOVO ProjSsdt       10 INTL 20160527)
(XEN) ACPI: BOOT 7FFB5000, 0028 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: BATB 7FFB4000, 004A (r2 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: SLIC 7FFB3000, 0176 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: SSDT 7FFB1000, 17AE (r2 LENOVO  CpuSsdt     3000 INTL 20160527)
(XEN) ACPI: SSDT 7FFB0000, 056D (r2 LENOVO    CtdpB     1000 INTL 20160527)
(XEN) ACPI: SSDT 7FFAF000, 0678 (r2 LENOVO UsbCTabl     1000 INTL 20160527)
(XEN) ACPI: LPIT 7FFAE000, 0094 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: WSMT 7FFAD000, 0028 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: SSDT 7FFFD000, 01D8 (r2 LENOVO   HdaDsp        0 INTL 20160527)
(XEN) ACPI: SSDT 7FFAB000, 04FC (r2 LENOVO TbtTypeC        0 INTL 20160527)
(XEN) ACPI: SSDT 7FFAA000, 02D1 (r2 LENOVO     Wwan        1 INTL 20160527)
(XEN) ACPI: DBGP 7FFA9000, 0034 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: DBG2 7FFA8000, 0054 (r0 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: MSDM 7FFA7000, 0055 (r3 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: DMAR 7FFA6000, 00A8 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: ASF! 7FFA5000, 00A0 (r32 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: FPDT 7FFA4000, 0044 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: BGRT 7FFA3000, 0038 (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) ACPI: UEFI 7FF3B000, 013E (r1 LENOVO TP-N22       1560 PTEC        2)
(XEN) System RAM: 40700MB (41677372kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 7ff3e000/0000000000000000, using 32
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
(XEN) Switched to APIC driver x2apic_mixed
(XEN) CPU0: TSC: ratio: 158 / 2
(XEN) CPU0: bus: 100 MHz base: 1900 MHz max: 3600 MHz
(XEN) CPU0: 400 ... 1900 MHz
(XEN) xstate: size: 0x440 and states: 0x1f
(XEN) Speculative mitigation facilities:
(XEN)   Hardware hints: RSBA
(XEN)   Hardware features: IBPB IBRS STIBP SSBD L1D_FLUSH MD_CLEAR SRBDS_CTRL GDS_CTRL
(XEN)   Compiled-in support: INDIRECT_THUNK
(XEN)   Xen settings: BTI-Thunk JMP, SPEC_CTRL: IBRS+ STIBP+ SSBD-, Other: SRB_LOCK+ IBPB-ctxt L1D_FLUSH VERW BRANCH_HARDEN
(XEN)   L1TF: believed vulnerable, maxphysaddr L1D 46, CPUID 39, Safe address 8000000000
(XEN)   Support for HVM VMs: MSR_SPEC_CTRL MSR_VIRT_SPEC_CTRL RSB EAGER_FPU
(XEN)   Support for PV VMs: MSR_SPEC_CTRL EAGER_FPU MD_CLEAR
(XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled (with PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU enabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN) Disabling HPET for being unreliable
(XEN) Platform timer is 3.580MHz ACPI PM Timer
(XEN) Detected 1895.995 MHz processor.
(XEN) Unknown cachability for MFNs 0xa0-0xff
(XEN) Unknown cachability for MFNs 0x88600-0x8c7ff
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) Enabling APIC mode.  Using 1 I/O APICs
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) Allocated console ring of 16 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN)  - VMCS shadowing
(XEN)  - VM Functions
(XEN)  - Virtualisation Exceptions
(XEN)  - Page Modification Logging
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 4 CPUs
(XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
(XEN) Initializing Credit2 scheduler
(XEN) Dom0 has maximum 744 PIRQs
(XEN)  Xen  kernel: 64-bit, lsb
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x4600000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000a48000000->0000000a50000000 (1006222 pages to be allocated)
(XEN)  Init. ramdisk: 0000000a6f28e000->0000000a717ffad3
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff84600000
(XEN)  Phys-Mach map: 0000008000000000->0000008000800000
(XEN)  Start info:    ffffffff84600000->ffffffff846004b8
(XEN)  Page tables:   ffffffff84601000->ffffffff84628000
(XEN)  Boot stack:    ffffffff84628000->ffffffff84629000
(XEN)  TOTAL:         ffffffff80000000->ffffffff84800000
(XEN)  ENTRY ADDRESS: ffffffff8353c320
(XEN) Dom0 has maximum 4 VCPUs
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
(XEN) Freed 644kB init memory

I have no idea about this, I did not go any further on the subject.

In your xl dmesg logs, I see this SRBDS_CTRL part within Speculative mitigation facilities that was quoted in one of the links I provided, but I don’t see the other one (FB_CLEAR).

Also, I’m not sure why and where the difference is, but I have some microcode lines like this in mine, but you don’t have anything similar in yours:

(XEN) microcode: CPU2 updated from revision 0x22 to 0x32, date = 2023-06-07
>>> What it does
***************************************************************************
Deploy an Intel microcode. This tool is obsolete and the microcode is the
subject to be distributed via kernel-firmware, however Intel still does not
supply the microcode in a form consumable by the Linux's microcode driver. So
that this tool transform Intel's microcode as well as deploy it.

looks like that git is legacy and that microcodes on it stopped being updated in 2019 after looking into it further and they are done via kernel now, which makes sense as if they werent QSB-098 wouldn’t make any sense. Still interesting to see they used to do it all manually, maybe they still do but with Kernel stuff that’s over my head.

I saw something similar to yours when I was first researching how to confirm my microcode was up to date and if my cpu was effected by certain vulns. I remember trying to get mine to show the info lines you’re referring to and I couldn’t and went down another path, so this could definitely be a clue.

It should, maybe this has changed with newer kernels, but based on this answer it still does not seem to be the case?

No, it’s isn’t Linux loading the microcode. Xen does it. You can find relevant message in xl dmesg.

Since the repo is still maintained (version 20231114 as of writing), this may still be how it is done.

I’m really starting to think this issue DOES have something to do with underlying issue that resulted in QSB-098 and it’s possible the fix is not being applied correctly to all machines. In that thread there is also another user who shows no evidence on microcode in his xl dmesg…

I did try just now to regenerate Dracut but no success.

brendanhoar
21d

Is there an order of operations necessary to be sure microcode is loaded after this fix was deployed?

On a fresh R4.2, I have installed kernel-latest (6.6.2-1) and dracut 059-5.fc37, but I think I installed kernel latest before performing the update on dom0 that brought in the new dracut version.

I do not see any evidence of microcode loading in xl dmesg or linux dmesg/linux journalctl output.

--

apparatus
21d

I don’t know how to check it but you can regenerate initramfs just in case:

sudo dracut --regenerate-all --force