Installing Qubes 4.1 in a Xen HVM domU (nestedhvm=1)

User Support General
10

Thanks all for the links and information ! I’ll see what works best for me ^^ For now I don’t really need PCI PT as I’m using X on dom0. I don’t know if nested PCI PT is even possible. I’d only need it for audio, but why is audio handled by a separate domain in Qubes, security ? For now I use soundhw="hda" in domU config and pulseaudio/pax11publish to play in dom0.

Anyways, I finally managed to install and run Qubes 4.0.4 in a HVM domU, and I have questions :

  • how does the IOMMU work in case of nested virt ? I -think- I have one in dom0 (iommu: Default domain type: Translated [\n] AMD-Vi: AMD IOMMUv2 functionality not available on this system), but Qubes install reports I don’t have it. ls /sys/kernel/iommu_groups is empty on dom0.
  • is “hap = 1” needed ? I don’t know what it does really, but xen wiki/manpages tell to use it with nested virt. I’ve not seen (yet ?) any difference with or without, I even think it’s at 1 by default anyways. Although since I activated it in qubes.cfg, I have many lines like this in xenstored.log: “data/meminfo_free 2405480 [\n] control/feature-balloon 1”, don’t know if it’s related or not. Autoballooning is disabled in my dom0 (and by default in qubes’), and I use the same values for dom0_mem and max in both xen cmd lines. Well, I changed it in qubes only AFTER the first boot.
  • when qubes boots, I get a few “(XEN) d16v0: Invalid EFER update: 0x1d01 -> 0x3901 - Unknown bits set” lines, what does that mean ?
  • when qubes run, the dom0 xl dmesg log is filling with “(XEN) d53v1 Unexpected nested vmexit: reason 0x6e” and “[...] reason 0x87”, to the point that the command never stops ! Where can I read what those error messages mean ? And how can I increase xl dmesg size ? I lost all previous pertinent logs …
  • should I run qubes domU in HVM or PVH mode ?
  • do you know someone who tests nested virt ? ^^
  • off-topic, but all my attempts at installing qubes from a NFS-backed ISO failed with “Payload SHA256 digest: BAD” on various packages, any idea why ?

And please, don’t stop discussing, I learn a lot ; )

1 Like