Please do not drop support for PV

[enmus]

This could be a continuation of a discussing started with this post @brendanhoar posted on another topic, where he pointed out to a Github issue, which I can confirm too doesn’t have anything with Qubes OS 4.1, but obviously with some specific hardware components

@Demi argued existence of PV in general, but as I pointed out, significant number of USB controllers can run only in PV mode, especially xHCI controllers which obviously “don’t like” Qubes.

There is an issue on a Github, and the solution is ismple, but I don’t have an account there - just create sus-usb in PV mode:
https://github.com/QubesOS/qubes-issues/issues/5445

Issuing sudo journalctl -g 00:14.0 in dom0 gives

Summary

Feb 13 19:30:03 dom0 libvirtd[1865]: internal error: Unable to reset PCI device 0000:00:14.0: no FLR, PM reset or bus reset available
Feb 13 19:30:08 dom0 kernel: pciback 0000:00:14.0: xen_pciback: vpci: assign to virtual slot 0
Feb 13 19:30:08 dom0 kernel: pciback 0000:00:14.0: registering for 7
Feb 13 19:30:18 dom0 kernel: pciback 0000:00:14.0: can't derive routing for PCI INT A
Feb 13 19:30:18 dom0 kernel: pciback 0000:00:14.0: PCI INT A: no GSI

Documentation claims there is solution for the issue, but it won’t work.

Searching the web on this I found a patch on bugzilla.kernel.org from 2013 which needev kernel to be manually recompiled, but can’t seem to find it at the moment. I don’t know if this is related, since I am not competent by any means.

There is more recent bug reported for the same error but different device, related to Lenovo laptops.

There are also other cases for which PV mode is the only solution.

HCL list is already pretty limited, and dropping PV mode support could cause more users giving up on Qubes, so @marmarek and others, please do not drop support at least until Qubes 5.x.

Additional comments welcomed

Here’s several posts from the topic mentioned at the beginning and related to the subject.

Summary

@enmus

If there wasn’t a PV mode, a lot of USB controllers couldn’t/wouldn’t work/be used in Qubes via sys-usb…

@brendanhoar

@zithro

By the way, are Qubes domU in PV, HVM or PVHVM mode ? Is it selectable ? i don’t remember if Qubes uses Xen PV drivers or MUST use them.

dom0 is PV
domU can be HVM or PVH.

While PV domU is also supported in 4.0 under Xen 4.8 (but not recommended), that might not work in 4.1. I don’t have access to my 4.1 test box at the moment, but there’s an open issue for PV on 4.1 here: Linux PV domain crashes on Xen 4.14 · Issue #6052 · QubesOS/qubes-issues · GitHub .

B

@brendanhoar

Sys-usb in 4.0/4.1 is HVM.

And as far as I’m aware, under 4.0 and 4.1, VM’s that utilize sys-usb’s shared devices are PVH (aka PVHVM), not PV.

Brendan

@enmus

I meant on something like this .

@brendanhoar

Ah, ok. Right.

With R4.0 we saw use of PV mode being deprecated for IOMMU-less systems, but the installer allowed installing without IOMMU, and PV mode was the only way to passthrough PCI (less-safely).

However, R4.1 now requires a functioning IOMMU to install.

Are there that many USB controllers on systems with a functioning IOMMU that are incompatible with the IOMMU?**

B

** I shouldn’t be surprised if the answer is yes, I suppose.

@equbes

However, R4.1 now requires a functioning IOMMU to install.

Not quite true, the hard dependency was removed somewhere in the alpha iterations, you can proceed without IOMMU (the installer just show a warning)

However, sys-net / sys-usb will fail to launch as HVM without IOMMU, and if you try to change to PV, it will also fail to launch, because you need to explicitly enable this kernel flag: qubes.enable_insecure_pv_passthrough (commit)

So besides the less security of PV, for system without IOMMU at least they can enjoy other Qubes features.

[zithro]

Reading the docs you linked, I wanted to add my 2 cents, dunno if it’s useful or not in that case.
I have an AMD RX580 (no FLR support) passthrough’ed to a win7 domU, and rebooting it (or a crash) would cause the device to be in [b]locked state and couldn’t be re-assigned, even to the same host.
The sys bus removal trick wasn’t enough in my case, I had to also rescan the bus to recover the device, so:

echo "1" | tee -a /sys/bus/pci/devices/BDF/remove
echo "1" | tee -a /sys/bus/pci/rescan # this part is not in the docs

Unrelated to the PCI bus problem, but which may be of use about the “USB lock-ed out” problem, I found this post contains a method which could be useful. It’s about to PT devices sharing the same driver.
As I understand, the problem mainly happens on desktops, and if I’m correct, many platforms have two USB controllers (one on CPU, one on MoBo). With this method, you can PCI-PT one USB controller to a domU while keeping one for dom0. It’s what I use on my machine, and it works to PT an additional SATA controller to a domU too.

Edit: the link is about KVM and vfio-pci, but the steps are the same with Xen and pciback.

[zithro]

Just to add that PV support is also useful when doing nested virtualization.
HVM and PVH domUs won’t boot in a Qubes domU (but I’m still a noob with nesting so who knows).

[enmus]

It would be a shame PV to remain the only solution, but it’s +1 to my appeal, hahaha.

[zithro]

I agree ! ^^ PVH and HVM for the win.
Also, for various reasons, I don’t think Qubes should comletely remove functionnalities of Xen !
But on that topic I think I’ve understood there is/was a divergence even inside the Qubes team (read on the [Xen-devel] Xen 4.1.x security support discussion) : the eternal dilemna between security vs usability !