Building vmm-xen with VirtIO-GPU

User Support Testing Release
1

First, I understand that that what I am trying to do will break some of the security of Qubes. My personal use case for Qubes is to have a bare-metal client hypervisor with enhanced security for (mostly) development purposes. As such, I am willing to accept some risks. That is why I posted this in “Testing.”

From what I understand, VirtIO-GPU was added to Xen in either 4.15 or 4.17, and if the VMM and guest kernels are compiled with VirtIO support, the guest can use the host GPU for rendering. Xen 4.17 is the default in Qubes 4.2rc4, so I am using that as a base OS. However, documentation is sparse, so I wondered if anyone had tried this or could give me some help.

Without GPU acceleration, Qubes taxes the host system when using multiple monitors, and as a developer, this is an issue for me. So, I followed the Qubes builder instructions with the following changes:

  • ./builer.conf:
    • DIST_DOM0 ?= fc37
    • USE_QUBES_REPO_VERSION = 4.2
    • Comment out all components except vmm-xen and linux-kernel
  • ./qubes-src/vmm-xen/config:
    • CONFIG_VIRTIO=y
    • CONFIG_XEN_VIRTIO=y
  • ./qubes-src/linux-kernel/config-base:
    • CONFIG_DRM_VIRTIO_GPU=y

I then ran make vmm-xen linux-kernel. The next steps in my plan would be:

  • Package up the vmm-xen and linux-kernel files and install them in dom0
  • Add the following kernel params to /etc/default/grub and rebuid with grub2-mkconfig (this may not be necessary? I found it on this site):
    iommu=pt intel_iommu=on pcie_acs_override=downstream,multifunction
  • Change my VM from PVH to PV (PV seems to be broken on 4.2rc4?)
  • In dom0, run:
    qvm-prefs -s [test-vm-name] kernelopts 'device=virtio-gpu-device'

I don’t think that final step is correct - I’m pretty sure the device argument is supposed to be passed to Xen, xl, virt, or whatever command Qubes uses to launch VMs, but I’m not sure where to modify that.

Any thoughts/feedback/suggestions? What am I missing, and what else would be required to get this to work? If not . . . I may have to leave Qubes after 8 years and homebrew a KVM system to work somewhat like Qubes so I can use my GPU/multiple monitors.

For reference, some basic system specs:

  • Dell Precision M6800
  • CPU: i7-4910MQ
  • RAM: 32GB
  • GPU: AMD FirePro M6100*

*I THOUGHT I might be able to get this to work as a passthrough device to a GUI Domain VM (this isn’t a consumer-grade GPU, and I think it is supposed to be supported by Xen). Maybe I need to rebuild my kernel with support for this card? If I can get passthrough to work, I’ll just use that, instead!

3 Likes
2

I’m still trying to get this to work, but I’m not sure how to add/enable different build configurations for Xen. I tried editing ./qubes-src/vmm-xen/config, but that always causes the compile process to error out. How can I build vmm-xen but add in compile flags like CONFIG_VIRTIO?

3

I do not think Xen project Mailing List got into Xen 4.17.2, however it’s indeed merged in Xen 4.18 rc4.

4

Ah, I see. I saw a lot of patches and discussion on their mailing lists regarding VirtIO, and I thought they introduced it in 4.17. This could be really helpful for Qubes users.

Sad - it doesn’t look like the Qubes builder has been updated to support 4.18 rc4. I suppose I can take a crack at it if I get the time. I’ll report back here if I manage to get something to work.

1 Like
5

Maybe…better to backport that series of patches instead of trying to upgrade xen? Since I think there are related libraries and packages that also need to be upgraded when you bump xen’s version.