Why does seamless GPU acceleration only work when using template based HVMs?

General Discussion
,
1

Here is a screenshot of Cyberpunk 2077 running in window mode with full GPU acceleration, I only get 25 FPS on high settings but the GPU I use is only a 4060.

cyberpunk
cyberpunk1280×720 209 KB

Why does this only work when I use a HVM based on a template?

If I use any other type of HVM I can’t get it to work, I tried different Linux ISO/distributions, different nvidia drivers, I tried running the HVM appVM in debug mode with gui-agent removed, nothing works.

Is there any way I can do the same with a HVM that is exposing the entire desktop?

Most/all games that can run with Wine+Proton seem to work, but many games don’t work well with the Qubes OS virtual screen and input, which is why I’m trying to make it work with a HVM running a full desktop.

2

You don’t see the HVM desktop after this or you see it but GPU acceleration is not working?
If you don’t see the desktop then did you set the default target after removing qubes-gui-agent?

sudo systemctl set-default graphical.target

You can also try to use VNC:

1 Like
3

I see the desktop, and the GPU get detected, but the performance is a lot worse.

I’m guessing some driver stops working as soon as I stop using the virtual display driver that only shows the application and not the desktop.

I’ll try using the VNC and see if that works.

4

VNC partly works, the GPU acceleration works, and it allows games to use native full screen. Sadly, it seems to have issues if the screen/view port gets locked to the mouse.

5

I don’t understand what issue do you have from your description, but if it’s something related to the mouse input then you can try this:
Qubes redirect mouse input to VM · GitHub

6

Wait what…? GPU accelewho? When did y’all start gaming in qubes with GPUs and didn’t invite me??? :skull:

7

In some games, free camera motion is done by locking the mouse to the center of the screen, and running a continues loop that reads mouse movement.

In some games when using the VNC, the mouse doesn’t get locked or the position of the mouse is not read correctly, and the result is that the camera spins uncontrollably when you move the mouse.

The input proxy doesn’t work and attaching a USB controller also doesn’t work, but that could be a configuration issue in the VM and the input are not working in display localhost:1

8

I’ve tried it and it worked for me. Try like this:

export VGL_DISPLAY=egl
vglrun glxinfo | grep render

Also this worked for me, I’ve tested it in minetest and run this command in dom0 to attach the mouse from sys-usb to my gaming qube:

qvm-run -u root --pass-io --localcmd="qvm-run -u root --pass-io sys-usb \"input-proxy-sender /dev/input/by-id/usb-MY_MOUSE_ID-event-mouse\"" MyGamingQube "input-proxy-receiver --mouse"
1 Like
9

I’ve tested it some more and I’ve found the problem that you’ve encountered, it was Xorg. Using Wayland don’t have this performance drop.
Here is an example with sway, but it should work for KDE/GNOME/etc with Wayland if you prefer it:
Install sway in the template (e.g. sudo apt install sway xwayland for debian).
In the gaming app qube based on this template create the custom sway config:

mkdir -p ~/.config/sway
cp /etc/sway/config ~/.config/sway/
nano ~/.config/sway/config
echo "seat * pointer_constraint disable" >> ~/.config/sway/config

Attach mouse to the gaming qube by running this command in dom0 or attaching the USB mouse directly to the qube:

qvm-run -u root --pass-io --localcmd="qvm-run -u root --pass-io sys-usb \"input-proxy-sender /dev/input/by-id/usb-MY_MOUSE_ID-event-mouse\"" MyGamingQube "input-proxy-receiver --mouse"

Start sway in the gaming qube’s terminal:

WLR_NO_HARDWARE_CURSORS=1 sway --unsupported-gpu

Run the game inside the sway.

10

I spoke too soon, the mouse is not working properly with Qubes GUI agent.
The seat * pointer_constraint disable stops the mouse from spinning but it’s limiting the mouse movement.
Also passing the mouse input using input-proxy seems to only work in Xorg, it’s not working in Wayland so you’ll need to pass the USB mouse to the qube.
Running sway without Qubes GUI agent in the HVM stubdomain window works fine:

Enable debug mode for gaming qube and gaming qube’s template in dom0:

qvm-prefs MyGamingQube debug 1
qvm-prefs MyGamingQubeTemplate debug 1

Increase the gaming qube VRAM for stubdom, the default value didn’t work for 1920x1080x60Hz mode:
Create this file in dom0:

nano /usr/share/qubes/templates/libvirt/xen-user.xml

And add this content to it:

{% extends 'libvirt/xen.xml' %}
{% block devices %}
        {{ super() }}
            {% if vm.features.get('increased_vram', '0') == '1' -%}
                <video>
                    <model type='vga' vram='32768' heads='1' primary='yes'/>
                </video>
            {% endif -%}
{% endblock %}

And enable the increased_vram feature for your gaming qube in dom0:

qvm-feature MyGamingQube increased_vram 1

If you use kernel provided by qube then in gaming qube template add the missing kenel options compared to the kernel provided by dom0, e.g. for debian:
Add this line at the end of the /etc/default/grub in the template:

GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT rd_NO_PLYMOUTH rd.plymouth.enable=0 plymouth.enable=0 apparmor=1 security=apparmor"

And regenerate the GRUB config, e.g. for debian:

sudo update-grub

Plymouth is messing up the HVM console so you need to disable it.

Remove the qubes-gui-agent from the template, e.g. for debian:

sudo apt remove --purge qubes-gui-agent

Shitdown the gaming qube’s template.
Enable debug mode for gaming qube:

Start gaming qube, login as user user and create a custom sway config:

mkdir -p ~/.config/sway
cp /etc/sway/config ~/.config/sway/
nano ~/.config/sway/config
cat << 'EOF' | tee -a ~/.config/sway/config
output Virtual-1 {
    mode 1920x1080@60.000Hz
}
EOF

You can change the display mode from 1920x1080@60.000Hz to another one.
You can check the available display modes by running this command inside a terminal in sway:

swaymsg -t get_outputs

Run sway:

sway --unsupported-gpu

Then run game in the sway.