I’m stuck in the “attach that to qemu (via libvirt xml)” part, could you please give an example, when I tried using input type passtrough in libvirt as the example above it still says it’s missing the evdev path despite /dev/input/device0 being present in stubdom (I’ve created a dummy to be used until I run: sudo qrexec-client -d example-dm -l 'input-proxy-sender /dev/input/device*' root:'input-proxy-receiver --keyboard')
I am running this all from dom0 only because that is where my keyboard is attached, but the input proxy was designed to run from sys-usb. Ideally I would use sys-usb.
I modified the package but am only copying the rootfs to dom0. I followed the qubes builder page, had it download some sources so I didn’t have to build everything, and then ran make vmm-xen-stubdom-linux. I rebuilt it with CONFIG_INPUT_EVDEV and I do see the correct event device in /dev/input now. Let me know if you have more questions about this part, I can post what I did exactly.
How and at what time during the stub domain boot up are you creating the dummy device? I do not see how to call any qrexec commands before qemu has started in the stub domain. I imagine qemu is trying to use that dummy input before you have created it.
Sorry for taking so long to reply, I was hoping I would figure it out and than tell you the good news however that’s not the case.
The way I was creating a dummy device wasn’t the proper way of doing it as it was not a valid input, device I was just creating a file named event0 in /dev/input.
To use sys-usb we can just send the input to dom0 and use input-proxy-sender the same way.
Something that I’d like to try but I don’t know how to do is to create a valid dummy device so that qemu picks it up and then redirect the “real” input to the dummy
So far I’ve added the following to the rootfs init file:
evdev_args=$‘-object\ninput-linux,id=kbd1,evdev=/dev/input/event0,grab_all=on,repeat=on’
and $evdev_args to qemu just like $dm_args.
But since you can’t call qrexec commands before qemu starts; It won’t boot unless it gets a valid input device that’s why I’d like to create a fake input that receives the real input if that makes any sense (like a symlink).
I just realized you mentioned forwarding the input that’s exactly what I was trying to say have you made any progress in that?
I have found a solution that works but definitely needs to be cleaned up. The performance is great though. For example my use of mkfifo and cat feels very hacky and I imagine there is a better way using file descriptors (like in the init file). Anyways, here is the full setup:
0. Proceed with caution!
Make sure to make a backup copy of qemu-stubdom-linux-full-kernel and qemu-stubdom-linux-full-rootfs in /usr/libexec/xen/boot/ so you can always go back to a working system.
I do not know about the security concerns of this setup.
1. Modifying the stub domain kernel
Using Qubes builder I remade qemu-stubdom-linux-full-kernel by adding a file I called 60-input:
CONFIG_INPUT=y
CONFIG_INPUT_MISC=y
CONFIG_INPUT_UINPUT=y
CONFIG_INPUT_EVDEV=y
Then copy the kernel to /usr/libexec/xen/boot/
2. Modifying the stub domain rootfs
As stated above follow Neowutran’s gpu guide. For this you don’t need to do the “gpu_” modification but its the same steps for editing the init file. Be sure to edit the full version of stubdomain-linux-full for the kernel and rootfs
mkdir stubroot
cp /usr/libexec/xen/boot/qemu-stubdom-linux-full-rootfs stubroot/qemu-stubdom-linux-full-rootfs.gz
cd stubroot
gunzip qemu-stubdom-linux-full-rootfs.gz
cpio -i -d -H newc --no-absolute-filenames < qemu-stubdom-linux-full-rootfs
rm qemu-stubdom-linux-full-rootfs
edit init just after the usb_args= section to have:
# Setup input proxy devices
mkfifo /tmp/proxykeyboard
mkfifo /tmp/proxymouse
{ cat <>/tmp/proxykeyboard & } | USER=root input-proxy-receiver --keyboard &
USER=root cat /bin/kbhello.txt > /tmp/proxykeyboard &
{ cat <>/tmp/proxymouse & } | USER=root input-proxy-receiver --mouse &
USER=root cat /bin/mhello.txt > /tmp/proxymouse &
evdev_args=$'-object\ninput-linux,id=kbd1,evdev=/dev/input/event1,grab_all=on,repeat=on,\n-object\ninput-linux,id=mou1,evdev=/dev/input/event0,'
Note: You may have to swap event0 and event1 depending on how creates the devices. One could probably use the --name feature for input-proxy-receiver to find the correct event id.
Add $evdev_args to the qemu command at the end of init so it looks like:
qemu -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-chardev pipe,path=/tmp/qmp/qemu,id=m -mon chardev=m,mode=control \
-chardev socket,server,nowait,path=/tmp/qemu.qmp,id=m2 -mon chardev=m2,mode=control \
$dm_args $evdev_args $pa_args $usb_args $kernel &
Input-proxy-reciever needs some info before it creates the uinput device. We need to pass it to it via kbhello.txt and mhello.txt. You can name this whatever you want. To create it run this in dom0 or wherever your keyboard/mouse is located. Immediately stop the program running before moving your mouse or pressing any other keys via CTRL+C:
input-proxy-sender /dev/input/by-id/mykeyboard > kbhello.txt
Repeat for a mouse. Place these files in rootfs/bin or somewhere else.
Copy input-proxy-reciever to the bin folder in stubroot (I just copied this from dom0’s bin. This should probably be compiled in step 1 though).
Then finally put it back together and replace the original rootfs:
find . -print0 | cpio --null -ov \
--format=newc | gzip -9 > ../qemu-stubdom-linux-full-rootfs
sudo mv ../qemu-stubdom-linux-full-rootfs /usr/libexec/xen/boot/
3. Pass a mouse and keyboard to the stub domain
A mouse and keyboard can be passed to the stub domain of the HVM you want via this script. My hvm is called gpu_win10 and it’s stub domain is then gpu_win10-dm.
#!/bin/bash
qrexec-client -d gpu_win10-dm -l 'input-proxy-sender /dev/input/by-id/guest-mouse' root:'cat | cat > /tmp/proxymouse' &
qrexec-client -d gpu_win10-dm -l 'input-proxy-sender /dev/input/by-id/guest-keyboard' root:'cat | cat > /tmp/proxykeyboard' &
My mouse and keyboard are called guest-mouse and guest-keyboard because I am using GitHub - mciupak/libvirt-evdev: Libvirt evdev input service to split my keyboard into two devices: a host and guest device. By passing just the guest to gpu_win10-dm I can now switch back and forth between dom0 and gpu_win10 when I press my scroll lock key.
4. Final Steps and Notes
By default the qemu-stubdom-linux-full-* versions will not be used. In my setup I have my windows hvm qvm-features set to:
audio-model ich6 (this definitely forces use of full)
stubdom-qrexec 1 (this should force usage of full)
gui-emulated 0
gui 0
video-model none
By setting this I also disable input from qubes_drv which does not place nicely with this setup. (see comments below)
I also experience an issue where if too many keys are pressed while in guest mode, the extra keys are sent to the host keyboard which then dom0 reads.
Thank you for the help @h5409j. I will try to reply faster to any issues anyone has or errors I have made.
4 Likes
@khmartinsen Thank you for taking the time to write a reply explaining how you did it it’s really helpful, but unfortunately I wasn’t able to follow it. I think I’m messing up in the edit init just after the usb_args= seaction to have: part.
Is it supposed to look like this? (assuming I only want to pass the keyboard)
usb_args=
if test -e /bin/qrexec-agent; then
usb_args=$'-device\nnec-usb-xhci,id=xhci'
mkdir -p /var/run/qubes
USER=root qrexec-agent &
touch /dev/mdev.log
mdev -d
fi
mkfifo /tmp/proxykeyboard
{ cat <>/tmp/proxykeyboard & } | USER=root input-proxy-receiver --keyboard &
USER=root cat /bin/kbhello.txt > /tmp/proxykeyboard &
evdev_args=$'-object\ninput-linux,id=kbd1,evdev=/dev/input/event0,grab_all=on,repeat=on'
The reason I’m asking is because I still can’t run any qrexec-client commands before it tries to pick the input device to use, making it unable to boot
I forgot a major part and I will go back and add it to that post in a bit.
This line: USER=root cat /bin/kbhello.txt > /tmp/proxykeyboard & is referring to creating a file called kbhello.txt from running this command in dom0. Be sure to to type anything and then exiting the command. It may need sudo.
input-proxy-sender /dev/input/by-id/mykeyboard > kbhello.txt
Put this in bin or another folder in the rootfs.
Once you have all this setup you start your hvm then run the script in step 3.
1 Like
Thank you so much it works!
1 Like
@khmartinsen one last thing, do you happen to know what causes the mouse to move to the top left whenever I right click or left click?
I had this issue and I cannot remember what I did to be honest. Ill have to look and see if I omitted something. Make sure your keyboard and mouse are not swapped in the event0 event1 part of the stub domain. Also make sure you are passing the right device in dom0.
I’ve checked and they’re not being swapped and they’re the right events, the mouse works fine the clicks also work but when I click it teleports to the top left, did you change anything in the libvirt config? This might have to do with the qubes gui? The only thing I’ve done is set the guivm to none and pass the gpu, also when I pass the same event to sys-gui it works fine
@khmartinsen Turns out the mouse doesn’t only go to top left whenever I click or scroll, it teleports to whatever place the “primary” mouse input is. For example if I move the mouse with the window emulated mouse on dom0 (the default way) and than use the evdev mouse that QEMU uses and move it and click or scroll, it goes back to whatever position the other mouse is.
I think having a look at how you set up your input devices on the libvirt config would help but I’m not sure, hopefully you can tell what’s going on based on what I tried to explain. (I’m awful at explaining things)
That is exactly the issue I had. Its from the qubes_drv. In my windows 10 vm I don’t need the emulated window that qubes provides so I disable that by using qvm-features. I am not sure if all of these need to be turned off but i the following set:
gui 0
gui-emulated 0
video-model none
I don’t think you would want to change anything in libvirt but if you did you would want to remove the tablet input I think. Maybe in the VM you can disable that input device that is related to the qubes gui window tablet inputs.
1 Like
@khmartinsen Thank you simply removing tablet input solves the problem in dom0, for some reason the same setup while using sys-gui doesn’t.
Unfortunately I can’t remove the emulated window because I need to force MSI on my NVIDIA graphics card in Windows otherwise the display glitches and blacks out (making it unusable) and the VM eventually shutdowns.
I’ve been booting Windows in safe mode and adding a registry entry every time the address changes, but if I change the features you mentioned I won’t be able to force MSI because the GPU address will be different and I won’t be able to change it.
Do you know of an alternative to set my GPU to use MSI by default?
On dom0 lspci -v shows that MSI is disabled:
Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
Would that capability even be passed to the VM?
I am not sure. I have not used sys-gui yet. I would think there would be some way to just disable the tablet input in whatever VM you are in. So in sys-gui can you disable the tablet input by identifying which device it is in /dev/input/…?
Shouldn’t removing <input type='tablet' bus='usb'/> from the VM xml be enough? I find it strange that the outcome isn’t the same as it is in dom0, I should mention that it doesn’t display anything on the screen I think it’s related to the MSI shenanigans, I’ll double check
I would think that is enough. Which xml are you editing? The ones located in /etc/libvirt/libxl/ are auto-generated. You need to follow this: Custom libvirt config — core-admin v4.2.7-0-gf40e8c21-dirty documentation.
It looks like I had messed up somewhere else, it wasn’t related to any of your fixes I just had to reinstall the NVIDIA drivers and manually enable MSI again, not really sure how that happened but anyhow it’s working now and the performance seems better than with the usb proxy but that might be placebo, thank you for your patience and for your time!
1 Like
Can you elaborate on step 1?
Specifically:
- Where does the 60-input file go in the directory structure?
- What build target(s) did you use, or just ‘make qubes’ to make everything?
Also, can you explain what you’re doing with all the cat-ing?
Thanks for this guide! 