Windows 10 Questions

Oof. Have any more light reading you can recommend? :stuck_out_tongue:

Yes, I read the custom libvirt domain config doc, and the accompanying Jinja template documentation.

Well either devs come here and tell you how it’s done (wich is unlikely) or you find that out by yourself in sources (which I think will be faster).

So why don’t you just inherit from the original template and override the devices block as stated there?

The better way is to inherit from the original template and override any number of blocks. This is the point when we need the name of the original template.

{% extends 'libvirt/xen.xml' %}
{% block devices %}
    {{ super() }}
    <serial type='pty'>
        <target port='0'/>
    </serial>
{% endblock %}

{% extends %} specifies which template we inherit from. Then you may put any block by putting new content inside {% block %}{% endblock %}. {{ super() }} is substituted with original content of the block as specified in the parent template. Untouched blocks remain as they were.

Like this:

/etc/qubes/templates/libvirt/xen/by-name/YourWindowsVM.xml

{% extends 'libvirt/xen.xml' %}
{% block devices %}
    {{ super() }}
    <disk type='block' device='disk'>
      <driver name='phy' type='raw'/>
      <source dev='/some/device/path'/>
      <target dev='xvde' bus='xen'/>
    </disk>
{% endblock %}

Yeah, I know. Just adding a little levity. Though I don’t know if it would be any faster… there’s a LOT of code to read. XD

Yeah, I know. That’s what I am planning on doing. :slight_smile:

The way I work on these things is to try and understand how it works the best I can by reading the docs first, and anything that sticks out as a potential problem I ask questions about first. I am a bit leery of just “Try it and see!” when it comes to VMs because I don’t want to break something that will cause many more hours of work to get back to a known good current state. Plus, I want to try and reduce the problem space a bit up-front.

Yes, that’s sorta what I am doing, but 1) I don’t want to do {{ super() }} because I don’t want the pregenerated disk volumes on this VM, and 2) based on my current readings, will it do something like this:

>      <driver name='phy' type='raw' type='raw'/>
>      <source dev='/some/device/path'/>
>      <target dev='xvde' bus='xen' bus='xen'/>

or something stranger because I haven’t grokked the entirety of the xml generation process.

Basically, I don’t want the VM to launch with a bad config, since Windows is notoriously bad about clobbering stuff if it isn’t what it expects (it tries to ‘make it right’). So, I either would like to do “dry runs” where it generates the final xml, but doesn’t launch the vm, or I need to do “config proofing”, which is doing due diligence up-front to make sure the configs are as correct as possible before launch. Part of the process of doing the latter is making sure I understand the best I can what is happening by reducing/correcting issues or gaps in my understanding I see beforehand.

If the only way to do that is to keep studying the process with increasing depth until I get to that point, so be it. I am asking here in the hopes that someone might have a better understanding of what I am missing so that I can narrow the time window a little bit.

Anyway, I will keep on studying it. I DO appreciate your help, though. :slight_smile:

You can just create test AppVM and use it to generate test configs before you copy verified config to your Windows VM config.

If I pass it the real disks, will it futz with them? I don’t want it to write over anything.

You can create empty StandaloneVM not based on template.

Yeah, that’s what I am using to build my Win10 VM, but won’t it try to boot when I “start” it to generate the vm xml? Again, I don’t want it to try booting until I can verify the vm xml.

It depends on what will you change in config. The default empty StandaloneVM will start but will hung since no bootable device will be found.

Isn’t it possible with LVM to clone a dataset, without taking any disk space except “differences” (sorry ZFS lingo) ?
This way you could make your tests with a “fake” dataset, without messing your existing Win filesystem.

Hmm. maybe. There’s a method to mount a lvm filesystem as CoW, but I am not sure how to specify it in the config.

OK, I created a custom libvirt config which seems to generate a correct libvirt vm xml file, but the os doesn’t seem to boot.

I think it might be because the original vm was set up to boot using OVMF/UEFI, and has an EFI partition, but the stubdom is using hvmloader/fullvirt instead, which boots in CSM mode, basically. That definitely won’t work.

I am guessing if I switch it to booting UEFI using qvm-features it will likely cause OVMF hangs like I observed in my Xen attempts prior to Qubes. (That, and it is an unsupported feature, apparently)

Hmph. I guess I will just have to blow away that install and start over.

I am getting the impression that OVMF/UEFI BIOS is just too buggy to do VMs, which is sad. I would prefer to use it.

I tried setting my test windows install to boot UEFI using “qvm-features gpu_base-win10 UEFI 1”, but all it seemed to do was change loader type to “pflash” without changing anything else, which of course won’t work. I fiddled around a bit, trying and taking out , and the test vm booted like it has been, but it seems to have been a trick. Nothing indicated that it loaded efi at all. From the libvirt doc, it says that firmware attribute is just a signal to a later parsing state to put in proper and tags.

I tried to put in explicit and tags to load edk2-v86_64-code.fd and edk2-i386-data.fd from the /usr/share/qemu-xen/qemu folder, but it hung as well.

I doesn’t seem UEFI support in VMs is working at all at this point. Eh well.

Nah, I was saying, create a clone/duplicate of your new working install, and then copy over your original W10 install ? And I mean only the OS partition, not the EFI and System Reserved ones.

Was that using a stubdom too ? Check the “xen-devel” mailing list, maybe that was discussed about,
and maybe there’s a patch/fix using recent versions ?

Then a few pointers, and sorry I’m a libvirt noob, so it will apply to Xen cfg files, not libvirt XML files. You’re on your own to translate cfg->xml ; )
I’ve also lost track a bit of all your tests, and what’s working what’s not ^^

  • Maybe you could read the Xenstore (xenstore-ls from dom0) to compare your working and non-working qubes ?
  • Xen uses firmware="uefi" not efi
  • Try different UEFI versions via bios_path_override=“PATH” ?
  • Xen has passthrough="sync_pt|share_pt", but it’s fiddling, I dunno what I’m talking about ; )

Well, it doesn’t matter now. I blew away the Win10 installation on my win10 volume group, and have now been trying to get it to reinstall clean on it. So far, I have a Windows install set up on a secondary pool. Still tons of quirks I am working through (cdrom with QWT not attaching, user issues with qvm-run, etc), but it is progressing again.

This entire situation has been infuriating and fatiguing. All I wanted to do was to install a VM on a separate, dedicated volume group on a dedicated physical device. I DID NOT want it thin-provisioned in a pool volume group, but I just accept that that’s the only way it’s going to work.

I named my Windows user Talarus, and it seems like qvm-run hates that, because it is still trying to log on as “user” to run commands, even if I pass -u Talarus on the command line. I had to create a separate user account called “user” so it could run. Otherwise, the wait-for-logon qrexec-wrapper utility hangs until I terminate it with task manager, and then the command will run finally.

Anyway, I’m tired and just wanted to give an update. I’ll continue this menagerie tomorrow. -.-

Did you change default_user?

Lastly to enable file copy operations to a Windows 10 VM the default_user property should be set the <username> that you use to login to the Windows VM. This can be done via the following command on a dom0 terminal: (where <VMname> is the name of your Windows 10 VM)

qvm-prefs <VMname> default_user <username>

Contents/docs/os/windows/windows-tools.md at master · Qubes-Community/Contents · GitHub

Ahh. OK. I will try that and see. Thanks!

In the meantime, I have been wrapping up PCI and USB passthrough. The only thing left to do is get the mouse and keyboard sorted. I think I may use a PS/2 mouse/kbd combo to use on dom0 until I get the info from khmartinsen on the other thread. I still would rather pass them through as I want to run the keyboard/mouse software so I can program them easier. I would like a way to hot-swap usb devices to different VMs by a keystroke, but that would take some serious magic. Just sharing the evdev will work for movement and keystrokes, but probably won’t support the programming software.

I also did some lvm housekeeping, getting rid of the old CentOS/Xen install and freeing up and absorbing that space into my qubes_dom0 volume. lvm tools are very good, even allowing online reconfiguration and movement of PVs. Was pretty pleased with the results.

I’ll be putting the Windows vm through some serious tests over the next week. Hopefully, I won’t encounter any crashes or corruption issues.

I have an alternate idea for keyboard/mouse passthrough.

Basically, I create a script which starts the windows VM using qvm-start, and after a short delay, uses qvm-usb attach to attach the keyboard/mouse to the windows VM. Then, while I am in the Windows VM, I can use some kind of qrexec call back to dom0 to trigger a second script which uses qvm-usb detach, and then whatever call I need to make to sys-usb to start up the input proxy senders again, as if the USB device has been hot-replugged. Then I can have a shorter version of the startup script to re-attach the mouse/keyboard to the Windows VM again.

The things I need to know are:

  1. What command(s) do I need to use to get sys-usb to re-connect the qubes-input-proxy-senders back to dom0?
  2. How can I use the vm inter-process communication via qrexec to tell dom0 to run the detach script?

That should solve the problem a lot better than the kbd-splitting hack. I mean, I would like to hotkey it, but that can be something later to make it “nicer”. I don’t mind getting dom0 to run a script for me to detach and I don’t mind starting the domain with a script or reattaching with a script (though… is there a qube startup/shutdown script hook?).

I think I figured out how to solve #1.

In the sys-usb qube:
sudo udevadm trigger --action=add --property-match=DEVNAME=/dev/input/event6

That reconnects the mouse to dom0 after a qvm-usb detach. The only thing I need to do is to make a smarter script which determines the best property-match value to make sure it gets the mouse every time, and also I need to be able to trigger that using qrexec-client-vm from dom0.

I think I have an idea to do #2 now as well.

OK, I had to change the reset script in sys-usb. Apparently the udevadm trick was a one-off. It fails to work if the mouse device did not get moved back from usbip-host driver to the usb driver. One time it did, but it didn’t trigger the input-proxy-sender events to reconstruct the dom0 bridge, which is why I had to go that route.

Now, the best way to do it is to unbind the device from whatever driver it is bound to and then rebind it back to the usb driver. Doing so should trigger the input-proxy-sender setup scripts. If it doesn’t I suppose I could check to see if they are missing and run the udevadm part to force them to run and reconnect to dom0.

I have now made all the scripts which allow me to attach/detach/reset the keyboard/mouse to and from a Windows Qube back to dom0. The last part I am missing is the piece to get qrexec-client-vm.exe in Windows to tell the RPC endpoint in dom0 to detach the keyboard/mouse, and then pass a reset RPC message to sys-usb to rebind and attach the input proxies for dom0. The scripts are in all three vms, just gotta get this RPC thing done.

When I make the call in windows I call it like this:
“C:\Program Files\Invisible Things Lab\Qubes Tools\qrexec-client-vm.exe” dom0 qubes.KVMDetach C:\Programs\qubesrpc\qKVMDetachClient.bat" Win10Qube

The program runs, but immediately returns. With the logging level turned up to 5, I get:
[time/date-process-D] ErrRegisterUEF: done
[time/date-process-E] wmain: Usage: qrexec-client-vm domain name|qrexec service name|local program [local program arguments]

I have created the qubes.KVMDetach endpoint in the /etc/qubes-rpc folder and create a policy file for it in the policy subfolder. The endpoint is a script and it is set to +x, but I don’t think we’re even getting out of the Windows Qube at this point.

I don’t think it is doing anything and it doesn’t seem to give much in the way of debug info. Is there any specific doc on the Windows qrexec-client-vm program which might inform me on its proper setup and usage on Windows?