Docker support inside Windows virtual machine

General Discussion
1

Hello,

I consider buying one of your two computers that are officially supported by QubesOS (not sure which one yet). As far as I understand, QubesOS is hypervisor that enables you to run virtual machines. So I want to install Windows 10 Professional (with Microsoft Visual Studio for C# .NET development) as new virtual machine on QubesOS.

And my question is - will I be able to use Docker during my .NET Core development on the virtual machine, as it is possible if I install Windows directly on my machine? Or won’t it work because it would be running Docker from within virtualized machine?

Thank you!

2

I cannot say for sure this is not possible, i can advise I messed around with it for about a week, around 6 months ago in a windows 10 image…and Never managed to make it work :confused:

3

Thank you a lot! So what exactly have you tried? Is there anyone on this forum who could help with this (as soon as I buy my new laptop)?

I also wonder what my other options are. From what I understand, JetBrains Rider would work on Linux so I may consider buying it. But would it work (as far as Docker is concerned) on QubesOS? Would I need to install it directly on the hypervisor or would I be able to install LVM on QubesOS and then Rider on that new Light Virtual Machine (or ordinary Virtual Machine)? Have you ever used Rider (because I don’t know whether its capabilities are as good as in MS Visual Studio)? Any ideas other than Rider?

4

Maybe you can have the new computer set as dual boot: either Windows or Qubes. Windows VM has other limitations, like not been able to attach USB, sound, etc.

5

I would not advise doing this, especially for someone who comes from a windows background. It’s likely to cause a lot of frustration. And it’s also not advisable from the security perspective (depends on the threat mode, of course!)

https://www.qubes-os.org/doc/multiboot/

6

I should note that on linux VMs I’ve been able to use docker just fine. So I’d say there should be no problem with a Windows VM as well. But I am no virtualization expert.

7

As far as I can tell, I don’t believe “Docker for Windows” will work on an HVM Windows qube without workarounds.

  1. Docker on Windows uses virtualization/Hyper-V.
  2. Qubes OS does not support nested virtualization.

You mention .NET containers - the last I remember .NET was cross-platform and could run on Linux platforms. This would remove the need for Windows altogether unless you are specifically attempting to run Windows-platform containers or require Windows for your IDE.

There is no problem running (linux) Docker containers in a PVH Fedora/Debian qube on Qubes OS, as this is all handled by the linux kernel with namespaces/cgroups. Docker on Windows (and macOS) requires virtualization support because it’s essentially booting a linux VM and doing operations within that VM.

If you don’t really need Windows, installing on Docker in Qubes OS is as simple as:

  1. Clone one of your templates (eg debian-10) into a standalone qube, call it debian-docker

    You can use Qubes Manager (create qube, option: standalone cloned from template), or:

    qvm-clone --class StandaloneVM debian-10 debian-docker

  2. Boot the debian-docker qube, install Docker and any necessary dependencies

  3. You now have Docker on Qubes OS and can do all your Docker things within that qube.

Alternative: clone template (but leave it as a template, not a standalone), install Docker. Then create AppVMs and make the various storage directories for Docker a bind-dir such that they are stored persistently with the AppVM. Now you can create AppVM qubes from debian-docker and essentially “start fresh” without contaminating different projects.

From a quick search, here’s a gist of someone doing this on R3.2 - https://gist.github.com/xahare/6b47526354a92f290aecd17e12108353

The above assumes you have no explicit need for Windows.

1 Like
8

Thank you!

That’s true, with .NET Core and .NET 5.0 you can develop multi-platform, for Windows and Linux. But unfortunately Visual Studio (IDE) is available only for Windows. For Linux you have simplified VS Code which is not enough.

Do you see what kind of workarounds might work on Qubes to make it possible to use Docker on an HVM Windows? Most likely those Docker VMs would be things like Redis cache or SQL Database, but there may be others. That would be writing docker file and executing it to get the correct images, make changes on those and run them.

9

If you create a Windows 10 VM with newer builds they will come with the ability to run WSL2. The newer Docker Desktop for Windows installers will default to using WSL2 and does not require Hyper-V unless you intend to run Windows-based Docker images. Install Windows 10, perform all updates, install WSL2 and the newest Kernel, install Docker Desktop, you might have some luck. I’ll give it a try when I have some time. I’m only using the Win7 VMs because I wanted to keep seemless mode, but this might be worth a try.

10

@newqubesosuser

The workarounds involve enabling nested virtualization so that Hyper-V is running in your HVM qube and can create its own VMs.

I haven’t tried this with Windows/Docker, but a quick way to enable nested virtualization:

  1. create a file (as root) in dom0:

    /etc/qubes/templates/libvirt/xen/by-name/[name of your windows hvm qube].xml

  2. Override the default CPU feature flags so that it passes your CPUs virtualization support to the guest (default is to explicitly disable):

    {% extends 'libvirt/xen.xml' %}
{% block cpu %}
<cpu mode='host-passthrough'>
    <feature name='svm' policy='optional' />
    <feature name='vmx' policy='optional' />
</cpu>
{% endblock %}

Doing this on a Fedora 32 HVM spit out stuff about nested virtualization by kvm in dmesg, so that at least enables it. How functional it is, I haven’t tried or had a need to. Feel free to report results.

This has been discussed at https://github.com/QubesOS/qubes-issues/issues/4104 but according to that thread there were some crashes or lack of Xen support for it - perhaps that’s improved now (on R4.0, Xen 4.8.5-27, Fedora 32, 5.8.16 kernel).

@chrisbensch

I believe WSL 2 still uses virtualization at its core. It may not be the whole suite of Hyper-V functionality, but I presume somewhere along the line there are some VMX/SVM instructions being used. Whereas pre-WSL2 one would explicitly need Hyper-V, my speculation is WSL 2 uses the essential core of Hyper-V that performs VM management. In any event, please report if you have any success!

2 Likes