Nested virtualization

Does qubes support nested virtualization? I would like to ask, for example, I install a debian of hvm in qubes, and then run ubuntu in debian, has anyone tried this? Can it be successful?

Hi there,
are You running qubes on a server instead of a pc ?
Then, is theorically possible: How to enable nested virtualization in KVM :: Fedora Docs
(but honestly I don’t know if it’s something easy to do in qubes…)

If not, I strongly say is not something useful at all, if not for simply test purpose (and at what price?)!

Oracle (not last born software house), use containers on HOST, not in VM (as tanzu): Oracle® Linux Oracle Linux Container Services for use with Kubernetes User's Guide - Oracle® Linux Oracle Linux Container Services for use with Kubernetes User's Guide

Do You think they are not enough smart?
:sweat_smile:

Cheers,
M.

My host is qubes and I want to try nested virtualization

well, of course bare metal virtualization is the best!
Cheers,
M.

There already were several attempts:

https://fishilico.github.io/generic-config/sysadmin/qubes-os.html#nested-virtualization

Though none of them worked for me. Still, maybe you’ll find them useful

You misunderstood what I mean, qubes is still bare metal, but after opening a virtual machine in qubes, a virtual machine is opened in the virtual machine

Yep,
I know well what You where saying…
nested virtualization upon a bare metal virtualization, is different from a kernel zone based one…
From my point of view not really, because resource used by second one are slightly higher than other one… and that’s the reason I asked You on which hw You’d like to use it… but it’s not really important now…
aronowski has kindly pointed You to some other attempts done on qubes, so: good luck!
:slight_smile:
Cheers,
M.

Nested virtualization is considered both experimental in Xen as well as a potential security liability, and is therefore not supported in Qubes.

B

1 Like

I think they are asking about running a VM in Qubes, not running Qubes in a VM.

You can run Qubes in a VM if it’s configured to allow nested virtualization, but it doesn’t work the other way.

In xen.xml the host-passthough for vmx and svm is disabled, maybe it would work if you enabled it. It’s probably not a good idea to change the settings, it could make it pointless to use Qubes at all.

What I want to ask is to install vm in qubes to achieve nesting, rather than installing qubes in a virtual machine. Several friends above misunderstood what I mean. According to the official documentation of xen, I found that nested virtualization has recently become is disabled in several versions, is it true? Is there a way to enable it?

1 Like

This could help you. Read before accusing me I didn’t understand you too :wink:

Hello, I don’t mean to accuse you, I just clarify it again, so that everyone can discuss the same goal smoothly

With the current version of Xen used by Qubes, including newer Xen versions available not yet used by Qubes, there is little to no hope for reliably running a VM in a VM on a Qubes system.

B

<!-- disable nested HVM -->
<feature name='vmx' policy='disable'/>
<feature name='svm' policy='disable'/>

You probably need to start with enabling that part of xen.xml

Keep in mind, it’s disabled for a reason.

So, what are the results ? I’m curious !

I know that you want to nest domUs from within Qubes, but here are my notes on my attempts to run Qubes from an existing non-Qubes dom0 (like running Qubes from AWS), which when you think of it is pretty similar :

  • the nested dom0, aka L1-dom0 (so the Debian HVM in your case), must have “nestedhvm=1” and “hap=1” in the config file
  • in my case, only PV nested domUs (your Ubuntu) work. HVM = 100% CPU usage + “VMEXIT” errors filling up L1-dom0 xl dmesg ; PVHVM = crash on boot

I tried in march though so maybe there are news on this, will retry.
I also have to investigate the linked posts, and the vmx/svm thingy.

But again, I would gladly like to read about your findings !

1 Like