Porting Qubes to hypervisors other than Xen (abstracting the functionality) - early stage

General Discussion
,
1

May we get Qubes developers interested on abstracting/decoupling Qubes great functionality to be applied over other hypervisors, please.

At this point in time (hype/buzz, 2024-01 Qubes 4.2 runs exclusively over Xen), what are the main challenges to this objective? What are the pros and cons? Do we have a case (and interest) for it, please?

Some requirements not met

  1. Xen does not provide hibernation
  2. Xen is not compatible with firmwares with wider than 16bit segment number, that do nvme calls to it during boot (could be solved with a hypercall sun-function)

I have explored topic #2 in xen-devel mailing list with logs, traces, journals and ring buffer evidences:

( Xen project Mailing List : No firmware ACIP and no boot )

Same discussion on StackExchange and a Taiwanese Hardware Manufacturer:
( unix.stackexchange DOT com/questions/710400/boot-into-xen-on-debian11-initrd-trouble )

( community.acer DOT com/en/discussion/669340/acer-aspire-5-a517-52g-firmware-w-16-bit-pci-segment-size/p1?new=1 )

  • Please, list more requirements not met.

A discussion on purpose and trade-offs

Having Qubes functionality over a hypervisor other than Xen, would:

  • Keep the user on the same interface across laptop computers, used for different purposes, knowingly with more security and less security.
  • Accelerate Qubes GUI tools development/coding.
  • Serve users which Use Case emphasise VM compartmentalisation over security. And, quick VM provisioning and use. And, educational/research purposes Use Cases. And, CLI-only and CLI first Use Cases.
  • Eventually offer interaction with VMs/Nodes/domains (QUEMU/KVN, QUEMU/GNS3).
  • ARM Use Cases (x86 issues and ME/PSP) ( A Secure and Formally Verified Linux KVM Hypervisor ) @fiftyfourthparallel @monocasa.
  • Increase the user base (in my point of view, this is a cons).
  • Please, list more pros and cons.
  • Please, list and detail alternative Qubes functionality Use Cases/User Profiles, and gather arguments for non-existence of such users, and make explicit (and well documented) the underlying tech limitations.

Useful resource: Who uses Qubes and Why discussion
( github DOT com/QubesOS/qubes-issues/issues/1906 )

From the Qubes FAQ ( www.qubes-os DOT org/faq/ ) :

Why does Qubes use Xen instead of KVM or some other hypervisor?
In short: we believe the Xen architecture allows for the creation of more secure systems (i.e. with a much smaller TCB, which translates to a smaller attack surface). We discuss this in much greater depth in our Architecture Specification document.

Possibly, Qubes focus has been anonymity and Whonix use. It seems that Qubes over a hypervisor other than Xen would invalidate the best use of Whonix.

( TODO Link Whonix - Xen discussion )

Challenges

It might be early to ask, but do we know how coupled/decoupled and documented are the hypervisors and Qubes functionalities, towards the integration of Qubes to a hypervisor different than Xen?

  • Identify most relevant challenges.

Possible Alternative path

Requirements could be met by successfully steering Xen into delivering laptop compatible functionality, on the dot. This would mean even more Qubes community involvement with the Xen community.

Apparently, Xen is focused on servers, not laptops (mentioned by someone on the discussion on xen-devel, link above).

Is there interest on abstracting Qubes functionality to be applied over other hypervisors?

1 Like
2

Someone (I forget who or where, the tab’s just been open my background for a while) recently posted a link to an old blog post about “Qubes Air”: Qubes Air: Generalizing the Qubes Architecture | Qubes OS

Much of it is focused on the cloud use-case (while still giving users control over exactly how much of their system is cloud-based, and supporting things like “local clouds” hooked up via LAN), but the section on “Qubes Zones” and the following section “Under the Hood” seem particularly relevant to your questions. They talk about some of the changes that need to happen/requirements that need to be implemented in order to use a different “hypervisor” (or some other equivalent technology, like cloud APIs).

Unfortunately, per Updates on Qubes Air? - #3 by fepitre, the QubesOS team does not have the bandwidth to actively work on the required architectural changes at the moment.

1 Like
3

Note: I am not a Qubes developer. I just have spent some time down in hypervisor spaces.

Does this matter? It provides (sort of, sometimes… on some hardware…) working standby. Hibernation seems high risk to me, as it requires full memory images stored in disk in a way that can be recovered on a cold reboot, which means at attacker with physical access can do so. I’d consider “lack of hibernation” a reasonable security requirement for something like Qubes.

Why? You can run whatever window manager you want for Dom0, or on your other computer.

Xen supports ARM. Just, nobody has ported Qubes to ARM yet.

The main question I don’t have an answer to is related to inter-VM memory mapping. Xen has some fairly straightforward ways to map guest physical regions into Dom0, and to map guest physical to guest physical, which I believe is used for a range of inter-VM data transfers. I don’t know KVM’s capabilities in that realm - if they had to be added, that would greatly increase the work and risk of adding KVM.

Huh? Whonix works fine with any virtualization solution. Though I agree that Xen being a smaller codebase is a good thing.

How many hypervisor hackers with lots of free time do we have? That’s the main limiting factor here. I don’t believe the Xen community is opposed to laptop related stuff, it’s just that they don’t care about it for their purposes. I’m fairly confident they’d take pull requests for such changes if they were well formed and reasonable.

I just don’t see what you gain from it. But neither am I opposed to it.

1 Like
4

Isn’t it the other way around?
In standby all the information is kept in RAM and susceptible to cold boot attack.
In hibernation the RAM image is stored in encrypted swap and is not accessible without the password.
So hibernation is more secure than standby.

5

Only if you use an older system, that doesn’t support memory encryption.

6

For Intel the TME feature is available only since 12th gen CPUs and only for models with Intel vPro® Enterprise. So even if you have latest 14th gen Intel CPU but without Intel vPro® Enterprise then you won’t get the memory encryption support.

7

Very interesting article on Qubes Air.

Indeed finding hardware is the main challenge (and it’s sad to lose hardware - therefore something like Qubes Air or a bridge would be helpful).

  • I love cheap hardware over one an expensive notebook - specially because my threat model involves going in places with little to no phisical security - it’s better to give out hardware than fight for it and lose own’s life. - In this case, a notebook with 16G and several mini desktops with 16G would be a better solution over a 64G notebook. But in this case, it would be better to have a seamless experience of using the computational resources.
    If Qubes supported other hypervisors, it could use computational resources of any mini desktop around on different levels of security.

  • Has anyone purchase a 17" or 18" machine that runs Qubes well, please?

8

Qubes os vs KVM on Linux(This is not war!)

Cons of Qubes os

Security by separation vms, Qubes os is most secure os.
Unless bug is exist on Xen, threats can not attack to dom0.
If user must copy data to dom0 from vm, this process is made very complexly, fail of user is been suppressed default.
Multi templates can use at the same time without dual boot(But Qubes os supports to Linux and Windrows only).
User can use disp-vm, all data on disp-vm are delete with shutdown, SSD usage suppresses.

Pros of Qubes OS

Qubes os eat a lot of RAM than KVM, it needs to larger RAM.
Qubes os is Xen distribution, so usability is difficult for light user.
Secondly storage uses difficultly on Qubes os, if standalone vm installs difficulty in HDD or USB, user needs to install neither Linux nor Windows os, SSD usage becomes bloated.
System requirement of Qubes os is strict, for example there are a few of PC with PS/2 connection today, Qubes os can run only on limited PC.
Functions of Qubes os dependents to systemd services, user does not have freedom of choice init.

Neither pros nor cons between Qubes os and KVM

Separation between vm and other vm.
Qubes os not yet supports to Wayland, but vms are separating about each other, X11 server can not know to task of other vm.
KVM supports to Wayland, Wayland is separation running apps, this security is same to design of Qubes os.
Text or file copies to other vm, it supports both Qubes os and KVM(But Qubes os supports to only Linux and Windows).
User can use full disk encryption.

Cons of KVM

KVM runs on Linux, and Virt-Manager can manage for KVM, so using KVM is more easily than Qubes os.
KVM users are larger than Qubes os users, so documents of KVM are larger than documents of Qubes os.
If guest os runs on KVM, security of host is safety than VirtualBox.
If user uses Alpine Linux as KVM host, Alpine can install to RAM and run from RAM, guest os of KVM on Alpine is perfectly disposable(Alpine supports to LBU, setting and date on guest vm are able to kept).
User can select lightweight distribution as if VOID(Minimal RAM is 96MB) or Alpine(Minimal RAM is 128MB), user cun use KVM on old and a few of RAM PC(Minimal RAM of Qubes os is 6GB).
KVM supports to os neither Linux nor Windows as guest, for example KVM can copy file to Haiku from FreeBSD, Qubes os can not this.
KVM can run without systemd, user can be free to choice host distribution.
User can install guest os in secondly storage same to in first storage.

Pros of KVM

KVM runs only on Linux, so user must select only Linux as host.
Linux is not designed for security, so not secure codes are existing a lot of on Linux kernel, threats can use their codes for attack to host.
Linux is written by C language, C language is not memory safely, this is attack surface.
KVM is not designed for security, so if user use KVM for security, user must set KVM myself.
KVM runs on Linux, so host Linux is not safely than dom0 of Qubes os(But host Linux can use guest vm for network access like as router).
KVM separates vms each other, but separation between host and guest is inferior to Qubes os.
KVM is inferior to VirtualBox about security of guest vm(KVM does not support disp-vm and snapshot, if guest vm compromised once, user must delete all of vm data).
KVM can guard from attack to host through guest vm, but KVM is weak defense from direct attack to host.
User must upgrade host Linux for security kept, but host Linux must be online for upgrade, this is not safely(But host Linux can access to repository through guest vm, or it can use Tor network for upgrade, host Linux can keep safety).

I think if KVM uses as alternative Qubes os, user choices Alpine Linux as host and enable LBU, and OpenBSD(Minimal RAM is 32MB) installs in KVM as router vm, and other lightweight os installs as guest.
For example KVM run on Alpine Linux, OpenBSD and VOID install as guest, requirement of minimal RAM is less than 512MB.
Minimal RAM of Qubes os is 6GB, and recommend RAM is 16GB.
So if user has PC of less than 4GB RAM, user can use KVM on Alpine Linux as alternative Qubes os.

9

Seems related: