Are qubes really isolated? Security questions

[qubist]

Hi,

While learning about Qubes OS, I am looking into how isolated individual qubes actually are and the possibility of vm-to-vm attacks.

A simple scenario: Two qubes - one attacker and one victim.

In attacker, create some malware.sh, then:

qvm-open-in-vm victim malware.sh

I tried various victims (all disposable, for the sake of testing):

A. Based on fedora-37-xfce-dvm

Result: malware.sh opens in Geany text editor in the victim qube.

B. Based on whonix-16-ws-dvm

Result: the victim qube opens an empty terminal window and a dialog box saying:

‘Failed to execute child process “”: Failed to execve: No such file or directory’

No idea what this is telling me.

C. Based on fedora-37-minimal-dvm

Result: nothing visible (no windows are opened)

If instead of malware.sh I use an URL:

qvm-open-in-vm victim http://example.com

Results:

A. fedora-37-xfce-dvm based qube opens Firefox
B. whonix-16-ws-dvm based qube opens Tor Browser
C. Here I think I may have found a bug:

The exact command I used in attacker was:

qvm-open-in-vm disp8300 http://example.com

where disp8300 is the name of a running qube based on fedora-37-minimal-dvm. Something very weird is happening: Qubes OS starts creating new dispXXXX VMs one after another to infinity. As I am typing this, there are already 20+ and counting. Even closing the attacker disposable doesn’t stop the process and new disposables keep piling up, so I will save and reboot, then finish this post…

OK. To make sure this is not an individual accident, I retried the latest test and the result was the same.

Based on all that, I am questioning:

1. If one can qvm-copy/move-to-vm between any domUs (including templates) with no restriction whatsoever, thus making any VM a potential attacker or victim, why is that called a “security-by-isolation” model? Isolation means no contact.

I understand the usability benefits but what can prevent an attacker to overfill victim’s storage by sending files with all the consequences from that?

2. Why is it allowed any qube to tell another “open this URL” and the other one obeys with no restriction whatsoever? This has quite serious security and privacy implications.

3. My “C” case shows a practical attack to the whole Qubes OS by causing this avalanche of disposables, exhausting system resources with all the consequences this may have. Is this an actual bug (as I think) or an expected behavior? Has it been reported?

4. What defines how a qube should act when another qube tells it “receive/open this”?

5. What mechanisms are there to restrict/control 4? (e.g. some security policy?)

6. How to create a truly isolated PVH VM to which one can’t send anything which is not explicitly allowed?

[SteveC]

@qubist

If one can qvm-copy/move-to-vm between any domUs (including templates) with no restriction whatsoever, thus making any VM a potential attacker or victim, why is that called a “security-by-isolation” model? Isolation means no contact.

Your examples all have to do with open-in-vm (something I very rarely do), but this asks about copy/move.

Yes, you can do this freely…but (unless you broke something) note that a popup is brought up to confirm the action. Not only that but the user has to select the recipient machine. (I would like for the sender to be able to specify a default in the dialog, but I can see arguments against implementing that, too…so it likely won’t happen.) In other words, attacker can’t simply send the file to victim without the carbon-unit user knowing about it and approving the action, and even selecting which VM is “victim.”

I am not a hundred percent sure but I think open-in-vm from one domU to another works the same way. (At least, I get a popup when I click on links in emails and I think that’s the open-in-vm mechanism at work.)

Now if the sending VM (attacker) is dom0, none of these safeguards are in place. Dom0 does not have to ask for confirmation and can specify the target (victim) on the command line.

But it is part of the design that dom0 can do what it wants, and we are warned endlessly that if dom0 gets compromised, it’s game over. (So: don’t surf the internet in dom0!)

Another thing you can do, is to not have firefox installed on every VM (and also not have every VM have access to the network). Nor word/librewrite, or any other app. That way an attacker, if it manages to get around the popup, can’t do a firefox attack on a qube that doesn’t have firefox at all.

The default qubes installation leaves you with a number of VMs (personal, vault, work, untrusted, etc) that all have every piece of software installed. They all derive from the same template. You can change this, and ensure that (say) vault doesn’t have firefox available. That makes an attack more difficult than it already is, because you can’t infect firefox on a VM that doesn’t have firefox on it. (If interested in this particular aspect, there are a lot of posts here on it.)

[aronowski]

@qubist

• My “C” case shows a practical attack to the whole Qubes OS by causing this avalanche of disposables, exhausting system resources with all the consequences this may have. Is this an actual bug (as I think) or an expected behavior? Has it been reported?

If you want, you can file an appropriate issue here for this case. Maybe it’s indeed a good idea to have a policy that will make a window pop up when a qube requests to create a disposable.

Although that’s not the only DoS attack that can be performed. Try, for instance, installing something in a qube that has little memory available like sys-net and most likely it’ll start swapping so much the whole computer will be rendered laggy or unresponsive until it’s closed - in my case it took literal seconds for the cursor to move and i have an NVMe drive, which should be quite fast.

[SteveC]

The C case arises when a disposable called to do something, has a default dispvm which it calls to do the work…and the default dispvm is itself.

I used to have this happen all the time. The way to avoid this (for now, until something systemic is done to fix it) is to ensure that no dispvm (i.e, no DVM Template) lists itself as its default dispvm. (Generally there’s a default applied to all qubes; go to that qube and make sure the default isn’t selected.)

There was talk somewhere (I have no recollection where) of setting things up so that a numbered dispvm couldn’t ever have a default dispvm. It presumably got called to handle a task, not to pass it off to someone else.

[adw]

@qubist, have a look at RPC policies | Qubes OS to better understand this aspect of Qubes OS.

[augsch]

@qubist

If one can qvm-copy/move-to-vm between any domUs (including templates) with no restriction whatsoever, thus making any VM a potential attacker or victim, why is that called a “security-by-isolation” model? Isolation means no contact.

There’s always a prompt in trusted dom0 that asks you WHICH qube you want to copy to.

@qubist

Why is it allowed any qube to tell another “open this URL” and the other one obeys with no restriction whatsoever? This has quite serious security and privacy implications.

Because you are opening urls in dispVMs, which by default store nothing valuable. So for usability, open-in-vm is configured to be auto accepted for dispVMs. You can certainly disable that by configuring qrexec RPC rules.

@qubist

My “C” case shows a practical attack to the whole Qubes OS by causing this avalanche of disposables, exhausting system resources with all the consequences this may have.

Yes, this is kind of DOS attack, but again it’s also you that configured the default dispVM to be a function-lacking qube. As long as there’s a way to deal with that link/file ( which by default, is true, because the default dispVM is fully-functional fedora-XX-dvm ), the chain won’t come into exist.
Anyway, the chain startup of dispVMs is a issue that’s being tracked on github.

@qubist

• What defines how a qube should act when another qube tells it “receive/open this”?
• What mechanisms are there to restrict/control 4? (e.g. some security policy?)

Please take a look at qubes RPC policies.

[unman]

These are interesting questions, but your initial testing was entirely
biased by your use of disposables, and your use of minimal templates.
Qubes allows some actions to disposables for ease of use: they are a
special case.
Minimal templates are for use by advanced users - If “something
weird” happens with a minimal template, then almost certainly it is a
package or configuration problem. (Sometimes it is indicative of a wider
problem, but rarely.)

If one can qvm-copy/move-to-vm between any domUs (including templates) with no restriction whatsoever, thus making any VM a potential attacker or victim, why is that called a “security-by-isolation” model? Isolation means no contact.

Although your tests were done with qvm-open, the principle is the same
and the answer the same: you cant do with with no restriction.
There are policies set in /etc/qubes/policy.d/90-default.policy that
govern what can be done between qubes.
By default, the policy for qvm-copy is:
qubes.Filecopy * @anyvm @anyvm ask
That means that any use of qvm-copy between qubes is restricted by a
pop-up validation.

For qvm-open the situation is the same except that transactions to
a disposable opened from a qube, are allowed. (You cant qvm-open to just
any old disposable.)

These policies govern the level of compartmentalisation.
You can adjust them to your own use case, by creating a new policy file.
(E.g. you may want to stop any copying to/from the vault.)

Why is it allowed any qube to tell another “open this URL” and the other one obeys with no restriction whatsoever? This has quite serious security and privacy implications.

See above - restrictions are in place by default.
Your use of disposables masked this.

My “C” case shows a practical attack to the whole Qubes OS by causing this avalanche of disposables, exhausting system resources with all the consequences this may have. Is this an actual bug (as I think) or an expected behavior? Has it been reported?

Yes.
In this case you used a minimal template (for advanced users), without
knowing the consequences. This is never a good idea.

What defines how a qube should act when another qube tells it “receive/open this”?

A policy set in dom0 under /etc/qubes/policy.d/

What mechanisms are there to restrict/control 4? (e.g. some security policy?)

See above.

How to create a truly isolated PVH VM to which one can’t send anything which is not explicitly allowed?

If you are not content with the current “ask” rule, you create a new policy:
qubes.Filecopy * @anyvm ISOLATED_QUBE deny
New policies in /etc/qubes/policy.d/ must be named with lower number
than 90 to over ride the default.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.

[Isolator]

@SteveC

Yes, you can do this freely…but (unless you broke something) note that a popup is brought up to confirm the action. Not only that but the user has to select the recipient machine.

This confirmation popup is buggy as hell in R4.1, sometimes one command spawns 5+ popups. So I wouldn’t recommend relying on it for security.

I have questions regarding default rpc policies too - is it really necessary to set “qubes.OpenInVM”, “qubes.OpenURL”, and “qubes.StartApp” defaults to “ask”? Wouldn’t it be better to set the same defaults as for the “qubes.VMShell”, ie to “deny”?

Firstly, “qubes.OpenInVM” and “qubes.OpenURL” duplicate “qubes.Filecopy” and “qubes.ClipboardPaste” respectively - if you want to open some file in a vm, it is easier to just copy the file to the vm and open it there than to bother with some command. The same is if you want to open a URL - it is easier to just copy a link to the vm where you want to open it.

Second, the only case where you would want to use “qubes.StartApp” that I can think of is if you want to open something in a disposable - which is already allowed for a spawning appvm when the target is “@dispvm”, which is the first line by default already.

So I’ve just changed my policies to deny to any vm except for the spawned disposable, ie…

# Open a file in a VM
qubes.OpenInVM		*	@anyvm			@dispvm			allow
qubes.OpenInVM		*	@anyvm			@anyvm			deny

# Open URL in a VM
qubes.OpenURL		*	@anyvm			@dispvm			allow
qubes.OpenURL		*	@anyvm			@anyvm			deny

# Start application using its menu entry (only applications with menu entries
# are allowed, no arbitrary command). Argument is an application name (in case
# of Linux, basename of .desktop file from /usr/share/applications or similar
# location).
qubes.StartApp		*	@anyvm			@dispvm			allow
qubes.StartApp		*	@anyvm			@anyvm			deny

…and in general also to deny filecopy and clipboard by default, and allow only based on tags.

“Deny by default” is a universal rule of thumb when it comes to security policies.

[tempmail]

I do everything important in dispVMs. So, in my policies, ask is default for all dvm-templates, except for one, in which I don’t do anything important.

[SteveC]

@Isolator

So I’ve just changed my policies to deny to any vm except for the spawned disposable, ie…

Good.

Yes, the default is loose. However, sometimes people want something different than what you did. (For instance, me: almost everything I have an application on is a named disposable and I do want to be able to send things there.)

If the default were what you have, it would “break” a lot of people’s setups and most wouldn’t know (especially on their first day using qubes) what question to ask as to how to fix it.

Far better to leave it a little loose and let people tighten up once they tune their system. Even the default is hugely beneficial compared to, say, Micro$haft Windows or even (it pains me to say, as I do like them) any of the more popular/general purpose Linux distros, because at least the user will know when data is moving from one VM to another.

[unman]

Far better to leave it a little loose and let people tighten up once they tune their system. Even the default is hugely beneficial compared to, say, Micro$haft Windows or even (it pains me to say, as I do like them) any of the more popular/general purpose Linux distros, because at least the user will know when data is moving from one VM to another.

There are many things in Qubes which are a compromise between security
and usability. The defaults in the policies are an example.
users can (if they are aware of them) change the policies as they will.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.

[unman]

This confirmation popup is buggy as hell in R4.1, sometimes one command spawns 5+ popups. So I wouldn’t recommend relying on it for security.

I wanted to pick up on this.
It cant be that this is " buggy as hell in R4.1", or we would have read
many reports of it.
So there is, I would suggest, something about your setup or
configuration that is triggering this. Can you share some details? You
may have found some edge case that passes most users by.
What are those times when one command spans multiple popups? Is it
always the same commands? Which ones?

Perhaps you should open a new thread for this.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.

[qubist]

Thanks everyone for the explanations.

@SteveC

If interested in this particular aspect, there are a lot of posts here on it.

I am really interested in a setup in which networking, storage and file opening/editing happen in separate (as minimal and disposable as possible) VMs. I realize this is off-topic, so if you could link to proper threads, that would be very welcome.

For instance, me: almost everything I have an application on is a named disposable and I do want to be able to send things there

That’s the approach I try to use too. Currently, I am learning how to base everything on the most minimal templates possible, both from security and resource usage perspective. AFAICS, debian-11-minimal is twice smaller than fedora-37-minimal. No idea if whonix-minimal (based on the former) exists.

[qubist]

@unman

I have always liked the minimalist approach to security and I have tried to follow it on my systems during the years. I am learning about Qubes OS with my own pace too.

Minimal templates are for use by advanced users […]

This is being repeated on many occasions and it makes sense. The problem is that nobody defines what “advanced user” means. At which point can one be considered an “advanced user”? What are the criteria? What are the ones who are not advanced? Mediocre? Is one supposed to stay mediocre and not learn in order to be safe from mistakes? Most importantly: How does one become advanced? I am looking for documentation on that and I am not finding it.

[Isolator]

@unman

Perhaps you should open a new thread for this.

Problem is (as usual), I can’t simply reproduce this bug with a deterministic chain of actions, so there’s no point in wasting a thread. It “just happens” when it feels like it for all I know.

@unman

What are those times when one command spans multiple popups? Is it
always the same commands? Which ones?

I guess qvm-move and qvm-copy, or whatever command is executed behind the scenes when you press “Copy/Move to VM”. Sometimes it spawns only one window like it is supposed to, other times it drops its spaghetti all over my screen with 4-5 windows, all in the same spot. Started after switching to 4.1. Obviously was very low on my priority list, considering all the other issues I had to deal with.

My rough guess is that whatever qrexec call this command is supposed to make gets multiplied for whatever reason.

Also I think unrelated - have a bug in whonix with 3-4 duplicate context “Copy/Move to VM” menu entries. But this bug was in 4.0 already and should be reported to the whonix dev. But it’s similar in that it duplicates entities for whatever reason.

To reiterate, the bug with duplicate confirmation popup windows is not whonix specific. The duplicate menu entries bug is.

[qubist]

@augsch

There’s always a prompt in trusted dom0 that asks you WHICH qube you want to copy to.

Please look at this:

It is possible to have a qube with netvm=none and connect to the Internet from it, without any prompt whatsoever:

$ ping 1.1.1.1
ping: connect: Network is unreachable
$ qvm-run-vm --dispvm ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=49 time=79.0 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=49 time=24.0 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=49 time=23.8 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=49 time=24.2 ms
64 bytes from 1.1.1.1: icmp_seq=5 ttl=49 time=25.7 ms
[...]

It all comes down to the default_dispvm setting of the qube - if it is a DVM with Internet connectivity, then any qube having that default_dispvm can connect to the Internet.

Meanwhile, regardless of the default_dispvm, in the settings UI of the qube (Firewall rules), one reads the red text “This qube has no net qube. It will not be have any access anyway.” which is misleading in this case.

[DVM]

There’s a warning in the qube settings about this:

[qubist]

There’s a warning in the qube settings about this:

I would never see this if you didn’t point it out. It is not displayed clearly enough. It is not displayed at all while creating the qube, unless one checks “launch settings after creation”.

I would never see it if I was running things through the CLI.

It seems also easy to circumvent even in a long chain of qubes:

AAA-dvm: template=fedora-xfce, netvm=none, default_dispvm=fedora-xfce-dvm
BBB-dvm: template=fedora-xfce, netvm=none, default_dispvm=AAA-dvm, *no warning
CCC: template=fedora-xfce, netvm=none, default_dispvm=BBB-dvm, *no warning

And in CCC:

[user@CCC ~]$ ping 1.1.1.1
ping: connect: Network is unreachable
[user@CCC ~]$ qvm-run-vm --dispvm 'qvm-run-vm --dispvm "qvm-run-vm --dispvm ping 1.1.1.1"'
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=53 time=20.0 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=53 time=18.7 ms
64 bytes from 1.1.1.1: icmp_seq=3 ttl=53 time=27.8 ms
64 bytes from 1.1.1.1: icmp_seq=4 ttl=53 time=20.0 ms
[...]

[DVM]

@qubist

AAA-dvm: template=fedora-xfce, netvm=none, default_dispvm=fedora-xfce-dvm
BBB-dvm: template=fedora-xfce, netvm=none, default_dispvm=AAA-dvm, *no warning
CCC: template=fedora-xfce, netvm=none, default_dispvm=BBB-dvm, *no warning

This change nothing to the above.
Your base dvm “AAA-dvm” has an internet accessible default disposable template. “BBB-dvm” and “CCC” have no warning because both of them and the default disposable template have their netvm set to none. If you hop to AAA-dvm, of course you’ll be able to get out. Switching AAA-dvm default_dispvm to “none” stop this from happening.

[qubist]

If one knows about this, of course, one can do something about it.

The point of the example, however, is: one would not know, as the UI doesn’t inform well enough and that makes security issues possible.