Continuing the discussion from Running AIDE in dom0:
I’ve seen couple of times users talk about installing IDEs (Intrusion Detection Systems) in dom0 and I wanted to start a discussion on this to see people’s takes on it.
2 Likes
Through the way Qubes OS is made, I think intrusion detection in dom0 doesn’t make much sense. You are not only increasing the attack surface of the most trusted component of the system by installing additional software. At the same time that you are missing the majority of what would potentially need to be analyzed (domUs – aka. AppVMs & StandaloneVMs). For this second point the following (read-only) mailing list discussion may be more adequate: [qubes-users] Hardening Guide for Paranoid Noobs?.
3 Likes
@deeplow makes a good point - IDS should concentrate on the systems with the most attack surface.
That being said, if an unknown vector is used (perhaps a unknown Qubes Impacting XEN break out exploit, something via a higher ring level, or some unknown vector using graphics) it could make sense to have checks and balances on Dom0.
In terms of the Kernel attestation at boot time, that is easy enough to do - just use heads or Anti Evil Maid. This will provide reassurance that the XEN kernel and init have not been tampered with.
In terms of approach - I prefer heads as I can neuter ME & set HAP bit at the same time which AEM doesn’t support. Taking steps against ME also provides me some reassurance around attack surface operating at different ring level (clarity: some reassurance, I am well aware we don’t know the whole in and out of the ME system and processor). Heads also provides me attestation my firmware has not been tampered with, which removes yet another persistence vector.
So if we have firmware attestation from heads/TPM, then kernel and initrd attestation from heads/TPM/GPG then any persistence of a threat that actually managed to get into Dom0 would need to be from init scripts or modification of a daemon/software a-la rootkiting.
I can see that being a situation where IDS may be valid in Dom0 - but lets be realistic, in order to get to that situation against a person running Heads, ME cleaned/HAP set, QubesOS all up to date and fully patched then adversary would be risking burning some pretty serious 0days just to get into Dom0.
Thats not to say it cant happen, you just need to justify the likelihood of that in your own threat model vs the risk of importing binaries into Dom0 to do IDS.
To close that gap for me - I simply rolled my own IDS in Dom0. Its pretty simple, a script does a shasum of each init file, library and binary and then dumps the output to a file. It then shasums that file and prints it to screen. It then prints the shasum of the dump from previous run (which I compare manually to the picture I took). It then compares the hashes contained in the two dump files for differences. I therefore have some attestation that im pretty safe, without needing to import any non bundled binaries into Dom0 and I know the script is safe as I wrote it.
Put all that together and when i power up, im pretty certain my system is in a known good state. So my focus is on, as Deeplow points out, the VMs themselves and protecting those.
3 Likes
dom0 is surely not the place for such things…
-
A traditional IDS can be used in a proxy VM,
see my PoC:
http://zrubi.hu/en/2017/traffic-analysis-qubes/ -
The ‘new generation’ of IDS might be running in a privileged VM:
https://drakvuf.com/ - for Xen
(AFAIK there is no real products based on this project)or VMware vShield Endpoint
dead ended by vmware:
VMware Knowledge Base
3 Likes
In my more paranoid times, and on particular machines, I run tripwire in
dom0, and significant qubes, and snort on selected netvms.
I use it in dom0 because if it’s game over, I want to know asap.
I use it in in selected qubes so that I am aware of any compromise that
affects inter qube isolation.
I check the network traffic because, well just because.
These are programs I am familiar with.
The greatest issue, as with any IDS system, is tuning the rules and
monitoring alerts. This isn’t for beginners tbh.
Unifying alerts in to a single dashboard would be a start. But knowing
what any particular alert means requires some knowledge.
6 Likes
My knee-jerk reaction to installing IDS in dom0 is that it’s got far more costs than benefits, mainly because the benefits are close to nil. AFAIK, IDS detecting an attacker in dom0 would already be far, far too late.
Using the engineering swiss cheese analogy I made in the compartmentalization thread, the attacker would have had to pass through all the pieces of cheese to the last and most important piece that controls everything else. That’s a lot of missed opportunities for earlier detection, and having dom0 access would open the doors to many ways of evading detection, in my uninformed opinion. An attacker with enough competence and resources to make it this far wouldn’t let an off-the-shelf IDS trip him up. It’d be like an invader getting to the gold in Fort Knox, then screwing up by slipping on a banana peel.
Basically, I believe a compromised dom0 = GameOver™ and that banking on detection of this after the fact doesn’t do much to justify whatever cost the IDS incurs.
(As usual, I must add that I’m not a technical person, so I might be mistaken about some things.)
2 Likes
I’d agree with the general gist here: once you’ve done all you can to
secure your DomU’s and employ good trade craft (OpSec) in the way you
use your computer and other technology … THEN geek out about Dom0 and
ME as much as you want.
2 Likes
once you’ve done all you can to
secure your DomU’s and employ good trade craft (OpSec) in the way you
use your computer and other technology … THEN geek out about Dom0 and
ME as much as you want.
I wholeheartedly approve of this message 
2 Likes
People who focus only on dom0 are kind of like kings who focus only on protecting the throne room.
On this topic, a big problem is that the official Qubes documentation and the default configuration seem to go out of their way to discourage users from securing DomUs. A prime example is Joanna’s “WTF” rant copied into the docs from the old version of /etc/sudoers.d/qubes (copied in its full glory below).
On that note, I strongly support the idea of having sudo prompt as the default in Qubes, not passwordless root, and don’t think I’m in the minority. If WhiskerMenus vs XFCE menus warrants a poll, this key security issue surely does too.
user ALL=(ALL) NOPASSWD: ALL
# WTF?! Have you lost your mind?!
#
# In Qubes VMs there is no point in isolating the root account from
# the user account. This is because all the user data is already
# accessible from the user account, so there is no direct benefit for
# the attacker if she could escalate to root (there is even no benefit
# in trying to install some persistent rootkits, as the VM's root
# filesystem modifications are lost upon each start of a VM).
#
# One might argue that some hypothetical attacks against the
# hypervisor or the few daemons/backends in Dom0 (so VM escape
# attacks) most likely would require root access in the VM to trigger
# the attack.
#
# That's true, but mere existence of such a bug in the hypervisor or
# Dom0 that could be exploited by a malicious VM, no matter whether
# requiring user, root, or even kernel access in the VM, would be
# FATAL. In such situation (if there was such a bug in Xen) there
# really is no comforting that: "oh, but the mitigating factor was
# that the attacker needed root in VM!" We're not M$, and we're not
# gonna BS our users that there are mitigating factors in that case,
# and for sure, root/user isolation is not a mitigating factor.
#
# Because, really, if somebody could find and exploit a bug in the Xen
# hypervisor -- as of 2016, there have been only three publicly disclosed
# exploitable bugs in the Xen hypervisor from a VM -- then it would be
# highly unlikely if that person couldn't also found a user-to-root
# escalation in VM (which as we know from history of UNIX/Linux
# happens all the time).
#
# At the same time allowing for easy user-to-root escalation in a VM
# is simply convenient for users, especially for update installation.
#
# Currently this still doesn't work as expected, because some idotic
# piece of software called PolKit uses own set of policies. We're
# planning to address this in Beta 2. (Why PolKit is an idiocy? Do a
# simple experiment: start 'xinput test' in one xterm, running as
# user, then open some app that uses PolKit and asks for root
# password, e.g. gpk-update-viewer -- observe how all the keystrokes
# with root password you enter into the "secure" PolKit dialog box can
# be seen by the xinput program...)
#
# joanna.
Sounds like a good topic for a new thread 
. Also another one for how to secure domUs.
“dom0 is surely not the place for such things…”
Exactly. It would just open up another attack path.
E.g. considering the numerous vulnerabilities in AV software they even seem to do more harm than good for some people…
- A traditional IDS can be used in a proxy VM,
see my PoC:
Traffic Analysis in Qubes OS – Zrubi.hu
Honestly I’m not sure about the value of IDS nowadays though. Considering
most (incl. malware) traffic is TLS encrypted nowadays and one usually doesn’t want to mitm/break TLS for oneself except in very specific cases, an IDS nowadays doesn’t do much more than checking for “bad” DNS requests and IPs. You can also do that with netflow sensors and keep all traffic data pretty much for forever (1MB per day or so).
Side Note: Proxy VMs are quite useful to break TLS if needed… I succeeded with sslsplit.
The ‘new generation’ of IDS might be running in a privileged VM:
https://drakvuf.com/ - for Xenor (Trend Micro) Deep Securtiy - for VMware
Thanks for mentioning that one as I didn’t know it and it looks promising!
It probably opens up a few attack paths as well, but might be worth it under certain circumstances.
1 Like
“People who focus only on dom0 are kind of like kings who focus only on
protecting the throne room.”
Who suggested this?
Securing dom0 is a key part of securing a Qubes system, and using some
IDS is a good way of monitoring the system for changes in dom0.
I see no evidence that Qubes “goes out of its way to discourage securing
DomUs” - nor does it go out of its way to give a false sense of security.
If you are able to correctly configure and use security hardening tools
then you are free to do so - if you have good support then a naive user
may benefit from this, but the default position is aimed at enabling
naive users to benefit from Qubes.
Many distros have access to root from a user account - look at Ubuntu
for one, (despite the requirement to enter user password).
Wouldn’t that thread be a general Linux security thread? Not that I’d be against it–I have much to learn and a thread on the topic (perhaps resulting in a guide) would be enlightening.
I was agreeing with Sven saying that one should focus on securing domUs before thinking about securing dom0. Some people (naive users) focus too much on dom0 since know enough to understand that a compromised dom0 means GameOver™ (the throne room is important) but not enough to understand that securing domUs is the first and arguably most important defense for dom0 (the best way to secure the throne room is by securing the kingdom).
For naive users, a sudo prompt provides far more benefits than costs.
All that’s required is smashing the ‘Enter’ key when the prompt shows up–it therefore costs almost nothing.
On the other hand, the benefits are potentially large (assuming I’m correct that a domU with passwordless root is more vulnerable than one with sudo prompt).
As usual, I’m not technical, so I’m open to being wrong
1 Like
Wouldn’t that thread be a general Linux security thread? Not that I’d be against it–I have much to learn and a thread on the topic (perhaps resulting in a guide) would be enlightening.
I was agreeing with Sven saying that one should focus on securing domUs before thinking about securing dom0. Some people (naive users) focus too much on dom0 since know enough to understand that a compromised dom0 means GameOver™ (the throne room is important) but not enough to understand that securing domUs is the first and arguably most important defense for dom0 (the best way to secure the throne room is by securing the kingdom).
For naive users, a sudo prompt provides far more benefits than costs.
All that’s required is smashing the ‘Enter’ key when the prompt shows up–it therefore costs almost nothing.
It’s hard enough to get users to check what is on the screen without
training them to smash the enter key when a prompt appears.
On the other hand, the benefits are potentially large (assuming I’m correct that a domU with passwordless root is more vulnerable than one with sudo prompt).
It’s the assumption that a qube with passwordless root is more
vulnerable than one without that I don’t accept, but let’s agree to differ.
As always, there’s a trade off between security and usability - I think
the current model works well for most users, you draw the line in a
different place.
Some changes in Qubes I have not accepted - e.g, the removal of filtering
on tinyproxy - and if this change were made it would be one of those
for me, and those I work with.
Great topic! perhaps, the IDE would need to be tested to provide hashing outputs after every run and compare its results with the previous outputs. This will only check for changes in dom0 and check for gaps.
However, if an issue was discovered; how would we solve it?
Can we just swap a dom0 for another dom0?
Or is there a dom0 available in a repository that is stable and easily installed? i.e. Thus not using an update as that might not fix the issue per say, as that would most likely fix any vulnerabilities previously discovered.
If a clean, updated and hardended dom0 was available to download and install; some of us might feel more at ease in case exploit.
Perhaps, there is and I haven’t read enough to find it and install it.
the IDE would need to be tested to provide hashing outputs after every run and compare its results with the previous outputs
That is how the old tripwire was working - however if anybody tried to use it for a real production service… may knows that this is just not a viable solution in practice.
However, if an issue was discovered; how would we solve it?
In case of a dom0 compromise? - that is a game over.
It is hard to accept, but to compromise dom0 there should be an already compromised domU AND several 0 days (or just not patched) bug/vulnerability in any of:
- hardware
- BIOS/UEFI
- hypervisor
- kernel
so simply swapping out the dom0 is a very optimistic way to ‘solve’ a dom0 compromise.
In such situation you better of replacing your hardware, and recreating your whole digital life from scratch.
1 Like
We run tripwire on production servers, and it still seems workable to
us.
I run tripwire on (some) qubes and in dom0 on some machines. Again, it
seems workable.
It’s old venerable, not old decrepit. And, of course, there are new
variants on the theme.
As to network IDS, I don’t know if anyone has yet tried out opensnitch
from the packages I posted. I’d be interested in feedback in the Qubes
context.
1 Like
We run tripwire on production servers, and it still seems workable to us.
And how many ‘exception’ folders you have?
Not to mention if you run it on a live system how do you trust the results?
(As if your box is compromised the attacker can give you a fake result)
In my opinion such filesystem checks can be trusted only if you mount it on another system and do the scan there… which is just not possible most of the time.
But, I not questioning if it is works or not, but how much resource you need to ‘use’ compared to the benefit you got…