[guide] how-to setup a sys-dns qube

Community Guides
81

My bad, I think I went too fast with iptables-nft. I deleted the package and it still works…
for sudo iptables -t nat -L , i had “PR-QBS” chain with the template from 4.1 but nothing when i upgrade it to 4.2
So there is no need to install iptables-nft as I thought…

82

Maybe you or @fepitre could create one. it’s a very handy script!

83

If you don’t have PR-QBS and no rules in dnat-dns, then I don’t know how you got this to work. Are you able to resolve domains without leaks in your qubes?

84

@adw Can you take a look at the previous answers about the script and see if it is a good idea to add it to the clean install steps?

85

Maybe I wasn’t clear enough in my explanations (i’m a french woman so my English may not be very good). I’ll try to be clearer :slight_smile:
I update my minimal template to 4.2, now i don’t have PR-QBS with sudo iptables -t nat -L.
In my sys-dns and sys-wall, I’m using the script you “translated” into NFTtables (in /rw/config/rc.local). With dnsleaktest.com i could see all is allright and i don’t have leak…
Hope i was being clear :confused:

1 Like
86

Sorry, I thought you were still using the iptables rules. If you are using the translated ones, then yes, it will work properly.
To check the nftables rules, you can use sudo nft list table ip qubes. You should see the “dnat-dns” chain redirecting to 127.0.0.1.

87

sys-dns:

chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		meta l4proto { tcp, udp } th dport 53 dnat to 127.0.0.1
	}

sys-wall:

chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		meta l4proto { tcp, udp } ip daddr { 10.139.1.1, 10.139.1.2 } th dport 53 dnat to 10.138.23.81
		meta l4proto { tcp, udp } th dport 53 drop
	}

10.138.23.81 is my sys-dns’s ip :slight_smile:

88

@adw Can you take a look at the previous answers about the script and see if it is a good idea to add it to the clean install steps?

Can you continue this in a separate thread?

1 Like
89

What is the problem here? How can it be solved?

[user@dom0 ~]$ qubesctl --show-output --skip-dom0 --targets fedora-38-minimal state.sls update.qubes-vm
fedora-38-minimal: ERROR (exception [('/srv/reactor', '/tmp/tmpz6bjosb6/srv/reactor', "[Errno 13] Permission denied: '/srv/reactor'"), ('/srv/formulas', '/tmp/tmpz6bjosb6/srv/formulas', "[Errno 13] Permission denied: '/srv/formulas'"), ('/srv/pillar', '/tmp/tmpz6bjosb6/srv/pillar', "[Errno 13] Permission denied: '/srv/pillar'"), ('/srv/salt', '/tmp/tmpz6bjosb6/srv/salt', "[Errno 13] Permission denied: '/srv/salt'")])
[user@dom0 ~]$
90

You need to run this command with sudo.

1 Like
91

Sorry, this falls outside my expertise. I don’t know enough to comment.

92

Thanks @qubist I was able to get this running mostly following your posts. A major deviation however was using fedora-38-minimal as a starting point and using nft instead as iptables isn’t included in qubes-core-agent-networking on fedora 38.

The changes are as follows;
Here’s /rw/config/rc.local for f38-m-dns-dvm

#!/bin/sh

# This script will be executed at every VM startup, you can place your own
# custom commands here. This includes overriding some configuration in /etc,
# starting services etc.

nft='/usr/sbin/nft'

# allow redirects to localhost
/usr/sbin/sysctl -w net.ipv4.conf.all.route_localnet=1

#redirect dns requests to local host
nft flush chain ip qubes dnat-dns
nft 'add rule ip qubes dnat-dns ip daddr 10.139.1.1 udp dport 53 dnat to 127.0.0.1:53'
nft 'add rule ip qubes dnat-dns ip daddr 10.139.1.1 tcp dport 53 dnat to 127.0.0.1:53'
nft 'add rule ip qubes dnat-dns ip daddr 10.139.1.2 udp dport 53 dnat to 127.0.0.1:53'
nft 'add rule ip qubes dnat-dns ip daddr 10.139.1.2 tcp dport 53 dnat to 127.0.0.1:53'

#allow downstream qubes to connect over 53
nft 'add rule ip qubes custom-input iifname "vif*" tcp dport 53 accept'
nft 'add rule ip qubes custom-input iifname "vif*" udp dport 53 accept'

#drop alternate host dns requests
nft 'insert rule ip qubes postrouting ip daddr != 127.0.0.1 tcp dport 53 counter drop'
nft 'insert rule ip qubes postrouting ip daddr != 127.0.0.1 udp dport 53 counter drop'

echo 'nameserver 127.0.0.1' > /etc/resolv.conf
# https://github.com/DNSCrypt/dnscrypt-proxy/wiki/Installation-linux
# https://wiki.archlinux.org/title/Dnscrypt-proxy#Enable_EDNS0
echo 'options edns0' >> /etc/resolv.conf

ln -s /rw/dnscrypt-proxy /etc/dnscrypt-proxy
/usr/bin/systemctl start dnscrypt-proxy.service

I also uninstalled systemd-resolved on f38-m-dns-dvm to prevent potential conflicts with dnscrypt-proxy and to minimize attack surfaces. No need to uninstall anywhere else as systemd-resolved will simply point to sys-dns.

I also didn’t need to do anything in /rw/config/rc.local for f38-m-firewall-dvm/sys-wall, at least for dnscrypt-proxy anyway.

Here’s a few tests I ran on a qube with sys-wall as it’s network qube:

[user@f38-browser ~]$ host google-analytics.com
google-analytics.com host information "This query has been locally blocked" "by dnscrypt-proxy"
google-analytics.com host information "This query has been locally blocked" "by dnscrypt-proxy"
google-analytics.com host information "This query has been locally blocked" "by dnscrypt-proxy"
[user@f38-browser ~]$ host google.com
google.com has address 142.250.181.142
google.com has IPv6 address 2607:f8b0:4007:814::200e
google.com mail is handled by 10 smtp.google.com.
[user@f38-browser ~]$ host google.com 8.8.8.8
;; communications error to 8.8.8.8#53: timed out
;; communications error to 8.8.8.8#53: timed out
;; no servers could be reached

[user@f38-browser ~]$ host google-analytics.com 8.8.8.8
;; communications error to 8.8.8.8#53: timed out
;; communications error to 8.8.8.8#53: timed out
;; no servers could be reached

It should be noted this is the same behavior on sys-dns as the drop rule is added to the “postrouting” chain in nft.

I’m very much a beginner with qubes os and nft for that matter, if anybody sees any glaring security issues here, please let me know.

1 Like
93

I also uninstalled systemd-resolved on f38-m-dns-dvm to prevent potential conflicts with dnscrypt-proxy and to minimize attack surfaces.

The DVM is an AppVM. If you want persistent removal of system software, you need to do it in the template.

94

So I reinstalled dnscrypt-proxy and the rc.local script above “stopped working”. Not exactly sure what changed, but basically the nft rules prevented dnscrypt-proxy from the initial resolver download. The solution was to start the service before nft changes for the initial download, here’s the new script for sys-dns://rw/config/rc.local

#!/bin/sh

# This script will be executed at every VM startup, you can place your own
# custom commands here. This includes overriding some configuration in /etc,
# starting services etc.

nft='/usr/sbin/nft'

echo 'nameserver 127.0.0.1' > /etc/resolv.conf
# https://github.com/DNSCrypt/dnscrypt-proxy/wiki/Installation-linux
# https://wiki.archlinux.org/title/Dnscrypt-proxy#Enable_EDNS0
echo 'options edns0' >> /etc/resolv.conf

# allow redirects to localhost
/usr/sbin/sysctl -w net.ipv4.conf.all.route_localnet=1

ln -s /rw/dnscrypt-proxy /etc/dnscrypt-proxy
/usr/bin/systemctl start dnscrypt-proxy.service

sleep 2

count=0
max=10
while ! test "$(systemctl is-active dnscrypt-proxy.service)" = "active"; do
  count=$((count+1))
  #just break out and hope for the best?
  if test ${count} -ge ${max}; then
    break
  fi
  sleep .5
done

#redirect dns requests to local host
${nft} flush chain ip qubes dnat-dns
${nft} 'add rule ip qubes dnat-dns ip daddr 10.139.1.1 udp dport 53 dnat to 127.0.0.1:53'
${nft} 'add rule ip qubes dnat-dns ip daddr 10.139.1.1 tcp dport 53 dnat to 127.0.0.1:53'
${nft} 'add rule ip qubes dnat-dns ip daddr 10.139.1.2 udp dport 53 dnat to 127.0.0.1:53'
${nft} 'add rule ip qubes dnat-dns ip daddr 10.139.1.2 tcp dport 53 dnat to 127.0.0.1:53'

#allow downstream qubes to connect over 53
${nft} 'add rule ip qubes custom-input iifname "vif*" tcp dport 53 accept'
${nft} 'add rule ip qubes custom-input iifname "vif*" udp dport 53 accept'

#drop alternate host dns requests
${nft} 'insert rule ip qubes postrouting ip daddr != 127.0.0.1 tcp dport 53 counter drop'
${nft} 'insert rule ip qubes postrouting ip daddr != 127.0.0.1 udp dport 53 counter drop'
95

Specifically it’s the

rules. 1 of 2 things is happening, #1 the initial resolver is doing dns requests over port 53 and not on local host, or more likely #2 The service fails to start until the resolvers are downloaded effectively preventing ANY dns requests. I didn’t investigate too thoroughly, only saw a “traffic rejected” or something indicative of the port not being open.

96

Not exactly sure what changed

Have you checked logs? Maybe increase log level too?

Generally, the firewall should start before any network service (even before networking is enabled), not after it. If it is blocking something, you may need to check what exactly.

I will look at all this and hopefully refresh this guide as soon as I find some time. There is just too much on my head recently, so I apologize for not being helpful.

1 Like
97

I mean, if you want to. Wasn’t really seeking that, do appreciate it though! Mostly just wanted to document the changes I made to get it working in case someone ended up using my nft rules. To be clear; the firewall rules ARE up before dnscrypt-proxy, which was causing the problem. Like I said, these two rules in particular were causing the issue;

${nft} 'insert rule ip qubes postrouting ip daddr != 127.0.0.1 tcp dport 53 counter drop'
${nft} 'insert rule ip qubes postrouting ip daddr != 127.0.0.1 udp dport 53 counter drop'

This tells me dnscrypt proxy is making external dns requests as part of its initialization. Once that’s done, preventing all dns requests not on localhost is fine. There is a brief window where forgeries can occur, but I figure since it’s dnscrypt-proxy making the requests, it’s probably OK.

98

First of all a big thank you to all of you that have maintained this info.

I was hoping someone could help me figure out what I missed setting this up so far I have followed the instructions for the latest version of qubes and edited my .toml file but I think I missed something because when I try to run it the bootstrap resolvers aren’t responding.

So far I’m just trying to get it up and running everything is vanilla in my toml file besides the changes mentioned here with username and such for within this post.

I have also uncommented the bootstrap resolvers line with 9.x.x.x.x and 8.x.x.x.x and changed my cache location under sources.

Do I need to forge on with the rest of the steps for everything to sync up?

99

I got it up and responding but now when I create a test qube and use sys-wall as the network the “host” command gives me an error “command not found”

100

I am not using it currently, as it stopped working for me on F37 and I haven’t had the time to fix it.

If you are on F38, you may be interested to read:

DNSCrypt leak ISP DNS on MY Fedora 38 !!