I know that inter-Qubes communication bypassing qrexec is generally discouraged. However, it is still quite common when you want to run a network service on a headless Qube for any purpose. It would be nice if we could address another AppVM IP by name, not just by number. Of course, we can populate the /etc/hosts file in all templates as a quick and dirty solution, but I would like to do it more elegantly.
2 Likes
The following can be done in a seperate sys-dns, in my personal setup the service runs in sys-net based on a minimal template with root access disabled:
> /rw/config/rc.local
#!/bin/sh
# This script will be executed at every VM startup, you can place your own
# custom commands here. This includes overriding some configuration in /etc,
# starting services etc.
nmcli radio wifi off
systemctl start dnscrypt-proxy.service
sysctl -w net.ipv4.conf.all.route_localnet=1
/home/user/dns_crypt
> /home/user/dns_crypt
#! /bin/bash
# must be run as root
iptables -I INPUT -i vif+ -p udp --dport 53 -d 127.0.0.1 -j ACCEPT ;
rm /etc/NetworkManager/NetworkManager.conf ;
ln -s /home/user/NetworkManager.conf.dnscrypt /etc/NetworkManager/NetworkManager.conf ;
systemctl restart NetworkManager
and
> /home/user/NetworkManager.conf.dnscrypt
[main]
dns=default
[global-dns-domain-*]
servers=127.0.0.1
[logging]
[keyfile]
unmanaged-devices=mac:fe:ff:ff:ff:ff:ff
All three files are owned by root:root and chmod-ed to 744.
My sys-net uses fedora-35-minimal as it’s template, which I have tweaked like this:
# sudo dnf install qubes-core-agent-networking qubes-core-agent-network-manager NetworkManager-wifi network-manager-applet wireless-tools notification-daemon gnome-keyring polkit
# sudo dnf install @hardware-support (or specific driver)
and for setting up the dns-resolver:
# sudo dnf install dnscrypt-proxy
# sudo systemctl disable --now systemd-resolved
# sudo systemctl disable --now dnscrypt-proxy.service
# vi /etc/dnscrypt-proxy/dnscrypt-proxy.toml
# listen_addresses = ['127.0.0.1:53']
# further tweaks like relays, captive portals, blocking domains possible
dnscrypt-proxy is fast, reliable and highly customizable. Traffic is encrypted, you can block ad- and tracking-server, and it can use relays so the dnscrypt-resolver does not know your IP.
A remark about security: if dnscrypt-proxy is vulnerable an attacker would still have to lurk the victim to request a malicious DNS answer or to poisen a DNS resolver (which one? 
) with malicious DNS answers. Let’s say a malicous TXT record for www.evil.com leads to a buffer overflow in dnscrypt-proxy and gave an attacker RCE on sys-dns or sys-net. As root on sys-dns or sys-net an attacker could spoof all DNS answers, i.e. for www.yourbank.com. HSTS aside, the same applies if dnsmasq or unbound on your router was vulnerable.
PS: I tried to get the nmcli radio wifi off in my NetworkManager.conf, but did not succeed. Ideas and stackexchange.com links welcomed.
PPS: it might be better to keep and disable systemd-resolved instead of removing it
6 Likes
It seems reasonable to run dnscrypt-proxy as non-root and let the service listen on :53000 instead of :53. I’m going to fix that and will update my post - as soon as I find some time. Should be the listening port and the iptables rule only.
edit: ok, this is actually a lot easier. Edit /etc/dnscrypt-proxy/dnscrypt-proxy.toml in the template (i.e. fedora-35-minimal) you use for sys-net or sys-dns and remove the comment ‘#’ in line 55 so it reads:
user_name ='nobody'
After listening sockets have been created, dnscrypt-proxy drops it’s privileges and continues to run as nobody. Redirecting traffic from :53 to :53000 is too messy for me, since I lack profound iptables skills. For changing ports it looks to me one has to DNAT the traffic.
2 Likes
Setting route_localnet=1 may not be optimal from security POV, even if sys-net is untrusted - am still trying to stick to the best practices 
It’s also possible to get a custom DNS setup working by giving it a virtual network device to listen on and bind to.
I.e., here’s how I had it set up in Qubes 4.2 sys-net in my case. My setup is simpler, instead of dnscrypt I only run dnmasq for DNS caching for now. In this config dnsmasq listens on the interface dnsmasqDummy with address 10.222.222.222, and uses only the manually specified DNS servers and ignores the DHCP-provided DNS.
/rw/config/rc.local:
# disable SELinux in fc39, else dnsmasq is disallowed from reading its /etc configs
/usr/sbin/setenforce 0
cp -an /rw/config/etc/* /etc
# create a dummy virtual network device for dnsmasq to bind to
ip link add name dnsmasqDummy type dummy
ip address add 10.222.222.222 dev dnsmasqDummy
ip link set dnsmasqDummy up
ip route add to 10.222.222.222/32 dev dnsmasqDummy
# /usr/lib/qubes/qubes-setup-dnat-to-ns will set up DNAT for AppVM DNS traffic to 10.222.222.222, but we also need to add allow rules.
# Firewall rules both for 10.139.1.1, 10.139.1.2 (default Qubes DNS addresses) and for our DNS server 10.222.222.222
nft add rule qubes custom-input iifname == "vif*" ip daddr 10.139.1.0/30 tcp dport 53 accept
nft add rule qubes custom-input iifname == "vif*" ip daddr 10.139.1.0/30 udp dport 53 accept
nft add rule qubes custom-forward iifname == "vif*" ip daddr 10.139.1.0/30 tcp dport 53 accept
nft add rule qubes custom-forward iifname == "vif*" ip daddr 10.139.1.0/30 udp dport 53 accept
nft add rule qubes custom-input iifname == "vif*" ip daddr 10.222.222.222 tcp dport 53 accept
nft add rule qubes custom-input iifname == "vif*" ip daddr 10.222.222.222 udp dport 53 accept
nft add rule qubes custom-forward iifname == "vif*" ip daddr 10.222.222.222 tcp dport 53 accept
nft add rule qubes custom-forward iifname == "vif*" ip daddr 10.222.222.222 udp dport 53 accept
# restart the services with updated config
systemctl --no-block restart systemd-resolved NetworkManager
/etc/NetworkManager/conf.d/99-use-dnsmasq.conf:
[main]
dns=dnsmasq
rc-manager=resolvconf
[global-dns-domain-*]
servers=10.222.222.222
/etc/NetworkManager/dnsmasq.d/99-dnsmasq.conf:
# To verify cache size and DNS servers being used, run in different terminal tabs:
# sudo journalctl -b0 -af
# sudo killall -USR1 dnsmasq
interface=dnsmasq*
cache-size=1000
# dot.libredns.gr
server=116.202.176.26#853
# ... more DNS servers
/etc/systemd/resolved.conf.d/dnsmasq_only.conf:
[Resolve]
DNS=10.222.222.222
Domains=~.
2 Likes