I’ll primarily answer your first question for information purposes - how to get logs of blocked connections by qubes-firewall, then follow up with some additional notes regarding the debugging you’ve already done.
The qubes-firewall uses nftables to configure allowed and denied hosts. What you can do is modify a nft rule to add logging which will go to your kernel log (dmesg or journalctl).
-
sudo nft list ruleset -a
-
Look for the handle number in the ‘reject with icmp type admin-prohibited’ of the right chain
-
Replace the rejection rule with a counter + log + reject statement
There is an ipv4 qubes-firewall table, a forward chain, and the forward chain references chains specific to each downstream qube connected to it. There’s also an ipv6 qubes-firewall table…but to keep things simple I’ll leave it out. If you have ipv6 enabled in your environment, ensure both address families are accounted for.
If I have a qube connected to sys-firewall named “onlycloudflare” and I want it to only have access to IP address 1.1.1.1, I could do the following to get more debugging info out:
Showing the current chains:
sudo nft list -a table qubes-firewall
Will show something like this, where qbs-10-138-26-180 is the “onlycloudflare” qube’s IP address:
table ip qubes-firewall {
chain forward {
type filter hook forward priority filter; policy drop;
ct state established,related accept
iifname != "vif*" accept
ip saddr 10.138.26.180 jump qbs-10-138-26-180
}
chain qbs-10-138-26-180 {
ip daddr 1.1.1.1 accept # handle 10
ip daddr { 10.139.1.1-10.139.1.2 } tcp dport 53 accept # handle 11
ip daddr { 10.139.1.1-10.139.1.2 } udp dport 53 accept # handle 12
ip protocol icmp accept # handle 13
reject with icmp type admin-prohibited # handle 14
reject with icmp type admin-prohibited # handle 15
}
}
Replace the rule with handle 14 to add logging and a counter:
sudo nft replace rule qubes-firewall qbs-10-138-26-180 handle 14 counter log prefix \"[qubes-firewall-BLOCKED] \" reject with icmp type admin-prohibited
If you want stats and logs for all initial packets (using the ‘info’ level; by default it uses ‘warn’ because nobody logs everything),
sudo nft insert rule qubes-firewall qbs-10-138-26-180 counter log prefix \"[qubes-firewall-eval] \" level info
This will now have rules such as:
chain qbs-10-138-26-180 {
counter packets 0 bytes 0 log prefix "[qubes-firewall-eval] " level info # handle 42
ip daddr 1.1.1.1 accept # handle 10
ip daddr { 10.139.1.1-10.139.1.2 } tcp dport 53 accept # handle 11
ip daddr { 10.139.1.1-10.139.1.2 } udp dport 53 accept # handle 12
ip protocol icmp accept # handle 13
counter packets 0 bytes 0 log prefix "[qubes-firewall-BLOCKED] " reject with icmp type admin-prohibited # handle 14
reject with icmp type admin-prohibited # handle 15
}
In sys-firewall, await kernel logs with: sudo dmesg -w
In “onlycloudflare” qube, run wget commands to allowed and denied addresses:
wget https://1.1.1.2
Logs should arise:
[689.942410] [qubes-firewall-eval] IN=vif31.0 OUT=eth0 MAC=fe:ff:ff:ff:ff:ff:00:16:3e:5e:6c:00:08:00 SRC=10.138.26.180 DST=1.1.1.2 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=54098 DF PROTO=TCP SPT=57174 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
[689.942530] [qubes-firewall-BLOCKED]: IN=vif31.0 OUT=eth0 MAC=fe:ff:ff:ff:ff:ff:00:16:3e:5e:6c:00:08:00 SRC=10.138.26.180 DST=1.1.1.2 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=54098 DF PROTO=TCP SPT=57174 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Something not blocked will not have a corresponding qubes-firewall-BLOCKED printk:
wget https://1.1.1.1
[704.271777] [qubes-firewall-eval] IN=vif31.0 OUT=eth0 MAC=fe:ff:ff:ff:ff:ff:00:16:3e:5e:6c:00:08:00 SRC=10.138.26.180 DST=1.1.1.1 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=43637 DF PROTO=TCP SPT=35744 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Showing the counters you can manually diff packets that came in (rule 1) and were blocked (before the ‘reject’) - this shows of 2 initial packets to be forwarded, 1 was blocked:
chain qbs-10-138-26-180 {
counter packets 2 bytes 104 log prefix "[qubes-firewall-eval] "
ip daddr 1.1.1.1 accept
ip daddr { 10.139.1.1-10.139.1.2 } tcp dport 53 accept
ip daddr { 10.139.1.1-10.139.1.2 } udp dport 53 accept
ip protocol icmp accept
counter packets 1 bytes 52 log prefix "[qubes-firewall-BLOCKED]: " reject with icmp type admin-prohibited
reject with icmp type admin-prohibited
}
Now to provide insight to your earlier debugging:
-
qubes-iptables is a service that establishes the baseline iptables rules for all Qubes
-
qubes-firewall is a service that runs additional user-specified custom firewall rules at FirewallVM boot, as well as dynamically activating firewall rules for attached downstream qubes.
-
sys-firewall acts as a router; you will not see IP connections that are being forwarded on behalf of another qube with netstat. If you want to see those, you want to look into conntrack table - sudo cat /proc/net/nf_conntrack or sudo conntrack -L from conntrack-tools package in Fedora.
-
netstat is typically already available in Windows, netstat /an
- The
journalctl -u qubes-firewall output tells you the real reason why nothing worked: “Failed to resolve windowsupdate.com…blocking traffic”…while the second-level domain name is valid, there is no IP address associated with the windowsupdate.com. The WSUS document specifies *.windowsupdate.com - an host name such as this is not possible to specify using the Qubes Firewall - you must know each individual domain. Some additional notes on the qubes firewall: How is the QubesOS firewall implemented? - #13
Overall, @ddevz has provided valuable input and I agree with the advice - use a proxy…because filtering by IP address gets very messy if the hosts utilize load balancing and CNAME masquerading with very tight DNS TTLs.
One such mechanism (using tinyproxy, something already available in QubesOS by default) was discussed in a guide devised by @Rooftop: Restricting a Qube to selected websites