Qubes OS 4.2 nftables / nft firewall guide

Hi!

I started fiddling with NFT on Qubes OS 4.2 and the official documentation is quite bad in this area. Let’s gather material here so we could enhance the documentation once we have enough content.

Port forwarding from outside to a qube

I got port forwarding to work, that wasn’t easy as the default ruleset didn’t have a chain for the nat rules. I made a simple script where you just have to fill the port and the destination.

Typically, you would do:

  • run the snippet Forwarding in sys-net using sys-firewall IP as a destination
  • run the snippet Forwarding in sys-firewall using the qube IP as a destination
  • run the snippet Accept only in the destination qube

This is an example for a TCP redirection, for UDP you would have to replace tcp by udp.

Forwarding

#!/bin/sh

PORT=8000
DESTINATION=10.138.0.52

if ! nft -nn list table ip qubes | grep "chain nat {" ; then
    nft add chain qubes nat { type nat hook prerouting priority dstnat\; }
fi
nft add rule qubes custom-input tcp dport "${PORT}" accept
nft add rule qubes custom-forward tcp dport "${PORT}" accept
nft add rule qubes nat iifname != "vif*" tcp dport "${PORT}" dnat "${DESTINATION}"

Accept only

#!/bin/sh

PORT=8000

nft add rule qubes custom-input tcp dport "${PORT}" accept
10 Likes

I tried to run chiaki (Remote Play program for a Playstation 5 or 4) from a qube, works great :+1:

Lot of firewall required, that was a good real world problem to solve :smiley:

1 Like

What kind of firewall rules are missing in their nftables equivalent?

if I want to connect qube1 and qube2, will this work by running forwarding in qube1, and accept in qube2?

i don’t need port forwarding to outside, but between my qubes.

also, does it work for standalone qubes as well? thanks :smile:

I didn’t try yet, but from the current documentation, that the same steps except:

  • you would need to forward only for a source address (otherwise all the other qubes would be allowed to reach that qube) AND only forward for internal qubes interfaces, so the rule would be a bit different, maybe something like this
#!/bin/sh

# allows qube $SOURCE to reach $DESTINATION on port $PORT
PORT=8000
DESTINATION=10.137.0.16
SOURCE=10.137.0.15

nft add rule qubes custom-forward ip saddr "${SOURCE}" tcp dport "${PORT}" accept
  • you don’t need to do anything in sys-net

It’s working fine with the script above on sys-firewall :ok_hand:

In the destination qube, don’t forget to allow the port using nft add rule qubes custom-input tcp dport 8000 accept

Ah, this makes me want to install 4.2 :slight_smile:
I think that the rules can be simpler, if you just want to access qubeX from qubeY, and you know the IPs (which you do, because you have them in the snippets). No NAT is required in this case, just forward.
But I don’t have 4.2 (to test) and I don’t know exactly the intricacies of the nft qubes setup :man_shrugging:

Be my guest if you can improve the rules! I’m quite new to nftables, I used it a short time on my router, but I didn’t have a complex setup like in Qubes OS firewall… :frowning:

Ideally, I’d like to provide a script to have in dom0 that would on which you give the ports you want and the destination name, dom0 has all the information to figure the IP from the name and know the network of each qube.

@barto you were right, only the custom-input rule is required, I updated the script :ok_hand:

2 Likes

Understood. I may have an old laptop spare (dead battery but 32G of RAM) to test 4.2rc3 :thinking:
Got to schedule a trip to the storage facility … I mean the shed in the backyard :slight_smile:

1 Like

Thanks for your experimentation!

I stumble on that problem when trying to use @unman 's script to do port forwarding… It tries to detect whether the AppVM has nft. And wrongly chooses iptables because the test fails.

It’d be nice to have a fusion of your 4.2 scripts with what in.sh provides. But the latter needs some cleaning and linting (unused variables, etc.) :smiling_face_with_three_hearts:

Hi Solene, I like you was pushed onto nftables a couple of weeks ago by 4.2. Was not nearly as bad as I was expecting. I consider myself an intermediate level user so not really qualified to update the doco. I do notice that the doco on 4.0 has for years said that one needs to add nft rules… I quickly learnt that nothing goes thru them - all is done in iptables.

On your top post - I don’t think you should have an input chain entry when forwarding as the packet has to go to either the local host or on forward. Forwarding must have higher priority for that to work.

In a qube providing networking your rules should be in qubes-firewall-user-script and that should be able to be re-run without a problem, and it is nice to be able to edit it and re-run just that script when changes are required - not have to restart the qube. Your script will leave multiple copies and if ports are changed might not work…

As a PSA here is part of my sys-firewall user script. Happy to hear someone poke holes to get it fixed!

SSH=10.137.0.28
NFS=10.137.0.28
LAN=192.168.0.0/16

NFS_PORTS=sunrpc,nfs
BIOS_PORTS=netbios-ns,netbios-dgm,netbios-ssn	# For Samba

TCP_PORTS=$NFS_PORTS,$BIOS_PORTS,ssh	# All incoming
UDP_PORTS=$NFS_PORTS,$BIOS_PORTS

nft flush chain ip qubes custom-forward
nft add rule ip qubes custom-forward tcp dport { $TCP_PORTS } counter accept 
nft add rule ip qubes custom-forward udp dport { $UDP_PORTS } counter accept 

if nft list table ip qubes | grep custom-dnat >/dev/null; then nft delete chain ip qubes custom-dnat; fi
nft add chain ip qubes custom-dnat { type nat hook prerouting priority dstnat\; policy accept\; }
nft add rule  ip qubes custom-dnat   ip saddr $LAN tcp dport { $NFS_PORTS, $BIOS_PORTS } counter dnat to $NFS
nft add rule  ip qubes custom-dnat   ip saddr $LAN udp dport { $NFS_PORTS, $BIOS_PORTS } counter dnat to $NFS
nft add rule  ip qubes custom-dnat   ip saddr $LAN tcp dport ssh counter dnat to $SSH

I really find it difficult to believe that there is no option to nft to test run a command and do nothing, just return the exit code. Really having to list and fire up a grep just to test is very expensive. Should be put on Pablo’s todo list!

I opened a PR to update the Firewall guide to switch it to nftables Switch the Firewall guide to nftables by rapenne-s · Pull Request #1344 · QubesOS/qubes-doc · GitHub (I’m being sponsored with Open Collective for this work)

6 Likes

Well, in this particular case you don’t need the grep at all! You just want to delete the chain called “custom-dnat”, if it exists, so you can run:

nft delete chain ip qubes custom-dnat &>/dev/null

… and Bob’s your uncle!

Pedantic Note: the “&>/dev/null” redirection used above works only in bash/zsh; for more generic POSIX compatibility, use “>/dev/null 2>&1”.

In fact, there is no real need to delete it before adding it again. If you run the script twice, as the chain already exist it won’t be created again.

However, it may be good to use this pattern:

if nft create chain...
then
  flush chain
  add rules...
fi

I didn’t comment on that part… just on the fact that if he’s going to delete the chain if the chain exists, why not blindly delete the chain and ignore the output. :slight_smile:

1 Like

@barto did you test that? I found that nft always outputs its user friendly error messages whatever you do with fds on invocation. I guessed it must reopen its controlling terminal to output, and you cannot make it quiet.

I did not check whether create would say something if the chain already existed… I expect it would.

We really need a set of examples in the doco to get everyone started. Would have been a piece of cake if I could have seen some relevant examples.

Aside on my code clip above, one does not need UDP ports for NFS (infact it is by default disabled and one cannot use UDP with Gb ethernet reliably).

Never mind 2>&1 does that same as &> in bash.

That is exactly what I tried… now it seems to work?? Very odd. Plenty of other places I saw the list and grep used by others as well…

Seems in Debian-12 “2>&1 >/dev/null” does not work, but “>/dev/null 2>&1” does - both work in Fedora-38?

It’s always command > file 2>&1

If you do command 2>&1 > file the redirects are done before stdout is redirect to the file, and it doesn’t work.

1 Like