Qubes OS 4.2 nftables / nft firewall guide

Hello, I’m desperately trying to configure nftables and I would really appreciate your help

I’m trying to setup VNC connection via LAN with another physical debian device

It was pretty simple with iptables:

VNC server on physical device → AppVM with VNC client → netVM which sets LAN connection → sys-firewall

  1. connect both devices with LAN cable
  2. connect ethernet controller to netVM
  3. enable options “provide network” & “network-manager”
  4. create ethernet connection in network-manager, IPv4 Settings - Share to other computers
  5. check if it works on physical device by command “ip a” and try to use browser → if everything is ok, you now have an internal ip address & internet works in browser

First problem appeared on this stage. It worked flawlessly with iptables (you can see I don’t even interact with it now), but it simply doesnt work with nftables.

However, I was able to solve it by doing this:

nft flush ruleset

table ip filter {
	chain output {
		type filter hook output priority 100; policy accept;
	}

	chain input {
		type filter hook input priority 100; policy accept;
	}

	chain forward {
		type filter hook forward priority 100; policy accept;
	}
}

That is absolutely not elegant solution, but it works. I would like to hear what is the right approach for it.

Next problem:

Now in netVM I would use a single iptables rule:

iptables -I FORWARD -i vif+ -o *ethernet controller name* -j ACCEPT

Then I would simply go to AppVM and ssh to physical device with command:

ssh *internal ip of physical device* -L 9901:localhost:5901

And that’s it. Then I could use VNC client and finish my setup.

So, now I’ve tried to use iptables-translate and got this:

insert rule ip filter FORWARD iifname "vif*" oifname "ethernet controller name" counter accept

Unfortunately, it doesnt work and ssh simply gives up on connection with “connection timed out”.

I’m stuck on this stage because nftables is too hard for me (I was barely able to use iptables), I have zero idea how Qubes firewall works now and honestly very disappointed with this decision.

I don’t understand where the VNC server is running, in a qube or on another computer not running qubes os ?

why don’t you use the default sys-net qube for networking?

if I understand, you use ssh from an AppVM to connect to the remote VNC server, not running Qubes OS, to make a tunnel to connect to VNC in the AppVM? If so, you should not have to do anything with nftables because you have no inbound connection within Qubes OS.

1 Like

That is absolutely not elegant solution, but it works. I would like to hear what is the right approach for it.

The right approach is to clarify the actual goal first, then proceed from there.

What you show allows all connections on the input, forward and output hooks for IPv4. Try setting policies to drop and add log prefix "Dropped in <chainname>: " in each chain. Run journalct -kf in a console in your firewall qube and watch what happens when you try to connect. That will give you an idea what you need to allow selectively.

Check also:

https://wiki.nftables.org/wiki-nftables/index.php/Ruleset_debug/tracing

I’m stuck on this stage because nftables is too hard for me (I was barely able to use iptables), I have zero idea how Qubes firewall works now and honestly very disappointed with this decision.

Networking is a huge subject. Nobody can grasp it in a few minutes. Learning step by step is the only way.

Qubes firewall is a set of tables and chains. You should not modify anything you don’t understand. Chains custom-input and custom-forward are the ones where you generally put your own rules.

3 Likes

Thank you for your answer!

I don’t understand where the VNC server is running, in a qube or on another computer not running qubes os ?

VNC server is running on another non-qubes computer

why don’t you use the default sys-net qube for networking?

I guess it might increase security somehow and it really shouldn’t be an issue

if I understand, you use ssh from an AppVM to connect to the remote VNC server, not running Qubes OS, to make a tunnel to connect to VNC in the AppVM?

Right, however that remote VNC server is in the same network as AppVM (via LAN).

If so, you should not have to do anything with nftables because you have no inbound connection within Qubes OS.

Well, I’ve found a solution directly with nftables.

All I had to do is:

nft add rule ip qubes input iifname “ethernet controller name" accept

and to delete one string from table created by Network Manager:

table ip nm-shared-*ethernet controller name* {

chain filter_forward {
		oifname “ethernet controller name" reject – **delete this**

That is all, now my setup is working as usual. That might be useful for anyone who wants to set up LAN connection in Qubes OS.

Anyways, now I’m stuck at another problem and would appreciate any help:

I’m using redsocks and have no problem making it work inside netVM. However, I also want to route all traffic of AppVM connected to that netVM via redsocks.

Previously, I would use this:

# Any tcp incomming connection on vif+ should be redirected to REDSOCKS chain
sudo iptables -t nat -A PREROUTING --in-interface vif+ -p tcp -j REDSOCKS

# Any incomming tcp connection from port 12345 to vif+ should be accepted
iptables -I INPUT -i vif+ -p tcp --dport 12345 -j ACCEPT

Now I’ve tried to create chain prerouting in my custom table:

table ip nat {
	chain REDSOCKS {
		# hook to the output
		type nat hook output priority 0; policy accept;
		# skip if the user is not uid 1000
		ip protocol tcp skuid != 1000 return
		# skip for local ip ranges
		ip daddr 0.0.0.0/8      return
		ip daddr 10.0.0.0/8     return
		ip daddr 100.64.0.0/10  return
		ip daddr 127.0.0.0/8    return
		ip daddr 169.254.0.0/16 return
		ip daddr 172.16.0.0/12  return
		ip daddr 192.168.0.0/16 return
		ip daddr 198.18.0.0/15  return
		ip daddr 224.0.0.0/4    return
		ip daddr 240.0.0.0/4    return
		# everything else tcp = redirect to redsocks
		ip protocol tcp redirect to 12345
	}
	chain prerouting {
		# hook to the output
                type filter hook prerouting priority raw; policy accept;
                iifname "vif+" ip protocol tcp counter jump REDSOCKS
        }
}

However, I get this error - Could not process rule: Operation not supported with REDSOCKS marked as incorrect part of the string.

As per nftables wiki,

only jump and goto actions to regular chains are allowed.

Regular chains = chains without hook string. REDSOCKS is not a regular chain.

The problem is that I can’t delete this hook in REDSOCKS - type nat hook output priority 0; policy accept; because it breaks redsocks and even then I’m still getting the same error. I have no idea how to do this without jump, but I can’t use it.

What is the right way to adapt this to nftables?

I don’t understand why did you have any problem with outgoing connection from your AppVM to the VNC server in LAN and why was it necessary to allow incoming connections from your LAN.

Regarding your problem with redsocks, I can suggest you to take a look at sing-box instead of redsocks.

As per nftables wiki,

only jump and goto actions to regular chains are allowed.

Regular chains = chains without hook string. REDSOCKS is not a regular chain.

The problem is that I can’t delete this hook in REDSOCKS - type nat hook output priority 0; policy accept; because it breaks redsocks and even then I’m still getting the same error. I have no idea how to do this without jump, but I can’t use it.

What is the right way to adapt this to nftables?

You can add another chain and have both prerouting and REDSOCKS jump to it, e.g.

table ip nat {
	chain REDSOCKS {
		# hook to the output
		type nat hook output priority 0; policy accept;
		jump @another
	}
	chain prerouting {
		# hook to the output
		type filter hook prerouting priority raw; policy accept;
		iifname "vif+" ip protocol tcp counter jump @another
	}
	chain another {
		# skip if the user is not uid 1000
		ip protocol tcp skuid != 1000 return
		# skip for local ip ranges
		ip daddr {
			0.0.0.0/8
			10.0.0.0/8
			100.64.0.0/10
			127.0.0.0/8
			169.254.0.0/16
			172.16.0.0/12
			192.168.0.0/16
			198.18.0.0/15
			224.0.0.0/4
			240.0.0.0/4
		} return
		# everything else tcp = redirect to redsocks
		ip protocol tcp redirect to 12345
	}
}

I have also optimized it a little. You can improve it further with a named set (see wiki). Instead of iifname "vif+" you can use iifgroup 2 (check the output of ip a).

Thanks @qubist

I had to return local ip ranges to its previous state, because it would give me an error - Error: syntax error, unexpected / for all these addresses. Also had to remove @ in jump @another for the same reason

However, this setup doesn’t work - Error: Could not process rule: Opeartion not supported ip protocol tcp redirect to 12345

As far as I understand, chain can’t use this rule without having an appropriate hook. I’ve tried to move this rule to chain REDSOCKS, but it doesn’t work. When I set a hook to chain another, it gives me an error which we started from - can’t jump to chain with hook string.

My bad. I forgot the commas in the set.
Try this:

#...
		ip daddr {
			0.0.0.0/8,
			10.0.0.0/8,
			100.64.0.0/10,
			127.0.0.0/8,
			169.254.0.0/16,
			172.16.0.0/12,
			192.168.0.0/16,
			198.18.0.0/15,
			224.0.0.0/4,
			240.0.0.0/4
		} return
#...

Of course, the @ is bad syntax too. I guess my workday has been too long and my attention is slipping away. Sorry about that.

Try moving the redirect rule to the prerouting chain (after the jump). If it still doesn’t work for some reason (which I am unable to test right now), check the wiki:

https://wiki.nftables.org/wiki-nftables/index.php/Performing_Network_Address_Translation_(NAT)#Redirect

Unfortunately, the same error when I move it to prerouting chain. Checking the wiki.

This gives no error for me:

#!/usr/sbin/nft -f

table ip nat {
        set local_ip_ranges {
                typeof ip daddr
                flags interval
                auto-merge
                elements = {
                        0.0.0.0/8,
                        10.0.0.0/8,
                        100.64.0.0/10,
                        127.0.0.0/8,
                        169.254.0.0/16,
                        172.16.0.0/12,
                        192.168.0.0/16,
                        198.18.0.0/15,
                        224.0.0.0/4,
                        240.0.0.0/4
                }
        }

        chain another {
                # skip if the user is not uid 1000
                ip protocol tcp skuid != 1000 return
                # skip for local ip ranges
                ip daddr @local_ip_ranges return
        }

        chain REDSOCKS {
                # hook to the output
                type nat hook output priority 0; policy accept;
                jump another
                # everything else tcp = redirect to redsocks
                ip protocol tcp redirect to 12345
        }

        chain prerouting {
                # hook to the output
                type filter hook prerouting priority raw; policy accept;
                iifgroup 2 ip protocol tcp counter jump another
        }

I don’t know what you are actually trying to do with this but it doesn’t do much per se. ‘another’ chain has no effect whatsoever - it will return with or without those rules. The prerouting chain does not restrict traffic either. The only rule that does something meaningful is the redirect. IOW, you can reduce the whole thing to:

table ip nat {
        chain REDSOCKS {
                type nat hook output priority 0; policy accept;
                ip protocol tcp redirect to 12345
        }
}