Need help with port forwarding. Doing everything as it says in the docs

User Support
1

I need help with this. I’m doing everything as it says in the docs, but I can’t make it work.

sys-net:

wls6 UP 192.168.0.16/24
vif60.0 UP 10.138.19.200/32
sys-firewall:
eth0 UP 10.138.18.170/32
vif61.0 UP 10.138.18.170/32
vif63.0 UP 10.138.18.170/32
nginx:
eth0 UP 10.137.0.27/32

I applied the following to sys-net’s qubes-firewall-user-script and checked that it persist:

if nft add chain qubes custom-dnat-nginx ‘{ type nat hook prerouting priority filter +1 ; policy accept; }’
then

nft add rule qubes custom-dnat-nginx iif == “wls6” ip saddr 192.168.0.0/24 tcp dport 80 ct state new,established,related counter dnat 10.138.18.170

nft add rule qubes custom-forward iif == “wls6” ip saddr 192.168.0.0/24 ip daddr 10.138.18.170 tcp dport 80 ct state new,established,related counter accept
fi

sys-firewall’s qubes-firewall-user-script:
if nft add chain qubes custom-dnat-nginx ‘{ type nat hook prerouting priority filter +1 ; policy accept; }’
then

nft add rule qubes custom-dnat-nginx iif == “eth0” ip saddr 192.168.0.0/24 tcp dport 80 ct state new,established,related counter dnat 10.137.0.27

nft add rule qubes custom-forward iif == “eth0” ip saddr 192.168.0.0/24 ip daddr 10.137.0.27 tcp dport 80 ct state new,established,related counter accept
fi

in my nginx’s rc.local:
nft add rule qubes custom-input tcp dport 80 ip daddr 10.137.0.27 ct state new,established,related counter accept

in the nginx qube I am able to http://localhost, but when I hit my public IP(say 72.72.72.72) it just hangs.

2

what if you try to open http://192.168.0.16 from your lan?

I wonder if the daddr part is useful, this may be creating problems here

in the nginx qube, if you run sudo tcpdump -i eth0 -nn tcp port 80 do you see lines appearing when trying to connect? From sys-firewall, you can try to connect with curl http://10.137.0.27

3

I can’t.
curl http://192.168.0.16
curl: (7) Failed to connect to 192.168.0.16 port 80 after 1 ms: Couldn’t connect to server

What should I write instead?

Only if I use curl 10.137.0.27 from the sys-firewall, but not from sys-net.
for the curl 192.168.0.16 I don’t see anything.
I can get results when I use curl 10.137.0.27 from sys-firewall, but not sys-net for some reason

4

this mean the rule in nginx qube is good

from sys-net, you need to query sys-firewall IP using curl http://10.138.18.170

5

I made a script to easily NAT a port and chain it through all netvm up to sys-net, [Qubes OS 4.2] Easily NAT qubes port to external network

You could easily modify it to generate and display the rules instead of running them in each qubes

6

It just hangs when I do that

7

That’s cool, but I want to learn by doing it myself first. Meanwhile as you can see I’m having issues with simple things like that, despite following the docs.

8

instead of this

try, this, without the source address. sys-net will not come with a 192.168.0.0/24 address

nft add rule qubes custom-dnat-nginx iif == “eth0” tcp dport 80 ct state new,established,related counter dnat 10.137.0.27
9

It may also be that “adding” rules may not work, because they are added after the catch-all “drop”. Can’t check now, but it’s worth trying to “insert” the rules instead.

10

Same thing, hangs on curl http://10.138.18.170 from sys-net.

I’m gonna check this now

11

Same thing.

12

Can someone please test this with the same rules and tell me if it’s working. I have a feeling it’s not only me

13

I daily use port forwarding to my qubes and it’s working fine.

I’m using this to redirect traffic to a VPN server to access my sys-gui-vnc from the outside through a VPN (it’s called by qubes rpc)

cat <<EOF | qvm-run -u root --pass-io "sys-firewall" "/bin/sh"
if ! nft -nn list table ip qubes | grep "chain nat {" >/dev/null ; then
    nft add chain qubes nat { type nat hook prerouting priority dstnat\; }
fi

nft insert rule qubes custom-input "udp" dport "7008" accept
nft insert rule qubes custom-forward "udp" dport "7008" accept
nft add rule qubes nat iifname != \"vif*\" "udp" dport "7008" dnat "10.137.0.59"
EOF

cat <<EOF | qvm-run -u root --pass-io "sys-net" "/bin/sh"
if ! nft -nn list table ip qubes | grep "chain nat {" >/dev/null ; then
    nft add chain qubes nat { type nat hook prerouting priority dstnat\; }
fi

nft insert rule qubes custom-input "udp" dport "7008" accept
nft insert rule qubes custom-forward "udp" dport "7008" accept
nft add rule qubes nat iifname != \"vif*\" "udp" dport "7008" dnat "10.138.13.219"
EOF

cat <<EOF | qvm-run -u root --pass-io "vpn-vnc" "/bin/sh"
nft add rule qubes custom-input "udp" dport "7008" accept
EOF
1 Like
14

I appreciate you sharing this, but I want to stick with the docs.

15

sure :+1:

make sure that you reset all your changes before trying anything new, otherwise you won’t be sure you are trying the new ruleset.

1 Like
16

can you share the output of this on sys-net?

sudo nft list chain ip qubes custom-dnat-nginx

The counters will allow to see if at least something is going through the rules.

I wrote the forwarding documentation, I tested it during the writing, I really want it to work for readers :sweat_smile:

17

That’s what I get:

table ip qubes {
        chain custom-dnat-nginx {
                type nat hook prerouting priority filter + 1; policy accept;
        }
}
18

hmm, you should have something like this in the chain :thinking:

iif == “wls6” ip saddr 192.168.0.0/24 tcp dport 80 ct state new,established,related counter dnat 10.138.18.170

what about nft list ruleset | grep 10.138.18.170 ?

19

So am I doing something wrong, is the documentation isn’t complete?

This is what I got in sys-net, and yeah it gets cut out as in here:

                elements = { 10.138.18.170 }
                elements = { "vif77.0" . 10.138.18.170 }
                ip saddr 10.138.18.170 jump qbs-10-138-18-170
                iifname != "vif*" ip saddr { 10.138.18.170, 10.138.20.195 } drop
                oifname != "vif*" ip daddr { 10.138.18.170, 10.138.20.195 } drop
        chain qbs-10-138-18-170 {
20

If the following code had worked, you should see something in the previous output. Could you run it again and paste the output of the command, if any?

nft add rule qubes custom-dnat-nginx iif == “wls6” ip saddr 192.168.0.0/24 tcp dport 80 ct state new,established,related counter dnat 10.138.18.170

nft add rule qubes custom-forward iif == “wls6” ip saddr 192.168.0.0/24 ip daddr 10.138.18.170 tcp dport 80 ct state new,established,related counter accept
fi