Configuring a ProxyVM VPN Gateway

[solene]

@smlt

insert rule ip nat dnat-dns iifname vif* udp dport 53 counter dnat to $addr
                                                                       ^^^^

it looks like you tried iptables-translate on a line with a variable, it doesn’t know what to do with $addr

[smlt]

i made it work, but now the problem is i have connection inside the appvm–vpn but cant use it as proxy to another qube… here is my topic

[annulet]

Regarding anti-leak rules:
In a netvm, I normally would have expected FORWARD rules instead of OUTPUT, as it basically is a router and forwards packets from a client qube to upstream.

Please confirm if following is right (thanks in advance!):
The OpenVPN client captures all traffic from all incoming network interfaces via

redirect-gateway def1

, so all relevant traffic belongs to OpenVPN client. And since OpenVPN is a local process, all network calls go through OUTPUT. NetworkManager analogue, but doesn’t need above change in .ovpn file. Seems it automatically grabs all incoming traffic and redirects it to VPN.

[apparatus]

With redirect-gateway def1 all traffic is redirected to the VPN only if the openvpn is connected. If the VPN is disconnected then all the traffic originating from sys-vpn will go through clearnet.
These rules block all outgoing traffic to eth0 that is not originating from openvpn process:

@taradiddles

#    Accept traffic to VPN
iptables -P OUTPUT ACCEPT
iptables -F OUTPUT

#    Add the `qvpn` group to system, if it doesn't already exist
if ! grep -q "^qvpn:" /etc/group ; then
     groupadd -rf qvpn
     sync
fi
sleep 2s

#    Block non-VPN traffic to clearnet
iptables -I OUTPUT -o eth0 -j DROP
#    Allow traffic from the `qvpn` group to the uplink interface (eth0);
#    Our VPN client will run with group `qvpn`.
iptables -I OUTPUT -p all -o eth0 -m owner --gid-owner qvpn -j ACCEPT

For example, if you have updates check enabled for your sys-vpn qube and the updates check will begin when VPN is connected, then VPN will disconnect during updates check and updates check will continue through clearnet this would be a leak.

[annulet]

Thanks @apparatus! Your answer makes it very clear. Good point with the background updates.

---

Update: I think, implementing a killswitch for NetworkManager is easier by whitelisting the OpenVPN/Wireguard network interface:

nft insert rule ip qubes output oifname 'wg*' counter accept

(Otherwise NetworkManager would need to be started as qvpn user somehow.)

Here is what I am doing:

# block forward hook as default
nft 'add chain ip qubes forward { counter; policy drop; }'
nft insert rule ip qubes custom-forward counter drop

# block output hook as default
nft 'add chain ip qubes output { type filter hook output priority 0; counter; policy drop; }'
nft insert rule ip qubes output counter drop

# allow establishing initial connection to VPN gateway (wireguard/udp)
nft insert rule ip qubes output ip daddr <VPN GATEWAY IP> udp dport <VPN GATEWAY PORT> accept

# allow outgoing traffic through link wg*
nft insert rule ip qubes output oifname 'wg*' counter accept

This assumes, there is one active WG connection at a time.

[annulet]

Hm there is one thing left, I don’t understand: Fully blocking all forward connections does not work. I could ping 1.1.1.1 from within sys-vpn, but not from client qube.

Instead of

# block forward hook as default
nft 'add chain ip qubes forward { counter; policy drop; }'
nft insert rule ip qubes custom-forward counter drop

, I needed to write:

nft insert rule ip qubes custom-forward oifname eth0 counter drop

and leave default forward policy accepted to get pings from clients.

Is this some weirdness of the VPN client (NetworkManager, OpenVPN)?
It seems to intercept and redirect client qube traffic to tunnel interface wg at some point before the killswitch eth0 drop rule could kick in. But still forwarding seems needed. Not sure about the internals, how exactly traffic is re-routed by these programs…

---

(tldr) Never mind: VPN clients probably just change the default IP route (+ default DNS). So we get wg (or what it’s named) instead of eth0 as output interface in routine table. nfttables does the same thing as before, meaning packet still needs accept to be forwarded. eth0 can be safely blacklisted, as not used during VPN session.

What confused me is that ip route entries are not changed, after NetworkManager has initiatiated VPN connection. I still see only the Qubes default gateway. Nameserver is changed though, looking at /etc/resolv.conf.

[apparatus]

@annulet

(Otherwise NetworkManager would need to be started as qvpn user somehow.)

You can set the group in the OpenVPN config.
But it won’t work for Wireguard.

@annulet

Here is what I am doing:

# block forward hook as default
nft 'add chain ip qubes forward { counter; policy drop; }'
nft insert rule ip qubes custom-forward counter drop

# block output hook as default
nft 'add chain ip qubes output { type filter hook output priority 0; counter; policy drop; }'
nft insert rule ip qubes output counter drop

# allow establishing initial connection to VPN gateway (wireguard/udp)
nft insert rule ip qubes output ip daddr <VPN GATEWAY IP> udp dport <VPN GATEWAY PORT> accept

# allow outgoing traffic through link wg*
nft insert rule ip qubes output oifname 'wg*' counter accept

This assumes, there is one active WG connection at a time.

In some cases you may need to allow the output connections from lo and vif+ interfaces.

@annulet

What confused me is that ip route entries are not changed, after NetworkManager has initiatiated VPN connection. I still see only the Qubes default gateway. Nameserver is changed though, looking at /etc/resolv.conf.

The Wireguard creates separate routing table.
You can show all tables with this command:

ip rule show

And check all routes in all tables using this command:

ip route show table all

[heinzl]

Hello,
I have several problems with the “iptables and CLI scripts method”. On my other laptop with an older Qubes OS I have no problems get the VPN connection work but now I have the following:
I came until 4. and wanted to test the connection and started OpenVPN in the terminal. The process is stopped due to fatal error:

“qubes-vpn-handler.sh: line 19: iptables: command not found
2024-08-17 12:22:41 WARNING: Failed running command (–up/–down): could not execute external program
2024-08-17 12:22:41 Exiting due to fatal error”

I copied the code, why he cannot find “iptables”?

Thanks.

[apparatus]

The guide is for Qubes OS 4.1 so it’s using iptables firewall.
In Qubes OS 4.2 the iptables firewall was replaced with nftables firewall.
To use this guide in Qubes OS 4.2 you need to replace the firewall rules from iptables to nftables in /rw/config/vpn/qubes-vpn-handler.sh and /rw/config/qubes-firewall-user-script files.
You can use the files from this post:

Configuring a ProxyVM VPN Gateway

No problem, below is a copy/paste from my vpn qube.

`/rw/config/vpn/qubes-vpn-handler.sh`

#!/bin/bash set -e export PATH="$PATH:/usr/sbin:/sbin" case "$1" in up) # To override DHCP DNS, assign DNS addresses to 'vpn_dns' env variable before calling this script; # Format is 'X.X.X.X Y.Y.Y.Y [...]' if [[ -z "$vpn_dns" ]] ; then # Parses DHCP foreign_option_* vars to automatically set DNS address translation: for optionname in ${!foreign_opti…

[heinzl]

Thank you very much for your help. After having some trouble it seems now to work but I have three more questions:

1. In the original qubes-firewall-user-script you can read:
   "# This script is called at AppVM boot if this AppVM has the qubes-firewall

service enabled."

But in the guide I can read nothing about that. So, do I have to enable this service or not to prevent leaks?

2. After having some troubles and finally had no error messages anymore when openvpn is running, I tried in the VPN-qube to open a website via Firefox but it didn’t work and there was only a blanket site and was loading all the time. As I stopped openvpn the website loaded with a TOR IP. BUT as I assign the VPN-qube to another AppVM, it worked and I reached a website with the VPN IP. Why is that, should I not also reach a website withing the VPN-qube with the vpn connection?

3. I would like to know, if it is possible to have multiple config files in the folder and make openvpn randomly use one everytime it starts?

Thanks again.

[apparatus]

@heinzl

1. In the original qubes-firewall-user-script you can read:
   "# This script is called at AppVM boot if this AppVM has the qubes-firewall

service enabled."

But in the guide I can read nothing about that. So, do I have to enable this service or not to prevent leaks?

It’s just a note for the users to understand in what cases is this file called.
You don’t need to take any additional actions.

@heinzl

After having some troubles and finally had no error messages anymore when openvpn is running, I tried in the VPN-qube to open a website via Firefox but it didn’t work and there was only a blanket site and was loading all the time. As I stopped openvpn the website loaded with a TOR IP. BUT as I assign the VPN-qube to another AppVM, it worked and I reached a website with the VPN IP. Why is that, should I not also reach a website withing the VPN-qube with the vpn connection?

Check your /etc/resolve.conf in VPN qube when you’re connected to the VPN. Most probably is that your VPN qube is not using the DNS server provided by your VPN so the DNS resolution is not working in the VPN qube.

@heinzl

I would like to know, if it is possible to have multiple config files in the folder and make openvpn randomly use one everytime it starts?

Yes, just pick a random file from the group of files in the /rw/config/rc.local instead of openvpn-client.ovpn specifically.
Here is an example:

Wireguard VPN setup

Good news, with fedora-38 the network manager supports Wireguard out of the box! The only thing required are extra firewall rules in the VPN qube, as explained in the community documentation about VPN. What you’ll need This guide assumes you are using a VPN service that has wireguard support, most of them do, but you can also add your own if you have a server. ProtonVPN has a free plan, it has limits but gives you a fully working VPN and they support WireGuard. This pro…

So you can do something like this:

VPN_CONF=$(ls /rw/config/vpn-configs-dir | sort -R | head -n 1)
VPN_OPTIONS='--cd /rw/config/vpn/ --config "vpn-configs-dir/$VPN_CONF" --daemon'

[heinzl]

Thanks, but it seems to be for WireGuard?

“It’s just a note for the users to understand in what cases is this file called.
You don’t need to take any additional actions.”

But I understand that like the code in there will not be executed if I don’t enable the service so I do not have the leak protection?
I am asking my self anyways, why I can open a website with my clear IP if I edited the firewall-script?

[apparatus]

@heinzl

Thanks, but it seems to be for WireGuard?

It doesn’t matter, I’m just referring to the idea of using | sort -R | head -n 1 commands on the list of the VPN config files.

@heinzl

But I understand that like the code in there will not be executed if I don’t enable the service so I do not have the leak protection?

The qubes-firewall service is enabled by default for qubes that provides network for other qubes.

@heinzl

I am asking my self anyways, why I can open a website with my clear IP if I edited the firewall-script?

Describe in more details what do you mean. I didn’t get your question.

[heinzl]

Ok, it is a bit weird for me: I set up the anti-leak rules, then I restarted the VPN-qube (I forgot that before, so don’t mind my question before) and if I then start openvpn manually, it doesn’t work. I start openvpn and it says
“Could not determine IPv4/IPv6 protocol”, “RESOLVE: Cannot resolve host address: XXXXXXX (Temporary failure in name resolution)”, “Restart pause, 64 second(s)” and so on.
But if I start the VPN-qube with configured openvpn autostart, it worked and the link is up and I have the vpn ip with another AppVM. As soon as I kill openvpn and have no restart enabled, I can’t start openvpn anymore, the terminal says that, what I wrote above. Why is that? If it is working on start-up then it should work too if I manually start openvpn with my config right?
And I am not able to get it working AFTER whonix only after sys-firewall. But for some cases I want both: VPN-TOR-VPN because some websites block TOR traffic.

Thank you again.

[apparatus]

@heinzl

Ok, it is a bit weird for me: I set up the anti-leak rules, then I restarted the VPN-qube (I forgot that before, so don’t mind my question before) and if I then start openvpn manually, it doesn’t work. I start openvpn and it says
“Could not determine IPv4/IPv6 protocol”, “RESOLVE: Cannot resolve host address: XXXXXXX (Temporary failure in name resolution)”, “Restart pause, 64 second(s)” and so on.
But if I start the VPN-qube with configured openvpn autostart, it worked and the link is up and I have the vpn ip with another AppVM. As soon as I kill openvpn and have no restart enabled, I can’t start openvpn anymore, the terminal says that, what I wrote above. Why is that? If it is working on start-up then it should work too if I manually start openvpn with my config right?

Did you run it as qvpn group?

sg qvpn -c "$VPN_CLIENT $VPN_OPTIONS"

The output traffic that is coming from other groups are blocked:

@etaz

#    Block non-VPN traffic to clearnet
nft insert rule ip qubes output oifname eth0 counter drop

@heinzl

And I am not able to get it working AFTER whonix only after sys-firewall. But for some cases I want both: VPN-TOR-VPN because some websites block TOR traffic.

Do you use TCP or UDP to connect to your VPN?
Tor only works for TCP.

[heinzl]

@apparatus

Did you run it as qvpn group?

sg qvpn -c "$VPN_CLIENT $VPN_OPTIONS"

I don’t know what you mean. I just made the scripts (with nft) and typed in the console while I am in the config folder: sudo openvpn --config CONFIG.ovpn. Then it hast these “resolve problems”. If I set up the autostart and restart the VPN-qube, it works. But then the same setting are used or not?

@apparatus

Do you use TCP or UDP to connect to your VPN?
Tor only works for TCP.

I’m not sure, I didn’t change anything in that matter.
So I can’t one of my vpn configurations after TOR?

[apparatus]

@heinzl

I don’t know what you mean. I just made the scripts (with nft) and typed in the console while I am in the config folder: sudo openvpn --config CONFIG.ovpn. Then it hast these “resolve problems”. If I set up the autostart and restart the VPN-qube, it works. But then the same setting are used or not?

Run it like this:

sudo sg qvpn -c "openvpn --config CONFIG.ovpn"

@heinzl

I’m not sure, I didn’t change anything in that matter.
So I can’t one of my vpn configurations after TOR?

What do you have in your openvpn config? proto tcp or proto udp?

[heinzl]

In the third line in the config there is
“proto udp”
Can I just add “tcp”?

For now I tried: VPN-Tor and if I make an IP check with the AppVM using sys-whonix as NetVM, I got an Tor ip. But I am not sure, if sys-whonix really uses sys-vpn - is it possible to check that he is using an vpn ip? In my understanding not, because Tor uses 3 relais and with an ip check I can only see the exit node right?

[solene]

yes you can use proto tcp if your VPN providers supports TCP on that port.

[heinzl]

Ok, I changed that to proto tcp and a vpn connection is etablished, but it still don’t work AFTER Tor. I can’t open a website if I using sys-whonix as NetVM for sys-vpn. Using sys-firewall as NetVM for sys-vpn, I can open a website.