Configuring a ProxyVM VPN Gateway

taradiddles · June 3, 2023, 12:42pm

Note: If you seek to enhance your privacy, you may also wish to consider Whonix. You should also be aware of the potential risks of VPNs.

Although setting up a VPN connection is not by itself Qubes specific, Qubes includes a number of tools that can make the client-side setup of your VPN more versatile and secure. This document is a Qubes-specific outline for choosing the type of VM to use, and shows how to prepare a ProxyVM for either NetworkManager or a set of fail-safe VPN scripts.

Please refer to your guest OS and VPN service documentation when considering the specific steps and parameters for your connection(s); The relevant documentation for the Qubes default guest OS (Fedora) is Establishing a VPN Connection.

NetVM

The simplest case is to set up a VPN connection using the NetworkManager service inside your NetVM. Because the NetworkManager service is already started, you are ready to set up your VPN connection. However this has some disadvantages:

You have to place (and probably save) your VPN credentials inside the NetVM, which is directly connected to the outside world
All your AppVMs which are connected to the NetVM will be connected to the VPN (by default)

AppVM

While the NetworkManager service is not started here (for a good reason), you can configure any kind of VPN client in your AppVM as well. However this is only suggested if your VPN client has special requirements.

ProxyVM

One of the best unique features of Qubes OS is its special type of VM called a ProxyVM. The special thing is that your AppVMs see this as a NetVM (or uplink), and your NetVMs see it as a downstream AppVM. Because of this, you can place a ProxyVM between your AppVMs and your NetVM. This is how the default sys-firewall VM functions.

Using a ProxyVM to set up a VPN client gives you the ability to:

Separate your VPN credentials from your NetVM.
Separate your VPN credentials from your AppVM data.
Easily control which of your AppVMs are connected to your VPN by simply setting it as a NetVM of the desired AppVM.

Set up a ProxyVM as a VPN gateway using NetworkManager

Create a new VM, name it, click the ProxyVM radio button, and choose a color and template.
Add the network-manager service to this new VM.

Settings-services.png710×511 33.1 KB
Set up your VPN as described in the NetworkManager documentation linked above.
(Optional) Make your VPN start automatically.

Edit /rw/config/rc.local and add these lines:
```
# Automatically connect to the VPN once Internet is up
while ! ping -c 1 -W 1 1.1.1.1; do
   sleep 1
done
PWDFILE="/rw/config/NM-system-connections/secrets/passwd-file.txt"
nmcli connection up file-vpn-conn passwd-file $PWDFILE
```
You can find the actual “file-vpn-conn” in /rw/config/NM-system-connections/.

Create directory /rw/config/NM-system-connections/secrets/ (You can put your *.crt and *.pem files here too). Create a new file /rw/config/NM-system-connections/secrets/passwd-file.txt:
```
vpn.secrets.password:XXXXXXXXXXXXXX
```
And substitute “XXXXXXXXXXXXXX” for the actual password. The contents of passwd-file.txt may differ depending on your VPN settings. See the documentation for nmcli up.

(Optional) Make the network fail-close for the AppVMs if the connection to the VPN breaks.

Edit /rw/config/qubes-firewall-user-script and add these lines:

# Block forwarding of connections through upstream network device
# (in case the vpn tunnel breaks)
iptables -I FORWARD -o eth0 -j DROP
iptables -I FORWARD -i eth0 -j DROP
ip6tables -I FORWARD -o eth0 -j DROP
ip6tables -I FORWARD -i eth0 -j DROP

Configure your AppVMs to use the new VM as a NetVM.

Settings-NetVM.png721×269 27.3 KB
Optionally, you can install some custom icons for your VPN

Set up a ProxyVM as a VPN gateway using iptables and CLI scripts

This method is more involved than the one above, but has anti-leak features that also make the connection fail closed should it be interrupted. It has been tested with Fedora 30 and Debian 10 templates.

Before proceeding, you will need to download a copy of your VPN provider’s configuration file(s) and have your VPN login information handy.

Create a new VM, name it, choose “provides network”, and choose a color and template.

Note: Do not enable NetworkManager in the ProxyVM, as it can interfere with the scripts’ DNS features. If you enabled NetworkManager or used other methods in a previous attempt, do not re-use the old ProxyVM… Create a new one according to this step.

If your choice of TemplateVM doesn’t already have the VPN client software, you’ll need to install the software in the template before proceeding. The ‘openvpn’ package comes installed in the Fedora template, and in Debian it can be installed with the following command:
```
sudo apt-get install openvpn
```
Disable any auto-starting service that comes with the software package. For example for OpenVPN.
```
sudo systemctl disable openvpn-server@.service
sudo systemctl disable openvpn-client@.service
```
Set up and test the VPN client. Make sure the VPN VM and its TemplateVM is not running. Run a terminal (CLI) in the VPN VM – this will start the VM. Then create a new /rw/config/vpn folder with:
```
sudo mkdir /rw/config/vpn
```
Copy your VPN configuration files to /rw/config/vpn. Your VPN config file should be named openvpn-client.ovpn so you can use the scripts below as is without modification. Otherwise you would have to replace the file name. Files accompanying the main config such as *.crt and *.pem should also be placed in the /rw/config/vpn folder.

Check or modify configuration file contents using a text editor:
```
sudo gedit /rw/config/vpn/openvpn-client.ovpn
```
Files referenced in openvpn-client.ovpn should not use absolute paths such as /etc/....

The config should route all traffic through your VPN’s interface after a connection is created; For OpenVPN the directive for this is redirect-gateway def1.

Make sure it already includes or add:
```
redirect-gateway def1
```
The VPN client may not be able to prompt you for credentials when connecting to the server, so we’ll add a reference to a file containing the VPN username and password. For example for OpenVPN, add or modify auth-user-pass like so:
```
auth-user-pass pass.txt
```
Save the /rw/config/vpn/openvpn-client.ovpn file.

Now make sure a /rw/config/vpn/pass.txt file actually exists.
```
sudo gedit /rw/config/vpn/pass.txt
```
Add:
```
username
password
```
Replace username and password with your actual username and password.

Test your client configuration: Run the client from a CLI prompt in the ‘vpn’ folder, preferably as root. For example:
```
sudo openvpn --cd /rw/config/vpn --config openvpn-client.ovpn
```
Watch for status messages that indicate whether the connection is successful and test from another VPN VM terminal window with ping.
```
ping 1.1.1.1
```
ping can be aborted by pressing the two keys ctrl + c at the same time. DNS may be tested at this point by replacing addresses in /etc/resolv.conf with ones appropriate for your VPN (although this file will not be used when setup is complete). Diagnose any connection problems using resources such as client documentation and help from your VPN service provider. Proceed to the next step when you’re sure the basic VPN connection is working.

Create the DNS-handling script.

sudo gedit /rw/config/vpn/qubes-vpn-handler.sh

Add the following:

#!/bin/bash
set -e
export PATH="$PATH:/usr/sbin:/sbin"
  
case "$1" in
  
up)
# To override DHCP DNS, assign DNS addresses to 'vpn_dns' env variable before calling this script;
# Format is 'X.X.X.X  Y.Y.Y.Y [...]'
if [[ -z "$vpn_dns" ]] ; then
    # Parses DHCP foreign_option_* vars to automatically set DNS address translation:
    for optionname in ${!foreign_option_*} ; do
        option="${!optionname}"
        unset fops; fops=($option)
        if [ ${fops[1]} == "DNS" ] ; then vpn_dns="$vpn_dns ${fops[2]}" ; fi
    done
fi
  
iptables -t nat -F PR-QBS
if [[ -n "$vpn_dns" ]] ; then
    # Set DNS address translation in firewall:
    for addr in $vpn_dns; do
        iptables -t nat -A PR-QBS -i vif+ -p udp --dport 53 -j DNAT --to $addr
        iptables -t nat -A PR-QBS -i vif+ -p tcp --dport 53 -j DNAT --to $addr
    done
    su - -c 'notify-send "$(hostname): LINK IS UP." --icon=network-idle' user
else
    su - -c 'notify-send "$(hostname): LINK UP, NO DNS!" --icon=dialog-error' user
fi
  
;;
down)
su - -c 'notify-send "$(hostname): LINK IS DOWN !" --icon=dialog-error' user

# Restart the VPN automatically
sleep 5s
sudo /rw/config/rc.local
;;
esac

Save the script. Make it executable.

sudo chmod +x /rw/config/vpn/qubes-vpn-handler.sh

Configure client to use the DNS handling script. Using openvpn as an example, edit the config.
```
sudo gedit /rw/config/vpn/openvpn-client.ovpn
```
Add the following.
```
script-security 2
up 'qubes-vpn-handler.sh up'
down 'qubes-vpn-handler.sh down'
```
Remove other instances of lines starting with script-security, up or down should there be any others. Save the script. Restart the client and test the connection again …this time from an AppVM!

Set up iptables anti-leak rules. Edit the firewall script.

sudo gedit /rw/config/qubes-firewall-user-script

Clear out the existing lines and add:

#!/bin/bash
#    Block forwarding of connections through upstream network device
#    (in case the vpn tunnel breaks):
iptables -I FORWARD -o eth0 -j DROP
iptables -I FORWARD -i eth0 -j DROP
ip6tables -I FORWARD -o eth0 -j DROP
ip6tables -I FORWARD -i eth0 -j DROP

#    Accept traffic to VPN
iptables -P OUTPUT ACCEPT
iptables -F OUTPUT

#    Add the `qvpn` group to system, if it doesn't already exist
if ! grep -q "^qvpn:" /etc/group ; then
     groupadd -rf qvpn
     sync
fi
sleep 2s

#    Block non-VPN traffic to clearnet
iptables -I OUTPUT -o eth0 -j DROP
#    Allow traffic from the `qvpn` group to the uplink interface (eth0);
#    Our VPN client will run with group `qvpn`.
iptables -I OUTPUT -p all -o eth0 -m owner --gid-owner qvpn -j ACCEPT

Save the script. Make it executable.

sudo chmod +x /rw/config/qubes-firewall-user-script

Set up the VPN’s autostart.

sudo gedit /rw/config/rc.local

Clear out the existing lines and add:

#!/bin/bash
VPN_CLIENT='openvpn'
VPN_OPTIONS='--cd /rw/config/vpn/ --config openvpn-client.ovpn --daemon'

groupadd -rf qvpn ; sleep 2s
sg qvpn -c "$VPN_CLIENT $VPN_OPTIONS"
su - -c 'notify-send "$(hostname): Starting $VPN_CLIENT..." --icon=network-idle' user

If you are using anything other than OpenVPN, change the VPN_CLIENT and VPN_OPTIONS variables to match your VPN software. Save the script. Make it executable.

sudo chmod +x /rw/config/rc.local

Restart the new VM! The link should then be established automatically with a popup notification to that effect.

Usage

Configure your AppVMs to use the VPN VM as a NetVM…

If you want to update your TemplateVMs through the VPN, you can enable the qubes-updates-proxy service for your new VPN VM and configure the qubes-rpc policy.

Troubleshooting

See the VPN Troubleshooting guide for tips on how to fix common VPN issues.

This document was migrated from the qubes-community project

Page archive
First commit: 08 Dec 2020. Last commit: 01 Apr 2022.
Applicable Qubes OS releases based on commit dates and supported releases: 4.0, 4.1
Original author(s) (GitHub usernames): bakeromso, observinglynx, kakooloukia
Original author(s) (forum usernames):
Document license: GPLv2

etaz · January 29, 2024, 5:14pm

In case anyone is interested in needed changes for Qubes OS 4.2 (iptables → nftables), the following did work for me:

# /rw/config/vpn/qubes-vpn-handler.sh

#...
nft flush chain ip qubes dnat-dns
if [[ -n "$vpn_dns" ]] ; then
    # Set DNS address translation in firewall:
    for addr in $vpn_dns; do
        nft add rule qubes dnat-dns iifname == "vif*" tcp dport 53 dnat "$addr"
        nft add rule qubes dnat-dns iifname == "vif*" udp dport 53 dnat "$addr"

# /rw/config/qubes-firewall-user-script

nft add rule qubes custom-forward oifname eth0 counter drop
nft add rule ip6 qubes custom-forward oifname eth0 counter drop
nft add rule qubes custom-forward iifname eth0 counter drop
nft add rule ip6 qubes custom-forward iifname eth0 counter drop

#    Accept traffic to VPN
nft 'add chain qubes output { type filter hook output priority 0; policy accept; }'

# ...

#    Block non-VPN traffic to clearnet
nft insert rule ip qubes output oifname eth0 counter drop
#    Allow traffic from the `qvpn` group to the uplink interface (eth0);
#    Our VPN client will run with group `qvpn`.
nft insert rule ip qubes output oifname eth0 skgid qvpn accept

You can also use iptables-translate to try out translation from one to the other.

Also to disable OpenVPN in Debian:

systemctl disable --now openvpn.service

mirome · February 5, 2024, 9:27am

Thx for the nft rules.
But I think there is a little issue there. The rules in nft need to have priority specified. If not, they get the priority depending on when they were created/evaluated. In the qubes-firewall-user-script the last rules are for blocking all eth0 traffic and then to allow vpn traffic. this two rules need to switch places:


#    Allow traffic from the `qvpn` group to the uplink interface (eth0);
#    Our VPN client will run with group `qvpn`.
nft add rule ip qubes output oifname eth0 skgid qvpn accept

#    Block non-VPN traffic to clearnet
nft add rule ip qubes output oifname eth0 counter drop

etaz · February 5, 2024, 11:02am

@mirome Good catch - thanks! This was a typo, I accidently replaced nft insert by nft add.

I cannot edit post currently, will notify mods.

–

Update: Fixed Configuring a ProxyVM VPN Gateway - #2 by etaz to use nft insert. This also aligns with iptables -I of original guide post and should be equivalent to Configuring a ProxyVM VPN Gateway - #3 by mirome.

stardustspaceship · February 10, 2024, 2:39pm

@etaz, thank you for your nftables update - it was one of my first questions when updating to 4.2. For the setup guides for

or

do they both need to make an update to qubes-vpn-handler.sh?

scallyob · February 11, 2024, 2:23am

Just found this update for 4.2, my openVPN connection has not been working since upgrading to 4.2. I tried the new qubes-vpn-handler.sh by @etaz, but getting the same results as I did with old version:

2024-02-10 18:17:45 OpenVPN 2.6.8 x86_64-redhat-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
2024-02-10 18:17:45 library versions: OpenSSL 3.0.9 30 May 2023, LZO 2.10
2024-02-10 18:17:45 DCO version: N/A
2024-02-10 18:17:45 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
2024-02-10 18:17:45 TCP/UDP: Preserving recently used remote address: [AF_INET]999.999.999.99:443
2024-02-10 18:17:45 Socket Buffers: R=[131072->131072] S=[16384->16384]
2024-02-10 18:17:45 Attempting to establish TCP connection with [AF_INET]999.999.999.99:443
2024-02-10 18:17:45 TCP connection established with [AF_INET]999.999.999.99:443
2024-02-10 18:17:45 TCPv4_CLIENT link local: (not bound)
2024-02-10 18:17:45 TCPv4_CLIENT link remote: [AF_INET]999.999.999.99:443
2024-02-10 18:18:45 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
2024-02-10 18:18:45 TLS Error: TLS handshake failed
2024-02-10 18:18:45 Fatal TLS error (check_tls_errors_co), restarting
2024-02-10 18:18:45 SIGUSR1[soft,tls-error] received, process restarting
2024-02-10 18:18:45 Restart pause, 1 second(s)

etaz · February 12, 2024, 12:17pm

Above commands are to be used in conjunction with the second alternative:

Haven’t tried the first alternative with NetworkManager, but it does not mention any steps involving qubes-vpn-handler.sh, so I guess you don’t need that.

Hm, hard to say with this log. Mine looks same till your TLS Error: TLS key negotiation failed to occur within. Have you tried the step-by-step approach in OP, which suggests intermediate validation steps?

stardustspaceship · February 14, 2024, 2:39am

I’ve confirmed that when using the NetworkManager approach, you only need to add the following lines to prevent traffic from qubes connected to the netvm:

etaz:

nft add rule qubes custom-forward oifname eth0 counter drop
nft add rule ip6 qubes custom-forward oifname eth0 counter drop
nft add rule qubes custom-forward iifname eth0 counter drop
nft add rule ip6 qubes custom-forward iifname eth0 counter drop

This may be off-topic, but when I was testing this, I noticed that when I disconnect from the VPN, traffic from within the netvm is still allowed while traffic from a depending qube will be blocked. For example, from within the netvm, if I run curl https://ip.me, I will get the clearnet IP. If I run the same command in another qube connected to the netvm (i.e. an appvm), the request will timeout.

Can someone explain this or point me to docs/posts?

etaz · February 26, 2024, 8:55am

I’ve not tested the network manager approach - but it seems OP is missing a kill switch / DNS leak protection from within that net qube?
Manual CLI approach effectively does DROP the OUTPUT (net qube → Inet) per default by:

#    Block non-VPN traffic to clearnet
nft insert rule ip qubes output oifname eth0 counter drop
#    Allow traffic from the `qvpn` group to the uplink interface (eth0);
#    Our VPN client will run with group `qvpn`.
nft insert rule ip qubes output oifname eth0 skgid qvpn accept

Have you tried to setup firewall rules for VPN endpoint + hardening as descripted in Wireguard VPN setup?

qubiq · March 14, 2024, 9:45am

Does this mean that all the rest of the code that is specified in the header of the topic should be here? Or does that mean everything else needs to be deleted? Or delete something certain and keep another? Excuse me, but not everyone here reads code as much as a human language. Could you please send the full contents of the file, as it should be?

etaz · March 15, 2024, 6:36am

No problem, below is a copy/paste from my vpn qube.

`/rw/config/vpn/qubes-vpn-handler.sh`

#!/bin/bash
set -e
export PATH="$PATH:/usr/sbin:/sbin"

case "$1" in

up)
# To override DHCP DNS, assign DNS addresses to 'vpn_dns' env variable before calling this script;
# Format is 'X.X.X.X  Y.Y.Y.Y [...]'
if [[ -z "$vpn_dns" ]] ; then
    # Parses DHCP foreign_option_* vars to automatically set DNS address translation:
    for optionname in ${!foreign_option_*} ; do
        option="${!optionname}"
        unset fops; fops=($option)
        if [ ${fops[1]} == "DNS" ] ; then vpn_dns="$vpn_dns ${fops[2]}" ; fi
    done
fi


nft flush chain ip qubes dnat-dns
#nft add chain qubes nat { type nat hook prerouting priority dstnat\; }
#iptables -t nat -F PR-QBS
if [[ -n "$vpn_dns" ]] ; then
    # Set DNS address translation in firewall:
    for addr in $vpn_dns; do
        nft add rule qubes dnat-dns iifname == "vif*" tcp dport 53 dnat "$addr"
        nft add rule qubes dnat-dns iifname == "vif*" udp dport 53 dnat "$addr"
        #iptables -t nat -A PR-QBS -i vif+ -p udp --dport 53 -j DNAT --to $addr
        #iptables -t nat -A PR-QBS -i vif+ -p tcp --dport 53 -j DNAT --to $addr
    done
    su - -c 'notify-send "$(hostname): LINK IS UP." --icon=network-idle' user
else
    su - -c 'notify-send "$(hostname): LINK UP, NO DNS!" --icon=dialog-error' user
fi

;;
down)
su - -c 'notify-send "$(hostname): LINK IS DOWN !" --icon=dialog-error' user

# Restart the VPN automatically
#sleep 5s
#sudo /rw/config/rc.local
;;
esac

Note: I disabled auto-restart at the bottom.

`/rw/config/qubes-firewall-user-script`

#!/bin/bash

#    Block forwarding of connections through upstream network device
#    (in case the vpn tunnel breaks):
# Prevent the qube to forward traffic outside of the VPN
nft insert rule qubes custom-forward oifname eth0 counter drop
nft insert rule ip6 qubes custom-forward oifname eth0 counter drop
nft insert rule qubes custom-forward iifname eth0 counter drop
nft insert rule ip6 qubes custom-forward iifname eth0 counter drop


#    Accept traffic to VPN
nft 'add chain qubes output { type filter hook output priority 0; policy accept; }'
#iptables -P OUTPUT ACCEPT
#iptables -F OUTPUT

#    Add the `qvpn` group to system, if it doesn't already exist
if ! grep -q "^qvpn:" /etc/group ; then
     groupadd -rf qvpn
     sync
fi
sleep 2s

#    Block non-VPN traffic to clearnet
nft insert rule ip qubes output oifname eth0 counter drop
#iptables -I OUTPUT -o eth0 -j DROP


#    Allow traffic from the `qvpn` group to the uplink interface (eth0);
#    Our VPN client will run with group `qvpn`.
nft insert rule ip qubes output oifname eth0 skgid qvpn accept
#iptables -I OUTPUT -p all -o eth0 -m owner --gid-owner qvpn -j ACCEPT

Old commands are commented, hence you can compare changes by looking at the lines around. If you are not so proficient with this topic yet (also for ease of use), I probably just would use above linked Wirguard VPN guide. Also note, in terms of security it is better to use Qubes firewall for a killswitch instead of a firewall script within the vpn qube.

DVM · March 15, 2024, 10:07am

Why do you create a new chain for your DNS forwarding rules? Qubes creates the dnat-dns chain by default when the qube provides network for others. It would be better to flush this chain and add the rules there instead.

etaz · March 15, 2024, 6:39pm

IIRC I was inspired by the firewall docs suggesting to create custom dnat chain for each qube. But that is a different context and your suggestion definitely makes sense. Also to flush existing Qubes DNS NAT rules, which aren’t needed for VPN. Thanks for mentioning dnat-dns is the new PR-QBS!

Will update qubes-vpn-handler.sh :

nft flush chain ip qubes dnat-dns
#nft add chain qubes nat { type nat hook prerouting priority dstnat\; }

#...

nft add rule qubes dnat-dns iifname == "vif*" tcp dport 53 dnat "$addr"
nft add rule qubes dnat-dns iifname == "vif*" udp dport 53 dnat "$addr"
#iptables -t nat -A PR-QBS -i vif+ -p udp --dport 53 -j DNAT --to $addr
#iptables -t nat -A PR-QBS -i vif+ -p tcp --dport 53 -j DNAT --to $addr

I think it also makes sense to polish /rw/config/qubes-firewall-user-script (but need to test):

nft 'add chain qubes output { type filter hook output priority 0; policy drop; }'
#nft 'add chain qubes output { type filter hook output priority 0; policy accept; }'

so that this line isn’t needed:

#    Block non-VPN traffic to clearnet
#nft insert rule ip qubes output oifname eth0 counter drop
#iptables -I OUTPUT -o eth0 -j DROP

I keep those old comments for now, so the changes make sense for future readers.

Also when having tested in a Fedora netvm, I had issues with notify-send executed as root and needed to install mate-notification-daemon. That lines otherwise might be just commented.

Tezeria · March 16, 2024, 2:16pm

I’ve been asking myself a question for a while. Since the change from iptables to nftables,
I use the network manager rather than the “manual way” as described here and in the official documentation (I use it because I find it more convenient to change vpn servers via the network manager).
I’d like to know if using the network manager can cause more security problems than the “manual way” (knowing that I limit connections only to the IPs of the vpn as @solene does in her tutorial, whether with wireguard or openvpn).

solene · March 16, 2024, 3:25pm

I don’t think NetworkManager is doing much more than the wireguard client interface.

It’s easier to script the cli method as you can run commands on start / stop if it’s useful for you.

etaz · March 16, 2024, 3:36pm

If asking me, it boils down to 1. choosing a type of VPN application 2. preventing leaks between VPN and clearnet.

For 1. the CLI approach in theory probably has less attack surface than NetworkManager and fits better with minimal templates. NetworkManager is more convenient.

For 2.: Imo you can omit all those iptables and nftables shenanigans and just use qvm-firewall. VPN gateway usually needs only one or couple fix VPN IPs. It also offers more security, if your VPN qube itself gets compromised.

It was a good learning expericene to do these things with nft, but I probably stick to NetworkManager + qvm-firewall as well.

Tezeria · March 16, 2024, 4:58pm

Thanks for your answers

I use minimal template with network-manager and all is easy ok

qubiq · March 20, 2024, 7:16pm

When I tried to use the built-in firewall, I was confused by this feature.

Do you understand what that means? And why are some DNS is accepted?
I don’t fully understand this. Please explain