I use f38 template to run appvm with vpn, I’m trying to migrate now to f39 but I don’t know how to modify the scripts from iptables to nftables, i would like your help to transform the scripts below so that they can work on f39.
On AppVM(VPN) with fedora-38 template i have 3 files:
/rw/config/rc.local
/rw/config/vpn/qubes-vpn-handler.sh
/rw/config/qubes-firewall-user-script
RC.LOCAL
#!/bin/bash
VPN_CLIENT='openvpn'
VPN_OPTIONS='--cd /rw/config/vpn/ --config openvpn-client.ovpn --daemon'
# Add the `matrix` group to system, if it doesn't already exist
if ! grep -q "^matrix:" /etc/group ; then
groupadd -rf matrix
su - -c 'notify-send "$(hostname) Creating matrix" --icon=network-server' user
sync
fi
sleep 1s
su - -c 'notify-send -u critical "$(hostname) entering matrix..." --icon=network-server' user
# matrix
sg matrix -c "$VPN_CLIENT $VPN_OPTIONS"
QUBES-VPN-HANDLER.SH
#!/bin/bash
set -e
export PATH="$PATH:/usr/sbin:/sbin"
case "$1" in
up)
# To override DHCP DNS, assign DNS addresses to 'vpn_dns' env variable before calling this script;
# Format is 'X.X.X.X Y.Y.Y.Y [...]'
if [[ -z "$vpn_dns" ]] ; then
# Parses DHCP foreign_option_* vars to automatically set DNS address translation:
for optionname in ${!foreign_option_*} ; do
option="${!optionname}"
unset fops; fops=($option)
if [ ${fops[1]} == "DNS" ] ; then vpn_dns="$vpn_dns ${fops[2]}" ; fi
done
fi
iptables -t nat -F PR-QBS
if [[ -n "$vpn_dns" ]] ; then
# Set DNS address translation in firewall:
for addr in $vpn_dns; do
iptables -t nat -A PR-QBS -i vif+ -p udp --dport 53 -j DNAT --to $addr
iptables -t nat -A PR-QBS -i vif+ -p tcp --dport 53 -j DNAT --to $addr
done
su - -c 'notify-send -u critical "WELCOME !" "$(hostname): ON." --icon=network-idle' user
else
su - -c 'notify-send "$(hostname): ON, no DNS!" --icon=dialog-error' user
fi
;;
down)
su - -c 'notify-send -u critical "TANGO DOWN !" "$(hostname): OFF !" --icon=dialog-warning' user
# Restart the VPN automatically
sleep 3s
sudo /rw/config/rc.local
;;
esac
QUBES-FIREWALL-USER-SCRIPT
#!/bin/bash
# Block forwarding of connections through upstream network device
# (in case the vpn tunnel breaks):
iptables -I FORWARD -o eth0 -j DROP
iptables -I FORWARD -i eth0 -j DROP
ip6tables -I FORWARD -o eth0 -j DROP
ip6tables -I FORWARD -i eth0 -j DROP
# Accept traffic to VPN
iptables -P OUTPUT ACCEPT
iptables -F OUTPUT
# Block non-VPN traffic to clearnet
iptables -I OUTPUT -o eth0 -j DROP
# Allow traffic from the `matrix` group to the uplink interface (eth0);
# Our VPN client will run with group `matrix`.
iptables -I OUTPUT -p all -o eth0 -m owner --gid-owner matrix -j ACCEPT
Do you have Qubes OS 4.1 or Qubes OS 4.2?
The switch from iptables to nftables happens when switching the Qubes OS versions, not when switching template versions. So if you’re using Qubes OS 4.1 then both fedora-38 and fedora-39 templates will still have iptables and if you have Qubes OS 4.2 then both templates should have nftables.
so… im using fedora 38 on qubes 4.2.1 and is working… didnt know about that…and i migrated my appvpm using a backup from qubes 4.1 and is working… i was thinking that the problem was from templates… and the f38 template has both… nft and iptables… so i dont need to change nothiing?
Since you’re using Qubes OS 4.2 then you need to switch the iptables to nftables.
And it’s better to use the fresh template installed from Qubes OS repository instead of upgrading your old fedora-38 template in-place since you have iptables installed there and maybe some other legacy things that could break something.
You can check this post as an example:
Don’t use iptables-translate to convert the rules, they won’t convert properly because Qubes OS is using custom table and chain names.
The linked post already contains two scripts /rw/config/vpn/qubes-vpn-handler.sh and /rw/config/qubes-firewall-user-script that have firewall rules converted from iptables to nftables. Use the firewall rules from there.
Open the link and click on the /rw/config/vpn/qubes-vpn-handler.sh and /rw/config/qubes-firewall-user-script lines to expand the hidden text.
So i did as you said, installed a fresh template fedora-39-minimal, cloned (f39m-vpn) and installed all the packages needed to run the vpn on this, copied your scripts and restart the vpn… open console and started vpn, it gets stucked here:
bash-5.2# sudo openvpn --cd /rw/config/vpn --config openvpn-client.ovpn
2024-07-21 06:50:28 WARNING: file 'user.key' is group or others accessible
2024-07-21 06:50:28 OpenVPN 2.6.9 x86_64-redhat-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
2024-07-21 06:50:28 library versions: OpenSSL 3.1.1 30 May 2023, LZO 2.10
2024-07-21 06:50:28 DCO version: N/A
2024-07-21 06:50:28 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
2024-07-21 06:50:28 TCP/UDP: Preserving recently used remote address: [AF_INET]x.x.x.x:yyyy
2024-07-21 06:50:28 Socket Buffers: R=[212992->212992] S=[212992->212992]
2024-07-21 06:50:28 UDPv4 link local: (not bound)
2024-07-21 06:50:28 UDPv4 link remote: [AF_INET]x.x.x.x:yyyy
2024-07-21 06:50:28 write UDPv4 []: Operation not permitted (fd=3,code=1)
2024-07-21 06:50:30 write UDPv4 []: Operation not permitted (fd=3,code=1)
2024-07-21 06:50:34 write UDPv4 []: Operation not permitted (fd=3,code=1)
i noticed that you have made some changes in qubes-firewall-user-script… perhaps something about that because you didnt test it?
Those are not scripts created by me, they are used in the guide in that link.
That guide assumes that you will run openvpn process with qvpn group permissions:
So the firewall rules in the script are restricting the outbound traffic and only allow the traffic originated from the processes started by qvpn group.
Since you use different group name in your script then change qvpn to matrix or vice versa.
And start the openvpn process from this group and not like this:
yes, observed that and made the modification before running, and i still cant connect.
if i changed drop to accept in this line inside qubes-firewall-user-script i can browser inside the appvm that is running the vpn but if i try to use this appvm as proxy to another qube, i cant get connection
so i have copied your scripts exactly as you posted here… to vpn connects but no traffic on vms…
/rw/config/qubes-firewall-user-script
#!/bin/bash
# Block forwarding of connections through upstream network device
# (in case the vpn tunnel breaks):
# Prevent the qube to forward traffic outside of the VPN
nft insert rule qubes custom-forward oifname eth0 counter drop
nft insert rule ip6 qubes custom-forward oifname eth0 counter drop
nft insert rule qubes custom-forward iifname eth0 counter drop
nft insert rule ip6 qubes custom-forward iifname eth0 counter drop
# Add the `qvpn` group to system, if it doesn't already exist
if ! grep -q "^qvpn:" /etc/group ; then
sleep 3s
groupadd -rf qvpn
sync
fi
sleep 2s
# Accept traffic to VPN
nft 'add chain qubes output { type filter hook output priority 0; policy drop; }'
#nft 'add chain qubes output { type filter hook output priority 0; policy accept; }'#iptables -P OUTPUT ACCEPT
#iptables -F OUTPUT
# Block non-VPN traffic to clearnet
#nft insert rule ip qubes output oifname eth0 counter drop
#iptables -I OUTPUT -o eth0 -j DROP
# Allow traffic from the `qvpn` group to the uplink interface (eth0);
# Our VPN client will run with group `qvpn`.
nft insert rule ip qubes output oifname eth0 skgid qvpn accept
#iptables -I OUTPUT -p all -o eth0 -m owner --gid-owner qvpn -j ACCEPT
/rw/config/vpn/qubes-vpn-handler.sh
#!/bin/bash
set -e
export PATH="$PATH:/usr/sbin:/sbin"
case "$1" in
up)
# To override DHCP DNS, assign DNS addresses to 'vpn_dns' env variable before calling this script;
# Format is 'X.X.X.X Y.Y.Y.Y [...]'
if [[ -z "$vpn_dns" ]] ; then
# Parses DHCP foreign_option_* vars to automatically set DNS address translation:
for optionname in ${!foreign_option_*} ; do
option="${!optionname}"
unset fops; fops=($option)
if [ ${fops[1]} == "DNS" ] ; then vpn_dns="$vpn_dns ${fops[2]}" ; fi
done
fi
nft flush chain ip qubes dnat-dns
#nft add chain qubes nat { type nat hook prerouting priority dstnat\; }
#iptables -t nat -F PR-QBS
if [[ -n "$vpn_dns" ]] ; then
# Set DNS address translation in firewall:
for addr in $vpn_dns; do
nft add rule qubes dnat-dns iifname == "vif*" tcp dport 53 dnat "$addr"
nft add rule qubes dnat-dns iifname == "vif*" udp dport 53 dnat "$addr"
#iptables -t nat -A PR-QBS -i vif+ -p udp --dport 53 -j DNAT --to $addr
#iptables -t nat -A PR-QBS -i vif+ -p tcp --dport 53 -j DNAT --to $addr
done
su - -c 'notify-send "$(hostname): LINK IS UP." --icon=network-idle' user
else
su - -c 'notify-send "$(hostname): LINK UP, NO DNS!" --icon=dialog-error' user
fi
;;
down)
su - -c 'notify-send "$(hostname): LINK IS DOWN !" --icon=dialog-error' user
# Restart the VPN automatically
#sleep 5s
#sudo /rw/config/rc.local
;;
esac
i disable rc.local and start the vm manually… it worked but now the problem is: no traffic insiide when using qubes-firewall-user-script with this line
# Accept traffic to VPN
nft 'add chain qubes output { type filter hook output priority 0; policy drop; }'
and if i change “drop” to accept" i have traffic inside vpn-vm, but can use it as proxyvm to another qube
It’s intended behavior to prevent VPN leaks and only allow the openvpn connection in the vpn-vm.
If you want to test the network in the vpn-vm then run the commands as qvpn group e.g.:
so, it looks like the problem was that the service network-manager was enable, i disabled it on qubes-manager “services” window and it is running like a charm. i would like to thank you @apparatus for your attention with this topic, you really helped me with this… theres something i forget to tell that is about the notification-daemon, on vpn template i had to make some conf about notifications.
