I updated to R4.2 yesterday, so I believe this is related.
Setup
Internet ↔ sys-net ↔ sys-net-fw ↔ vpn-qube ↔ vpn-qube-fw ↔ various qubes
The vpn-qube has a firewall setting to restrict traffic to only the VPN endpoint.
I have a wrapper script around my VPN client that runs the qubes-setup-dnat-to-ns.sh script after connecting to the VPN.
The VPN client creates a tun0 interface that should have all traffic routed through it (i.e., a full tunnel).
This was functional before the upgrade to 4.2.
Functioning
ICMP works from vpn-qube.
VPN connects from vpn-qube.
dnat-dns entries are being made in vpn-qube
Broken
Nothing works upstream of the vpn-qube.
ICMP from vpn-qube-fw and all connected qubes.
DNS is broken even when I specify the resolving server (e.g., from vpn-qube-fw dig @vpn_dns_server google.com)
Edit to add
When I attempt to change the network configuration for vpn-qube-fw, the Qubes Manager GUI crashes. Attempting to change from the shell fails with the following:
$ qvm-prefs vpn-qube-fw netvm sys-net
...
qvm-prefs: error: no such property: 'netvm'
Powering off vpn-qube-fw resolved the netvm changing problem. Both sys-net and vpn-qube were powered on at the time. After powering off vpn-qube-fw, I was able to change its netvm to sys-net.
With sys-net, vpn-qube-fw was able to ping and reach Internet hosts (e.g., ping 8.8.8.8).
Reviewing similar issues, I think the problem is in the vpn-qube not updating its downstream and allowed configuration. From https://forum.qubes-os.org/t/network-no-longer-work-after-upgrade-to-4-2/23081/6, I also do not see any entries for vpn-qube-fw in vpn-qube; however, when I switched vpn-qube-fw to sys-net, I saw the vpn-qube-fw IP address show up as an entry (in sys-net).
I have powered off both vpn-qube and vpn-qube-fw, and re-set the netvm to vpn-qube with no change to the nft ruleset in vpn-qube.
Did packages change for Debian 12 networking in R4.2? Networking is completely broken for me in Debian-based templates. I swapped my sys-net to debian-12 and I can replicate the broken behavior. Switching to Fedora 38 as a template restores networking.
This is all independent and before my VPN client is even touched, to be clear.
Did you do an in-place upgrade or a clean install with backup restoration?
Can you check in /etc/apt/sources.list.d/qubes-r4.list if you have r4.2 for each repos?
Let’s assume that a clean Debian install would resolve the issue, but is not an option at the moment.
In-place upgrade.
Checking /etc/apt/sources.list.d/:
/etc/apt/sources.list.d# ls -l
total 20
-rw-r--r-- 1 root root 375 Sep 28 09:30 qubes-contrib-r4.1.list
-rw-r--r-- 1 root root 360 Jul 23 02:09 qubes-contrib-r4.2.list
-rw-r--r-- 1 user user 2835 Dec 23 19:21 qubes-r4.list
-rw-r--r-- 1 root root 2355 Dec 23 19:21 qubes-r4.list.bak
-rw-r--r-- 1 user user 2955 Nov 13 20:32 qubes-r4.list.dpkg-dist
Just at first glance that is not a good sign. The qubes-contrib-r4.1.list is still present and the ownership of qubes-r4.list and qubes-r4.list.dpkg-dist are user.
qubes-r4.list does not contain any r4.1 references. It appears to have been successfully modified for r4.2.
# grep r4.1 qubes-r4.list
# grep r4.2 qubes-r4.list
deb [arch=amd64 signed-by=/usr/share/keyrings/qubes-archive-keyring-4.2.gpg signed-by=/usr/share/keyrings/qubes-archive-keyring-4.2.gpg] https://deb.qubes-os.org/r4.2/vm bookworm main
[snip]
I’ve commented out the r4.1deb source in qubes-contrib-r4.1.list and also changed the other source files’ ownership to root.
Running apt dist-upgrade only mentioned some packages for removal, so I let that process. I’m going to see if this changed/fixed anything and see if I can find what else qubes-dist-upgrade -t may have been missed.
Also, for clarity, the above is from a modified debian-minimal template but as mentioned earlier, the symptoms were present when I switched to my debian-12 template. My debian-12 template should be a stock template.
/etc/apt$ ls -la sources.list.d/
total 20
drwxr-xr-x 2 root root 4096 Dec 23 19:22 .
drwxr-xr-x 8 root root 4096 Jul 23 20:42 ..
-rw-r--r-- 1 root root 2835 Dec 23 19:22 qubes-r4.list
-rw-r--r-- 1 root root 2355 Dec 23 19:22 qubes-r4.list.bak
-rw-r--r-- 1 root root 2955 Nov 13 20:32 qubes-r4.list.dpkg-dist
/etc/apt$ grep r4.2 sources.list.d/qubes-r4.list
deb [arch=amd64 signed-by=/usr/share/keyrings/qubes-archive-keyring-4.2.gpg signed-by=/usr/share/keyrings/qubes-archive-keyring-4.2.gpg] https://deb.qubes-os.org/r4.2/vm bookworm main
[snip]