User -> Tor -> VPN -> Internet - 4.1 broken?

Hey, just installed 4.1 and I can’t get “User → Tor → VPN → Internet” connection working. here is the openvpn log, as you can see, once initialization is completed, a shortwhile later it will timeout due to inactivity. Below is what “ip route show” in terminal shows:

0.0.0.0/1 via 10.66.5.1 dev tun0 
default via 10.137.0.10 dev eth0 onlink 
10.66.5.0/24 dev tun0 proto kernel scope link src 10.66.5.69 
10.137.0.10 dev eth0 scope host onlink 
128.0.0.0/1 via 10.66.5.1 dev tun0 
212.129.0.80 via 10.137.0.10 dev eth0 

I’m just testing this with cryptofree, but here is the ovpn config I used. auth.txt can contain any username/password. Unfortunately, I can’t upload a cap file from wireshark in sys-tor-vpn on eth0, but essentially it does DNS, TCP/SSL blah blah openvpn connecting, and then once “Initialization Sequence Completed”, there will be a bunch of ARP requests “who has 212.129.0.80? tell 10.137.0.17” Perhaps someone else who can confirm the same situation, can upload better logs for the rest of us.

Any way to fix this? I’ve tried it with a lot of different free and even a paid VPN but every time, after initialization, it always does the ARP thing and eventually the connection will reset and it’ll do it again etc etc…

To reproduce: create a new/use existing sys-whonix. Create a new appVM with networking set to sys-whonix, and “provides network to other qubes”. use openvpn with any known good configuration.

I’ve found the solution! But unfortunately it isn’t a nice one. After starting openvpn session, we can see output of ip neigh:

bash-5.1# ip neigh show
10.137.0.6 dev eth0 lladdr fe:ff:ff:ff:ff:ff PERMANENT
212.129.0.80 dev eth0  INCOMPLETE

which of course happens because for some reason this ARP request isn’t answered. To fix this, simply execute arp -s IP -i eth0 fe:ff:ff:ff:ff:ff, where IP is the IP address who’s ARP request fails (in cryptofree’s case, it is 212.129.0.80). Then, ip neigh show will output

bash-5.1# ip neigh show
10.137.0.6 dev eth0 lladdr fe:ff:ff:ff:ff:ff PERMANENT
212.129.0.80 dev eth0 lladdr fe:ff:ff:ff:ff:ff PERMANENT

and the ARP requests will stop, and VPN will work just as you’d expect.

Unfortunately, starting the connection everytime, you will probably have to do this. The real question is why does this ARP request fail? There is no easy way to script this either

4 Likes

thank you. fell into this. It helps

This might indeed be a Qubes 4.1 networking bug.

Can someone please report it at https://github.com/QubesOS/qubes-issues/issues ?

TLDR: sudo sysctl -w net.ipv4.ip_forward=1 in your whonix gateway will resolve it, until reboot. In your template, you can edit /etc/sysctl.d/anonymizer-config-gateway.conf and comment out the line net.ipv4.ip_forward = 0, and this should fix ARP requests not getting replies.

Long: Qubes uses proxy arp, which is why every ARP request receives a fe:ff:ff:ff:ff:ff (MAC address of the corresponding netVM) reply. Behind sys-whonix, ARP requests for MAC address of sys-whonix receive the proper reply, but anything besides that do not work. It doesn’t really make sense to me personally why IP forwarding would need to be enabled in order for sys-whonix to simply proxy it? Maybe i’m misunderstanding. In the end, this seems more like a Whonix change more than a Qubes one.

1 Like

This should be explained at VPN documentaton if it’s not possible to enable it by defaukt at whonix

Thank you for new solution :slight_smile:

deserve to bump

Experienced the same issue with switch to Qubes 4.1.

works for me. I also asked in the Whonix forum for the second solution:

Here is the maintainer’s answer, so don’t this:

Don’t use IP forwarding inside Whonix-Gateway. It’s a milestone that IP forwarding isn’t required and immensely reduces the risk of IP leaks.

In the meanwhile: did anybody manage to find out more about 4.1. network internals, which make the existing 4.0. solution not workable?
The ARP setting requires cumbersome manual work each time and with each server IP change.

Issue reported here: Qubes 4.1 - VPN over Tor netvms: ARP request does not get resolved properly · Issue #7123 · QubesOS/qubes-issues · GitHub

Is that github issue saying it was closed/resolved?

I’m still having this problem. @aUsername solution worked for me.