[qubes-users] advanced networking setup: force some AppVMs via vpn

I’m trying to create a setup very similar to that shown in Joanna’s http://theinvisiblethings.blogspot.co.uk/2011/09/playing-with-qubes-networking-for-fun.html at the very bottom of the post:

AppVM01 -> firewallvm -> netvm -> NIC
AppVM02 -> firewallvm -> netvm -> NIC
AppVM03 -> [work-vpn] -> firewallvm -> netvm -> NIC
AppVM04 -> [work-vpn] -> firewallvm -> netvm -> NIC

The goal is to have some VMs reaching the internet directly, some (work) VMs reaching the Internet only via my work’s VPN.

I can connect to my work’s VPN by using the netvm’s NetworkManager applet, but of course that forces all network traffic from my Qubes system to go via the VPN which is not ideal.

So the question becomes, how to create this [work-vpn] VM?

I created a ProxyVM (seems like the right VM type for the job), but the NetworkManager service will not start - it just dies instantly.

Is ProxyVM the correct type, or should I create a second NetVM and ask my work appVMs to talk to it, bypassing the firewallVM? Doesn’t sound like a great idea.

Note: All network traffic must leave my Qubes machine through the same NIC.

Thanks,

Alex

I'm trying to create a setup very similar to that shown in Joanna's
http://theinvisiblethings.blogspot.co.uk/2011/09/playing-with-qubes-networking-for-fun.htmlat
the very bottom of the post:

AppVM01 -> firewallvm -> netvm -> NIC
AppVM02 -> firewallvm -> netvm -> NIC
AppVM03 -> [work-vpn] -> firewallvm -> netvm -> NIC
AppVM04 -> [work-vpn] -> firewallvm -> netvm -> NIC

The goal is to have some VMs reaching the internet directly, some (work)
VMs reaching the Internet only via my work's VPN.

I can connect to my work's VPN by using the netvm's NetworkManager applet,
but of course that forces all network traffic from my Qubes system to go
via the VPN which is not ideal.

So the question becomes, how to create this [work-vpn] VM?

I created a ProxyVM (seems like the right VM type for the job), but the
NetworkManager service will not start - it just dies instantly.

Yes, NetworkManager is disabled in non-netvm by default.

Is ProxyVM the correct type, or should I create a second NetVM and ask my
work appVMs to talk to it, bypassing the firewallVM? Doesn't sound like a
great idea.

ProxyVM is the correct type.

You have two options:
1. Setup VPN without NetworkManager. For OpenVPN it is quite simple: "openvpn
--config vpn-config.conf".
2. Enable NetworkManager in work-vpn: go to VM settings, services tab and add
there "network-manager". You need also somehow ensure that NetworkManager will
not touch eth0 there (there is no DHCP between Qubes VMs, so NM will probably
break network configuration otherwise). Read about unmanaged interfaces in
NetworkManager documentation.

I created a ProxyVM (seems like the right VM type for the job), but the
NetworkManager service will not start - it just dies instantly.

Sure, the ProxyVM is the correct type for this.
I do not know about NetwokManager service because I'm not using it
(it's is too limited)
I'm currently using CLI tools for the VPN (Cisco, OpenVPN)
setup/start/stop... and it is working fine. :slight_smile:

So your setup is just fine.
if you really need the NetworkManager you should just make it run somehow.

I'm trying to create a setup very similar to that shown in Joanna's The Invisible Things Lab's blog: Playing with Qubes Networking for Fun and Profit at the very bottom of the post:

AppVM01 -> firewallvm -> netvm -> NIC
AppVM02 -> firewallvm -> netvm -> NIC
AppVM03 -> [work-vpn] -> firewallvm -> netvm -> NIC
AppVM04 -> [work-vpn] -> firewallvm -> netvm -> NIC

The goal is to have some VMs reaching the internet directly, some (work) VMs reaching the Internet only via my work's VPN.

In my experience, a slight variation to your scenario worked best:

AppVM01 -> firewallvm -> netvm -> NIC
AppVM02 -> firewallvm -> netvm -> NIC
AppVM03 -> firewallvm-vpn -> [work-vpn] -> netvm -> NIC
AppVM04 -> firewallvm-vpn -> [work-vpn] -> netvm -> NIC

All the traffic out of the work-vpn will end up going out the tunX device, which caused issues for me when trying to filter URLs in the firewall rules (if I remember correctly, it's been awhile).

Thanks for the advice everyone - although I liked the idea of having a
pretty icon with a lock on my taskbar (with NetworkManager) I decided to
not deal with such a complex beast and just opted for command-line
invocation of vpnc - which establishes the tunnel just fine.

I now hit the following snag: the AppVMs I have placed behind this work-vpn
ProxyVM are not aware of the name resolution settings of the ProxyVM, hence
they cannot resolve "internal" work hostnames.

So, if my setup is
Work (AppVM) -> Work-VPN (ProxyVM) -> firewallVM (ProxyVM) -> netvm (NetVM)
-> NIC -> Internet
the problem now is that "Work" has the generic /etc/resolv.conf of the rest
of my Qubes system, while Work-VPN has the /etc/resolv.conf correctly set
by vpnc every time I connect to the VPN.

If I manually update /etc/resolv.conf on Work AppVM to match
/etc/resolv.conf of Work-VPN, then all is well. But surely there is a more
elegant way of doing this?

Alex

http://theinvisiblethings.blogspot.co.uk/2011/09/playing-with-qubes-networking-for-fun.html:
"Each Net and Proxy VM implements NAT, specifically masquerading, for all
the connected VMs. Additionally to this SNAT, each Net or Proxy VM provides
also DNAT redirection for DNS resolutions, so that each VM behind a Proxy
or Net VM thinks that it uses a DNS in the Net/Proxy VM, but in fact all
the DNS request are DNAT-ed by all the Proxy and Net VMs down the original
DNS that is provided to the final Net VM."

Does this mean that to get an alternate DNS resolver, one has to run
another netvm? In which case I should be running vpnc on a work-netvm?

Alex

Not necessary. You can redirect DNS traffic in work-vpn to vpn DNS. Just call
"sudo /usr/lib/qubes/qubes_setup_dnat_to_ns" to update it based on current
/etc/resolv.conf in work-vpn.

Just one more note:
I this case I'm not bothering with a separate firewall vm and a VPN
VM. it is the same VM type and actually you cant do nothing with your
firewall after your packetsa are encapsulated by the VPN.

So a VPN VM can be a firewall VM in teh same time...

Thank you for your help all - it works great.

Setup:
Work (AppVM) -> VPN (ProxyVM) -> netvm (NetVM) -> NIC

I have a hardware token for my employer's Cisco VPN, so this config works
for me, saved in /home/user/vpn.conf of my VPN ProxyVM:

[vpn.conf]
Xauth username xxxxxxxxxx@xxxxxxx.xxx
IPSec gateway xxxxxxxx.xxxxx.xxx
IPSec ID xxxxxxxxxxxx
IPSec secret xxxxxxxxxxxxxxxxxxxx

To connect to the VPN and use your VPN's nameservers to talk to internal
resources:

[start_vpn.sh ]
#!/bin/bash
sudo /usr/sbin/vpnc /home/user/vpn.conf
sleep 2
sudo /usr/lib/qubes/qubes_setup_dnat_to_ns

To terminate the VPN and restore public DNS resolution execute
./stop_vpn.sh on VPN ProxyVM:

[stop_vpn.sh]
#!/bin/bash
sudo /usr/sbin/vpnc-disconnect
sleep 2
sudo /usr/lib/qubes/qubes_setup_dnat_to_ns

Appears to work great!

Thanks,

Alex

For the record I've also documented this here:

Alex

Nice job documenting the process. The starting the VPN step could be changed with a /rw/config/rc.local file so the user does not need to manually launch it. I do use that file to mount a samba share to my home NAS.