I gather you want to connect a computer on your local network that will be running LLs (llama/ollama), to a qube that is using VPN (through sys-VPN →ssy-firewall → sys-net).
Yes, that can be done. You just need to configure the firewall rules on each qube to forward the data through the path you want.
Local LLM
First, I’d suggest you setup NginX on the Local LLM computer, so communication is encrypted even inside your home network.
If you don’t care about that, just make sure Ollama, for example, is configured to the Ethernet network card, or through all of them, OLLAMA_HOST=0.0.0.0
sys-net / sys-firewall / sys-proton VPN (qubes that provide network)
You need to setup the firewall for each to allow the incoming and outgoing data streams.
For example sys-net, edit /rw/config/qubes-firewall-user-script to include,
# DNAT Chain: Redirect external traffic to sys-firewall
if nft add chain qubes custom-dnat-AppVM ‘{ type nat hook prerouting priority filter +1 ; policy accept; }’
then
nft add rule qubes custom-dnat-AppVM iifname “enp6so” ip saddr 192.168.1.1-192.168.1.100 tcp dport 11434 ct state new,established,related counter dnat 10.138.18.10
# Forwarding: Allow the traffic to pass through to sys-firewall
nft add rule qubes custom-forward iifname “wls6” ip saddr 192.168.1.1-192.168.1.100 ip daddr 10.138.18.10 tcp dport 11434 ct state new,established,related counter accept
fi
Replace,
enp6so, with your network adapter (ip addr)
192.168.1.1-192.168.1.100, with your network ip range (or use Local LLM IP)
11434, with your ollama/llama.ccp port
10.138.18.10, the IP of the qube you are forwarding the data to, sys-firwall in this case.
Do this for sys-firewall and sys-proton VPN
AI Harness (appvm)
/rw/config/rc.local,
# Allow incoming traffic on ports 11434 for AI Harness
nft add rule qubes custom-input tcp dport 11434 ip daddr 10.137.0.20 ct state new,established,related counter accept
Replace,
10.137.0.20, with the AI Harness qube’s (internal) network IP.
Hope this helps.
