Help: Networking for Samba file sharing -- guideance on multicasting / scripting appreciated

Any help from an NFT / Qubes networking expert is greatly appreciated!

I have read the docs, looked at the localsend tutorial, and created scripts to reproduce the setup. For the life of me I cannot get Samba “auto-discovery” to work. This depends on multicasting. If I install inside of sys-net, then everything works perfect. However, inside of an AppVM the networking does not work.

What Works:

  • directly connecting to the share via the local network IP address of sys-net (port fowarding)

What does NOT work:

  • auto-discovering the shares on other LAN devices (multicasting most likely)

From my understanding, this “auto-discovery” works via the “wsdd2” package (debian). This service provides the capability and this has been confirmed by disabling / enabling it in a test install.

Specifically, it is done via port 3702 TCP/UDP multi-cast. In the docs the for it you can see details: wsdd2(8) — wsdd2 — Debian trixie — Debian Manpages

The multi-cast group address is 239.255.255.250 according to the docs. I am not an expert on this stuff, so anything could be wrong. This is just my understanding of how everything works.

Samba depends on port 445 TCP. Connecting to this port directly via the LAN IP will work. But the “auto-discovering” provided by “wsdd2” does NOT work.

I have created 3 different scripts to try and get this to work. Somewhere there must be a problem with the NFT rules, but I do not understand NFT enough to identify it. The scripts were adapted from the scripts provided for localsend multicast: WillyJL ─ LocalSend through 2 layers of NAT

sys-net script:

#!/bin/bash

iface=wls6  # discover via 'ip -a '
localip=192.168.1.75 # discover via 'ip -a '
firewallip=10.138.28.178 # look in qubes manager


########################################################################
# multi-casting
########################################################################
if true; then
	echo "Multi-casting executing"
	nft add chain qubes custom-prerouting
	nft insert rule qubes prerouting jump custom-prerouting

	port=3702  
	multicast=239.255.255.250  # WSDD multicast group
	nft add rule qubes custom-prerouting  iif != $iface udp dport $port ip daddr $multicast ip ttl set 2 ip saddr set $localip dup to $multicast device $iface notrack


	port=5355  
	multicast=224.0.0.252 # LLMNR query multicast group
	nft add rule qubes custom-prerouting  iif != $iface udp dport $port ip daddr $multicast ip ttl set 2 ip saddr set $localip dup to $multicast device $iface notrack
fi

########################################################################
# port forwarding 
########################################################################
nft add chain qubes custom-dnat-appvm '{ type nat hook prerouting priority filter +1; policy accept; }'

port=445  
for proto in tcp udp; do
    nft add rule qubes custom-dnat-appvm iif == $iface $proto dport $port ct state new,established,related counter dnat $firewallip
    nft add rule qubes custom-forward iif == $iface ip daddr $firewallip $proto dport $port ct state new,established,related counter accept
done

port=3702  
for proto in tcp udp; do
    nft add rule qubes custom-dnat-appvm iif == $iface $proto dport $port ct state new,established,related counter dnat $firewallip
    nft add rule qubes custom-forward iif == $iface ip daddr $firewallip $proto dport $port ct state new,established,related counter accept
done

port=5355  
for proto in tcp udp; do
    nft add rule qubes custom-dnat-appvm iif == $iface $proto dport $port ct state new,established,related counter dnat $firewallip
    nft add rule qubes custom-forward iif == $iface ip daddr $firewallip $proto dport $port ct state new,established,related counter accept
done

sys-firewall script:

#!/bin/bash

iface=eth0  # always eth0
appvmip=10.137.0.33 # look in qubes manager

########################################################################
# multi-casting
########################################################################
if true; then
	echo "Multi-casting executing"
	nft add chain qubes custom-prerouting
	nft insert rule qubes prerouting jump custom-prerouting

	port=3702
	multicast=239.255.255.250  # WSDD multicast group
	nft add rule qubes custom-prerouting iif != $iface udp dport $port ip daddr $multicast ip ttl set 2 dup to $multicast device $iface notrack


	port=5355
	multicast=224.0.0.252 # LLMNR query multicast group
	nft add rule qubes custom-prerouting iif != $iface udp dport $port ip daddr $multicast ip ttl set 2 dup to $multicast device $iface notrack
fi

########################################################################
# port forwarding 
########################################################################

nft add chain qubes custom-dnat-appvm '{ type nat hook prerouting priority filter +1; policy accept; }'

port=445
for proto in tcp udp; do
    nft add rule qubes custom-dnat-appvm  iif == $iface $proto dport $port ct state new,established,related counter  dnat $appvmip
    nft add rule qubes custom-forward iif == $iface ip daddr $appvmip $proto dport $port ct state new,established,related counter accept
done

port=3702
for proto in tcp udp; do
    nft add rule qubes custom-dnat-appvm  iif == $iface $proto dport $port ct state new,established,related counter  dnat $appvmip
    nft add rule qubes custom-forward iif == $iface ip daddr $appvmip $proto dport $port ct state new,established,related counter accept
done

port=5355
for proto in tcp udp; do
    nft add rule qubes custom-dnat-appvm  iif == $iface $proto dport $port ct state new,established,related counter  dnat $appvmip
    nft add rule qubes custom-forward iif == $iface ip daddr $appvmip $proto dport $port ct state new,established,related counter accept
done

appvm script:

#!/bin/bash

iface=eth0 # always eth0
appvmip=10.137.0.33 # look in qubes manager


########################################################################
# port forwarding 
########################################################################

port=445
for proto in tcp udp; do
    nft add rule qubes custom-input iif == $iface $proto dport $port ip daddr $appvmip ct state new,established,related counter accept
done

port=3702
for proto in tcp udp; do
    nft add rule qubes custom-input iif == $iface $proto dport $port ip daddr $appvmip ct state new,established,related counter accept
done

port=5355
for proto in tcp udp; do
    nft add rule qubes custom-input iif == $iface $proto dport $port ip daddr $appvmip ct state new,established,related counter accept
done

I haven’t tried to set up Samba auto-discovery, but since you tried mimicking the localsend guide without success I’ll point out that the following guide is likely to be the most helpful:

Thanks that is what I am basing my script on is that tutorial. However, for some reason it still does not work and I cannot figure out why. My scripts appear correct, but it won’t auto discover.

Hi @QAnon7346. Which scenario are you trying to achieve:

  1. AppVM → LAN: One AppVM on Qubes OS to auto-discover a device sitting on your local area network;
  2. LAN → AppVM: One device on your LAN to auto-discover an AppVM on Qubes OS.

I’ve found the solution to both, however, I want to give you the proper and easiest instructions.

Hello @deadbeef sorry for taking so long. I am looking for a device on the LAN to auto-discover the samba share provided in the AppVM. However, if you know how to do both and its not burdensome then knowing how to do both would be nice.

I look forward to any solution you can provide. I still have not been able to resolve it and have had to resort to manually and explicitly connecting to shares.

Hey. I’ll first share the LAN → AppVM discovery. The trick to make this work, and it’s important you know, is tricking wsdd2 into thinking it’s directly connected to the local network. Off course, it’s an illusion. As far as I understood, such a thing is mostly required due to M$ ws-discovery protocol. So that means the AppVM must know sys-net’s LAN IP address, and because of that I suggest you assign a static LAN IP to your sys-net. Hope that make sense.

To keep things simple, keep the follwogin variables in your head, because I’ll refer to them during the instructions:

  • APPVM: the name of the VM running wsdd2;
  • ETH_IFACE: network interface name inside “sys-net” which is directly connected to your LAN (eg, ens6);
  • VM_IP: IP address of your APPVM (eg, “10.137.0.57”);
  • EXT_LAN_CIDR: LAN IP address of your “sys-net” in CIDR notation (eg, “192.168.1.5/24”) - this is the LAN IP address I mentioned earlier.

Note: I’ll give every script a name, like “lan2appvm-*” so I can easily refer to them, and you’ll know what the hell I’m talking about.

I tested these following instructions as follows:

  • sys-net: Debian 13 - I guess it should work on Fedora as well, I just didn’t test
  • sys-firewall: Fedora 43 - I guess it should work on Debian as well, I just didn’t test
  • appvm: Debian 13 - I tried making it work on Fedora, and it should work, but Fedora wsdd and samba packages are different, so I just gave trying to test on Fedora. The scripts should work on Fedora tho

testing first

Let’s forget about “how to make this thing work automatically at boot”, alright? I’ll show you how you can test this with your running “sys-net” and “sys-firewall”. If all works out, at last I show you how to run them at boot.

sys-net

Step one: in dom0, run the following command manually in a terminal:

qvm-features sys-net vm-config.smb-eth-iface <ETH_IFACE>

Change “<ETH_IFACE>” with your ETH_IFACE, which I mentioned earlier. This is just an easy for the script I wrote to find out the ETH_IFACE without hardcoding that value. I mean, you can now easily change your ETH_IFACE from dom0, restart “sys-net”, and it will reflect your changes.


Two scripts involved. Why? Because I cared to create something that works reliably at boot (at least that’s what I expect).

Step two: we’ll add the network hook in “sys-net”. First, ensure this special directory exists (ref.: Config files — Qubes OS 4.3.1 documentation), by running this command in “sys-net” as root:

install -o root -g root -m 755 -d /rw/config/network-hooks.d/

Next, copy the following script to such directory (the destination name must be lan2appvm-net-hook.sh, it’s the name I use in the next script):

lan2appvm-net-hook.sh
#!/bin/bash
set -eo pipefail

nft flush chain qubes custom-smb-prerouting
nft flush chain qubes custom-smb-dnat
nft flush set qubes smb_vifs
nft flush set qubes smb_ips

eth_iface="$(qubesdb-read /vm-config/smb-eth-iface)"
if [ -z "$eth_iface" ]; then
    echo "[error] missing qvm-feature vm-config.smb-eth-iface" >&2
    exit 1
fi

for addr in $(qubesdb-list /qubes-firewall-handled/); do
    vif="$(ip -o route get "$addr" | cut -d' ' -f3)"

    # WS discovery - the probe
    nft add rule qubes custom-smb-prerouting \
        iif "$eth_iface" \
        ip daddr 239.255.255.250 \
        udp dport 3702 \
        ip ttl set 2 \
        counter \
        dup to 239.255.255.250 \
            device "$vif" notrack

    # LMNR query - the name to IP address translation
    nft add rule qubes custom-smb-prerouting \
        iif "$eth_iface" \
        ip daddr 224.0.0.252 \
        udp dport 5355 \
        ip ttl set 2 \
        counter \
        dup to 224.0.0.252 \
            device "$vif" notrack

    nft add rule qubes custom-smb-dnat \
        iif "$eth_iface" \
        tcp dport { 3702, 445 } \
        ct state established,related,new counter \
        dnat to "$addr"

    nft add element qubes smb_vifs { "$vif" }
    nft add element qubes smb_ips { "$addr" }
done

Step three: go ahead and run the following script inside your “sys-net”. No arguments are required. Example: sudo ./lan2appvm-net.sh

lan2appvm-net.sh
#!/bin/bash
set -eo pipefail

eth_iface="$(qubesdb-read /vm-config/smb-eth-iface)"
if [ -z "$eth_iface" ]; then
    echo "[error] missing qvm-feature vm-config.smb-eth-iface" >&2
    exit 1
fi

nft add chain qubes custom-smb-dnat '{ type nat hook prerouting priority dstnat; policy accept; }'
nft add chain qubes custom-smb-prerouting '{}'

nft add set qubes smb_vifs '{ type ifname; }'
nft add set qubes smb_ips '{ type ipv4_addr; }'

nft insert rule qubes prerouting jump custom-smb-prerouting

# make the kernel listen to ws-discovery and llmnr multicast pkts
ip maddr add 01:00:5e:7f:ff:fa dev "$eth_iface"
ip maddr add 01:00:5e:00:00:fc dev "$eth_iface"

# second ws-discovery stage and smb
nft add rule qubes custom-forward \
    iif "$eth_iface" \
    oifname @smb_vifs \
    ip daddr @smb_ips \
    tcp dport { 3702, 445 } \
    ct state established,related,new counter \
    accept

/rw/config/network-hooks.d/lan2appvm-net-hook.sh

sys-firewall

Two scripts involved, again. Why? It works reliably when you shutdown or start your APPVM. Meaning, when you start your APPVM, a script will be called inside “sys-firewall” to set up the firewall rules, so you can start and shutdown many times, and it should always work.


Step one: in dom0, run the following command manually in a terminal:

qvm-features sys-firewall vm-config.smb-backend-ip <VM_IP>

Change “<VM_IP>” with your VM_IP, which I mention earlier. This is just an easy for the script I wrote to find out the VM_IP without hardcoding that value. I mean, you can now easily change your VM_IP in dom0, and the script running in “sys-firewall” will reflect that change accordingly.


Step two: we’ll add the network hook in “sys-firewall”. First, ensure this special directory exists (ref.: Config files — Qubes OS 4.3.1 documentation), by running this command in “sys-firewall” as root:

install -o root -g root -m 755 -d /rw/config/network-hooks.d/

Next, copy the following script to such directory (the destination name must be lan2appvm-firewall-hook.sh, it’s the name I use in the next script):

lan2appvm-firewall-hook.sh
#!/usr/bin/bash
. /usr/lib/qubes/init/functions # get_qubes_managed_iface()

managed_vif="$(get_qubes_managed_iface)"

command="$1"
vif="$2"
vif_type="$3"
ip="$4"

qubes_ip="$(qubesdb-read /vm-config/smb-backend-ip)"
if [ -z "$qubes_ip" ]; then
    echo "[error] missing qvm-feature vm-config.smb-backend-ip" >&2
    exit 1
fi

if [ "$ip" = "$qubes_ip" ] && [ "$vif_type" = 'vif' ]; then
    nft flush chain qubes custom-smb-prerouting
    nft flush chain qubes custom-smb-dnat
    nft flush set qubes smb_vifs
    nft flush set qubes smb_ips

    vif="$(ip -o route get "$qubes_ip" | cut -d' ' -f3)"

    if ! ip link show "$vif" >/dev/null 2>&1; then
        echo "[error] no qubes found with given ip: $qubes_ip" >&2
        exit 2
    fi

    if [ "$command" = 'online' ]; then
        # forward WS discovery - the probe
        nft add rule qubes custom-smb-prerouting \
            iif "$managed_vif" \
            udp dport 3702 \
            ip daddr 239.255.255.250 \
            ip ttl set 2 counter \
            dup to 239.255.255.250 \
                device "$vif" notrack

        # forward LMNR query - the name to IP address translation
        nft add rule qubes custom-smb-prerouting \
            iif "$managed_vif" \
            udp dport 5355 \
            ip daddr 224.0.0.252 \
            ip ttl set 2 \
            counter \
            dup to 224.0.0.252 \
                device "$vif" notrack


        # forward smb and ws-discovery second stage
        nft add rule qubes custom-smb-dnat \
            iif "$managed_vif" \
            tcp dport { 3702, 445 } \
            ct state established,related,new counter \
            dnat to "$qubes_ip"

        nft add element qubes smb_vifs { "$vif" }
        nft add element qubes smb_ips { "$qubes_ip" }
    fi
fi
install -o root -g root -m 755 lan2appvm-firewall-hook.sh /rw/config/network-hooks.d/lan2appvm-firewall-hook.sh

Step three: the following script should be executed once in “sys-firewall”, and sets up the infrastructure that will be changed dynamically over time. It does not require parameters. Go ahead and execute it as “root” in “sys-firewall”.

lan2appvm-firewall.sh
#!/bin/bash
. /usr/lib/qubes/init/functions # get_qubes_managed_iface()

managed_vif="$(get_qubes_managed_iface)"

nft add set qubes smb_vifs '{ type ifname; }'
nft add set qubes smb_ips '{ type ipv4_addr; }'

nft add chain qubes custom-smb-dnat '{ type nat hook prerouting priority dstnat; policy accept; }'

nft add chain qubes custom-smb-prerouting '{}'
nft insert rule qubes prerouting jump custom-smb-prerouting

# allow second ws-discovery stage and smb
nft add rule qubes custom-forward \
    oifname @smb_vifs \
    ip daddr @smb_ips \
    tcp dport { 3702, 445 } \
    ct state established,related,new counter \
    accept

qubes_ip="$(qubesdb-read /vm-config/smb-backend-ip)"
/rw/config/network-hooks.d/lan2appvm-firewall-hook.sh online '' '' "$qubes_ip"

It may exit with failure and an error message like…

[error] no qubes found with given ip: ...

… that’s fine. It just means your APPVM is powered off and not connected at the moment.

appvm

Run the following script in your APPVM, as root. It expects as first and only argument the EXT_LAN_CIDR. Example: sudo ./lan2appvm-appvm.sh 192.168.1.5/24

lan2appvm-appvm.sh
#!/bin/bash
. /usr/lib/qubes/init/functions # get_qubes_managed_iface()

EXT_LAN_CIDR="$1"

set -euo pipefail
if [ -z "$EXT_LAN_CIDR" ]; then
    echo "Usage: $0 EXT_LAN_CIDR" >&2
    echo "" >&2
    echo "EXT_LAN_CIDR    the address of your external sys-net inside the local network, in CIDR format (eg.: 192.168.1.5/24)" >&2
    exit 1
fi

EXT_LAN_IP="${EXT_LAN_CIDR%/*}"
EXT_LAN_MASK="${EXT_LAN_CIDR#*/}"

if [ -z "$EXT_LAN_IP" ] || [ -z "$EXT_LAN_MASK" ] || [ "$EXT_LAN_MASK" = "$EXT_LAN_CIDR" ]; then
    echo "error: invalid $EXT_LAN_CIDR. expected format IP/MASK, eg.: 192.168.1.5/24" >&2
    exit 2
fi

my_ip="$(qubesdb-read /qubes-ip)"
managed_vif="$(get_qubes_managed_iface)"

# add external lan ip to vif, so wsdd2 can bind on it
ip add add "$EXT_LAN_CIDR" dev "$managed_vif"

# make sys-firewall happy, our appvm believes it's $EXT_LAN_IP,
# but sys-firewall keep treating us normally
nft insert rule qubes postrouting \
    ip saddr "$EXT_LAN_IP" \
    udp sport { 3702, 5355 } \
    snat to "$my_ip"

# WS discovery - allow probe in
nft add rule qubes custom-input \
    udp dport 3702 \
    ip daddr 239.255.255.250 \
    counter \
    accept

# LMNR query - the name to IP address translation, allow in
nft add rule qubes custom-input \
    udp dport 5355 \
    ip daddr 224.0.0.252 \
    counter \
    accept

# allow in smb and ws-discovery second stage
nft add rule qubes custom-input \
    ip daddr "$EXT_LAN_IP" \
    tcp dport { 445, 3702 } \
    ct state established,related,new \
    counter \
    accept

nft add chain qubes custom-dnat-appvm '{ type nat hook prerouting priority filter +1; policy accept; }'

# wsdd2 and the outside world still thinks we are $EXT_LAN_IP, so behave like it
nft add rule qubes custom-dnat-appvm \
    iif "$managed_vif" \
    ip daddr "$my_ip" \
    tcp dport { 445, 3702 } \
    ct state established,related,new \
    counter \
    dnat to "$EXT_LAN_IP"

Next, you should restart wsdd2 and samba services. I’ve tested this on Debian only, run as “root”:

systemctl restart wsdd2 smbd nmbd winbind

does it work?

That’s it. If everything worked out fine, keep reading how you can run them at boot.

persistence

Usually it seems recommended to create a new TemplateVM for this kind of custom stuff. So keep that in mind. The following instructions should work in whatever TemplateVM, though.

persistence: sys-net

Step one: in your “sys-net” template, and I mean your TemplateVM, copy “lan2appvm-net.sh” and “lan2appvm-net-hook.sh” scripts to usr/bin, run as “root”:

install -o root -g root -m 755 lan2appvm-net.sh lan2appvm-net-hook.sh /usr/bin/

Step two: copy this new script “qubes-smb-netvm.sh” to usr/bin.

qubes-smb-netvm.sh
#!/bin/sh
install -o root -g root -m 755 -d /rw/config/network-hooks.d/
install -o root -g root -m 755 /usr/bin/lan2appvm-net-hook.sh /rw/config/network-hooks.d/
exec /usr/bin/lan2appvm-net.sh

Run as “root”:

install -o root -g root -m 755 qubes-smb-netvm.sh /usr/bin/

Step three: copy this new “qubes-smb-netvm.service” to “/etc/systemd/system” and enable that service:

qubes-smb-netvm.service
[Unit]
Description=Forward ws-discovery and stuff like that to sys-firewalls
After=qubes-mount-dirs.service qubes-firewall.service rw.mount
DefaultDependencies=no
ConditionPathExists=/var/run/qubes/this-is-netvm

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/qubes-smb-netvm.sh

[Install]
WantedBy=multi-user.target

Run as “root”:

install -o root -g root -m 644 qubes-smb-netvm.service /etc/systemd/system
systemctl daemon-reload
systemctl enable qubes-smb-netvm

persistence: sys-firewall

Step one: in your “sys-firewall” template, and I mean your TemplateVM, copy the “lan2appvm-firewall.sh” and “lan2appvm-firewall-hook.sh” scripts to usr/bin, run as “root”:

install -o root -g root -m 755 lan2appvm-firewall.sh lan2appvm-firewall-hook.sh /usr/bin/

Step two: copy this new script called “qubes-smb-forwarder.sh” to usr/bin:

qubes-smb-forwarder.sh
#!/bin/sh
install -o root -g root -m 755 -d /rw/config/network-hooks.d/
install -o root -g root -m 755 /usr/bin/lan2appvm-firewall-hook.sh /rw/config/network-hooks.d/
exec /usr/bin/lan2appvm-firewall.sh

Run as “root”:

install -o root -g root -m 755 qubes-smb-forwarder.sh /usr/bin/

Step three: copy the “qubes-smb-forwarder.service” to “/etc/systemd/system” and enable that service:

qubes-smb-forwarder.service
[Unit]
Description=Forward ws-discovery and stuff like that to appvm
After=qubes-mount-dirs.service qubes-firewall.service rw.mount
DefaultDependencies=no
ConditionPathExists=/var/run/qubes/this-is-proxyvm

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/qubes-smb-forwarder.sh

[Install]
WantedBy=multi-user.target

Run as “root”:

install -o root -g root -m 644 qubes-smb-forwarder.service /etc/systemd/system
systemctl daemon-reload
systemctl enable qubes-smb-forwarder

persistence: appvm

I’ll assume this is really a VM of type “AppVM”. Therefore, you have a persistent “/home” and a “/rw/config/rc.local”. So…

Step one: copy “lan2appvm-appvm.sh” to a known location, run in appvm:

install -m 755 lan2appvm-appvm.sh /home/user/

Step two: add the following line to your /rw/config/rc.local file:

/home/user/lan2appvm-appvm.sh <EXT_LAN_CIDR>
systemctl restart wsdd2 smbd nmbd winbind

Change “<EXT_LAN_CIDR>” with your EXT_LAN_CIDR.

other stuff

  • Almost all my nft rules have counters. You can use them for troubleshooting, if things do not work. You probably should remove them later;

Let me know how it goes ; )