[qubes-users] Tails VM: network broken since Qubes r4.2 (was online in r4.1)

Hello everyone,

I have a difficult time with my Tails VM in Qubes (which I need for Tails specific developing and documentation work).
It gets no network connectivity no matter what I try. With "network connectivity" I mean the Tails VM can't even ping any network VM.

I set up a Tails VM [1] a while ago on an up-to-date Qubes r4.1 system (so it should be similar to r4.2?). After assigning the Tails VM a static ip [2],
it was online right away. Now I had to reinstall Qubes on new hardware, and installed r4.2. I copied the old Tails VM into the r4.2, and it is stuck offline.
I then created a new Tails VM, exactly the same way I did before with [1] and [2], it couldn't reach any networking VM neither.
Next, I purged iptable [3], removed all routes [4] except the default route and shutdown all network devices except eth0 [5].
Still, there is no ping response even from the networking VM (which does reply to other VM's pings).

Finally, I used a regular Debian 12 live image to create another standalone VM with [1]. It was online right away.
Tails is based on Debian 12 too.
The only meaningful difference between the Tails and the Debian VMs I could find was that their default routes [6] look a bit different, where I don't
know if this might be related.

So it does look like a Tails problem after all. But then, why was the same Tails VM online when hosted by an up-to-date r4.1 Qubes and offline on
a fresh installed r4.2 Qubes?
I found hints online that others experience the same [7] symptoms of non reachable networking VMs, where r4.1 vs r4.2 was brought up.

Does anyone have suggestions what else I might check and try?
I would be very grateful for any help. It would feel archaic and counterproductive to use another machine for working on Tails..

Stickstoff

[1] Installing a live linux into a standalone Qubes vm:
Create a new standalone qube: HVM, 2GB+ memory.
dom0: sudo sh -c 'qvm-run --pass-io BrowserVM "cat ~/downloads/tailsimage.img"' > /tmp/tailsimage.img
dom0: sudo dd if=/dev/zero of=root.img bs=1 count=0 seek=8G # new empty 8GB root.img as sparse file
dom0: sudo dd bs=32M conv=notrunc status=progress if=/tmp/tailsimage.img of=root.img # copy the image to the start of root.img
Tails: remove "live-media=removable" in grub bootloader (necessary at each boot of Tails)

[2] Setting up networking in Tails:
dom0: qvm-ls -n TailsVM # get the IP that dom0 assigned to the Tails VM
Tails: set static ip, netmask, gateway and dns

[3] purge iptable rules, allow everything:
Tails: sudo iptables -F
Tails: sudo iptables -X
Tails: sudo iptables -P INPUT ACCEPT
Tails: sudo iptables -P OUTPUT ACCEPT
Tails: sudo iptables -P FORWARD ACCEPT

[4] purge routes and add new default route:
Tails: sudo ip route del <rule>
Tails: sudo ip route add default via 10.137.0.9 dev eth0

[5] shutdown network devices:
sudo ip link set dev <devicename> down

[6]
ip route Tails:
default via 10.137.0.9 dev eth0 proto static metric 100
10.137.0.0/24 dev eth0 proto kernel scope link src 10.137.0.32 metric 100
^^^^^^^^^^^^^

ip route Debian:
default via 10.137.0.9 dev enX0 proto static metric 100
10.137.0.9 dev enX0 proto kernel scope link src 10.137.0.32 metric 100
^^^^^^^^^^

[7]

2 Likes

Does using the static route you have in Debian, and adding static
neighbor entries for the peer, fix the problem? If not, can you try
this command?

$ sudo ip neighbour replace to 10.137.0.9 dev eth0 \
  lladdr fe:ff:ff:ff:ff:ff nud permanent

That adds a permanent neighbour entry. If it changes stuff it means
that ARP is broken.
- --
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

2 Likes

Thank you for looking into this!

Does using the static route you have in Debian, and adding static
neighbor entries for the peer, fix the problem?

I noticed the default routes actually differ a bit more between Tails and Debian [8] than I wrote
in my original mail (I have to type anything going in and out of Tails, no global clipboard yet).
I replaced the Tails default route with the Debian default route, but that did not help.
As I only try to ping the network VM, in the same ip network, routes shouldn't be necessary
nor interfere I guess?

If not, can you try this command?
$ sudo ip neighbour replace to 10.137.0.9 dev eth0 lladdr fe:ff:ff:ff:ff:ff nud permanent

That adds a permanent neighbour entry. If it changes stuff it means
that ARP is broken.

I added that lookup entry, the network VM is still not reachable.
Afterwards, I checked the ARP cache. The ips of both VMs I tried to reach (firewall and networking)
are listed, with the same fe:ff:ff:ff:ff:ff mac. The newly added entry has "flags mask: CM", the other
has "C".

Stickstoff

[8]
Tails default route:
default via 10.137.0.9 dev eth0 proto static metric 100
10.137.0.0/24 dev eth0 proto kernel scope link src 10.137.0.32 metric 100

Debian default route:
default via 10.137.0.9 dev enX0 proto static metric 100
10.137.0.9 dev enX0 proto static scope link metric 100

2 Likes