[qubes-users] Networking issue after upgrading to Fedora-33

Hi everyone,

I’m looking for some help as to how to diagnose some app-VM networking issues. I have 2 vms, both based on the same template with identical config, but one can reach the internet and the other cannot.

Before upgrading:
2 standalone VMs based on Fedora-30. One with a bunch of dev tools installed, one relatively untouched. I had multiple VMs based on these two templates. I also updated my sys-net and sys-firewall to Fedora-33 at the same time.

Upgrade:
I upgraded to Fedora-33, and realised I could rationalise my VMs, so now every appVM is based off the same Fedora-33 template.

The issue:
Some of my migrated VMs are completely fine, others have no network.

I think it is the VMs that used to be based on my old “untouched” vm that have the issue.

VM1:
No networking at all.

VM2:
Networking is completely fine, everything works as expected.

Both VMs are based on the same Fedora-33 template, with the same (default) sys-firewall Networking, both with the firewall configured to allow all outgoing internet connections

vm1$ ping google.com
ping: google.com: Name or service not known

vm1$ dig google.com

; <<>> DiG 9.11.35-RedHat-9.11.35-1.fc33 <<>> google.com
;; global options: +cmd
;; connection timed out; no servers could be reached

vm1$ resolvectl dns
Global: 10.139.1.1 10.139.1.2
Link 2 (eth0):

vm2$ resolvectl dns
Global: 10.139.1.1 10.139.1.2
Link 2 (eth0):
Link 3 (br-11bfb2cd10e9):
Link 4 (docker0):
Link 5 (br-cf58034d074b):
Link 6 (br-f9686c41a7f5):

So there’s definitely something wrong, but I don’t know enough about Linux/Qubes networking to work out what.

Any pointers gratefully received.

There is something wrong, but there is nothing in the detail you have
provided that might explain it.
So:
Do you have any custom firewall rules set on vm1? (qvm-firewall vm1)
Can you ping the firewall from vm1, by IP address?
Can you access a host on the internet by IP address?(e.g https://9.9.9.9)
If you create a new qube from the template, does networking work as
expected?
If you change templates on vm1, does networking work?

Yes, there are custom firewall rules, but the firewall is set to “Allow all outgoing internet connections”. Which should ignore all the rules?

Yes, I can happily ping sys-firewall’s IP from both vm1 and vm2.

No, I can’t access a host on the internet by IP

vm1$ curl https://9.9.9.9
curl: (7) Failed to connect to 9.9.9.9 port 443: No route to host

Yes, creating a new qube from the same template works fine - networking is exactly as expected.

No, changing templates on vm1 doesn’t fix it (I thought it did when I tried a month or so ago, but I just gave up without really trying to get to the bottom of what’s wrong. Either way, it doesn’t work now)

Changing vm1’s network from sys-firewall to sys-net doesn’t fix it, either.

But this is interesting though. This is what I get when pinging the IP of sys-net (whilst networking is set to sys-firewall):

vm1$ ping 10.137.0.5
PING 10.137.0.5 (10.137.0.5) 56(84) bytes of data.
From 10.137.0.6 icmp_seq=1 Packet filtered
From 10.137.0.6 icmp_seq=2 Packet filtered
From 10.137.0.6 icmp_seq=3 Packet filtered
[…]
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2077ms

(0.5 is the IP of sys-net and 0.6 is sys-firewall)

So somehow when I ping sys-net I get a response from sys-firewall. Huh!?

This is what happens on my working VM:

vm2$ ping 10.137.0.5
PING 10.137.0.5 (10.137.0.5) 56(84) bytes of data.
64 bytes from 10.137.0.5: icmp_seq=1 ttl=63 time=0.286 ms
64 bytes from 10.137.0.5: icmp_seq=2 ttl=63 time=0.529 ms
64 bytes from 10.137.0.5: icmp_seq=3 ttl=63 time=0.551 ms

AFAIK, if you set custom firewall rules, they override the GUI firewall
setting.
Inspect the rules applying on sys-firewall.