Internet Routing Issues

User Support General
21

personal qube (attached to sys-firewall)

[user@personal-fedora-34 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
8: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:16:3e:5e:6c:00 brd ff:ff:ff:ff:ff:ff
    inet 10.137.0.17/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe5e:6c00/64 scope link 
       valid_lft forever preferred_lft forever
[user@personal-fedora-34 ~]$ ip r
default via 10.138.30.106 dev eth0 onlink 
10.138.30.106 dev eth0 scope host onlink 
[user@personal-fedora-34 ~]$

sys-firewall:

[user@sys-firewall ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:16:3e:5e:6c:00 brd ff:ff:ff:ff:ff:ff
    inet 10.138.30.106/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe5e:6c00/64 scope link 
       valid_lft forever preferred_lft forever
3: vif6.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 32
    link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
    inet 10.138.30.106/32 scope global vif6.0
       valid_lft forever preferred_lft forever
    inet6 fe80::fcff:ffff:feff:ffff/64 scope link 
       valid_lft forever preferred_lft forever
9: vif9.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 32
    link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
    inet 10.138.30.106/32 scope global vif9.0
       valid_lft forever preferred_lft forever
    inet6 fe80::fcff:ffff:feff:ffff/64 scope link 
       valid_lft forever preferred_lft forever
[user@sys-firewall ~]$ ip r
default via 10.137.0.5 dev eth0 onlink 
10.137.0.5 dev eth0 scope host onlink 
10.137.0.17 dev vif9.0 scope link metric 32743 
10.137.0.29 dev vif6.0 scope link metric 32746

Can you ping sys-firewall IP from personal qube?

How do I do that?

FYI: I was wondering why only sys-firewall is able to receive internet from sys-net. So I executed in dom0

qvm-clone sys-firewall sys-firewall-clone

started the qube and tried to ping but no success. Very weird.

Edit:
I was able to ping sys-firewall from personal qube

[user@personal-fedora-34 ~]$ ping 10.138.30.106
PING 10.138.30.106 (10.138.30.106) 56(84) bytes of data.
64 bytes from 10.138.30.106: icmp_seq=1 ttl=64 time=0.095 ms
64 bytes from 10.138.30.106: icmp_seq=2 ttl=64 time=0.113 ms
64 bytes from 10.138.30.106: icmp_seq=3 ttl=64 time=0.116 ms
^C
--- 10.138.30.106 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2048ms
rtt min/avg/max/mdev = 0.095/0.108/0.116/0.009 ms
[user@personal-fedora-34 ~]$
22

Also try to ping from personal qube sys-net IP (10.137.0.5) and your router IP (check ip r in sys-net terminal).

23

personal (no access to either sys-net and router)

[user@personal-fedora-34 ~]$ ping 10.137.0.5
PING 10.137.0.5 (10.137.0.5) 56(84) bytes of data.
^C
--- 10.137.0.5 ping statistics ---
18 packets transmitted, 0 received, 100% packet loss, time 17419ms

[user@personal-fedora-34 ~]$ ping 192.168.178.1
PING 192.168.178.1 (192.168.178.1) 56(84) bytes of data.
^C
--- 192.168.178.1 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3102ms

[user@personal-fedora-34 ~]$ 


[user@personal-fedora-34 ~]$ ^C

sys-firewall can access both sys-net and router.

[user@sys-firewall ~]$ ping 10.137.0.5
PING 10.137.0.5 (10.137.0.5) 56(84) bytes of data.
64 bytes from 10.137.0.5: icmp_seq=1 ttl=64 time=0.087 ms
64 bytes from 10.137.0.5: icmp_seq=2 ttl=64 time=0.076 ms
^C
--- 10.137.0.5 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1063ms
rtt min/avg/max/mdev = 0.076/0.081/0.087/0.005 ms
[user@sys-firewall ~]$ 


[user@sys-firewall ~]$ ping 192.168.178.1
PING 192.168.178.1 (192.168.178.1) 56(84) bytes of data.
64 bytes from 192.168.178.1: icmp_seq=1 ttl=63 time=1.42 ms
64 bytes from 192.168.178.1: icmp_seq=2 ttl=63 time=1.09 ms
64 bytes from 192.168.178.1: icmp_seq=3 ttl=63 time=1.05 ms
^C
--- 192.168.178.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.051/1.186/1.420/0.165 ms
[user@sys-firewall ~]$
24

What do you have in sys-firewall and sys-net firewall rules?

sudo nft list table qubes-firewall
sudo iptables -L -n -v
sudo iptables -t nat -L -n -v
25

How do I copy output from sys-net shell? Itā€™s a weird bash shell.

26

sys-firewall:

[user@sys-firewall ~]$ 
[user@sys-firewall ~]$ 
[user@sys-firewall ~]$ sudo nft list table qubes-firewall
table ip qubes-firewall {
	chain forward {
		type filter hook forward priority filter; policy drop;
		ct state established,related accept
		iifname != "vif*" accept
		ip saddr 10.137.0.29 jump qbs-10-137-0-29
	}

	chain prerouting {
		type filter hook prerouting priority raw; policy accept;
		iifname != "vif*" ip saddr { 10.137.0.8, 10.137.0.18, 10.137.0.26, 10.137.0.29, 10.137.0.32, 10.137.0.37, 10.137.0.43, 10.137.0.46, 10.137.0.48, 10.137.0.50, 10.137.0.52, 10.137.0.56, 10.137.0.58, 10.137.0.59, 10.137.0.61 } drop
	}

	chain postrouting {
		type filter hook postrouting priority raw; policy accept;
		oifname != "vif*" ip daddr { 10.137.0.8, 10.137.0.18, 10.137.0.26, 10.137.0.29, 10.137.0.32, 10.137.0.37, 10.137.0.43, 10.137.0.46, 10.137.0.48, 10.137.0.50, 10.137.0.52, 10.137.0.56, 10.137.0.58, 10.137.0.59, 10.137.0.61 } drop
	}

	chain qbs-10-137-0-29 {
		accept
		reject with icmp type admin-prohibited
	}
}





[user@sys-firewall ~]$ sudo iptables -L -n -v
Chain INPUT (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID
    0     0 DROP       udp  --  vif+   *       0.0.0.0/0            0.0.0.0/0            udp dpt:68
 906K  867M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    3   252 ACCEPT     icmp --  vif+   *       0.0.0.0/0            0.0.0.0/0           
    4   286 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
    0     0 REJECT     all  --  vif+   *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID
 629K  323M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    1   114 QBS-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  vif+   vif+    0.0.0.0/0            0.0.0.0/0           
    1   114 ACCEPT     all  --  vif+   *       0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 352K packets, 101M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain QBS-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination         







[user@sys-firewall ~]$ sudo iptables -t nat -L -n -v
Chain PREROUTING (policy ACCEPT 13509 packets, 1132K bytes)
 pkts bytes target     prot opt in     out     source               destination         
27068 2017K PR-QBS     all  --  *      *       0.0.0.0/0            0.0.0.0/0           
13509 1132K PR-QBS-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain INPUT (policy ACCEPT 3 packets, 252 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 6791 packets, 1155K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   51  9511 ACCEPT     all  --  *      vif+    0.0.0.0/0            0.0.0.0/0           
    4   286 ACCEPT     all  --  *      lo      0.0.0.0/0            0.0.0.0/0           
 6737 1145K MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain PR-QBS (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 6797  444K DNAT       udp  --  *      *       0.0.0.0/0            10.139.1.1           udp dpt:53 to:10.139.1.1
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            10.139.1.1           tcp dpt:53 to:10.139.1.1
 6762  441K DNAT       udp  --  *      *       0.0.0.0/0            10.139.1.2           udp dpt:53 to:10.139.1.2
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            10.139.1.2           tcp dpt:53 to:10.139.1.2

Chain PR-QBS-SERVICES (1 references)
 pkts bytes target     prot opt in     out     source               destination
27

Weird bash shell? Xterm? Donā€™t you have GNOME Terminal app there?

28

sys-net wonā€™t open a gnome shell.
in dom0

[user@dom0 Desktop]$ sudo qvm-run -- sys-net  'sh -c "exec gnome-terminal || exec xterm"' 
Running 'sh -c "exec gnome-terminal || exec xterm"' on sys-net

is getting stuck. So I open a terminal in sys-net via qubes-manager ā†’ right-click on sys-net ā†’ open terminal

Edit: This happens sometimes, sometimes not.

29

Your routing and firewall rules looks ok, so the problem is somewhere else.
This problem with sys-net terminal looks strange, maybe itā€™s related to this network issue.
Check the memory usage in sys-net xterm free -m.
And check the cpu load with top.
Maybe the terminal is not opening because thereā€™s not enough memory or some process caused cpu lockup.
Another cause may be that there was some failure during update.

30

Maybe the easiest way to fix it is to just install clean template fedora-35 and just use it.

31

Ok. How do I recreate sys-net and sys-firewall from scratch with the new template?

32

You can just choose fedora-35 instead of fedora-34 template in their Qube Settings.
But for sys-firewall, since itā€™s disposable, then youā€™ll need to create fedora-35-dvm first:

But if you want to create them from scratch you can refer to this guide:

UPD:
Iā€™ve referenced to the wrong guide and Iā€™ve updated the link, so check the new one.

33

So you suggest not to recreate them?

34

If they will work if you just change the templates then you can leave the old ones.

35

I create the fedora-35-dvm via

 qvm-create --template fedora-35 --label red fedora-35-dvm
[user@dom0 ~]$ qvm-prefs fedora-35-dvm template_for_dispvms True
[user@dom0 ~]$ qvm-features fedora-35-dvm appmenus-dispvm 1

right?

36

Yes.

37

Same issue. I have network access in both sys-net and sys-firewall (and personal if I use the routing sys-net>sys-firewall>vpn-gateway>personal)

Allthough I still canā€™t open a gnome terminal in sys-net and in qubes-manager it shows a yellow state of sys-net.

Edit: When I attach personal directly to sys-net then I get

[user@personal-fedora-34 ~]$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
From 10.137.0.17 icmp_seq=1 Destination Host Unreachable
From 10.137.0.17 icmp_seq=2 Destination Host Unreachable
From 10.137.0.17 icmp_seq=3 Destination Host Unreachable
From 10.137.0.17 icmp_seq=4 Destination Host Unreachable
From 10.137.0.17 icmp_seq=5 Destination Host Unreachable
From 10.137.0.17 icmp_seq=6 Destination Host Unreachable
From 10.137.0.17 icmp_seq=7 Destination Host Unreachable
From 10.137.0.17 icmp_seq=8 Destination Host Unreachable
From 10.137.0.17 icmp_seq=9 Destination Host Unreachable
From 10.137.0.17 icmp_seq=10 Destination Host Unreachable
From 10.137.0.17 icmp_seq=11 Destination Host Unreachable
From 10.137.0.17 icmp_seq=12 Destination Host Unreachable
^C
--- 8.8.8.8 ping statistics ---
13 packets transmitted, 0 received, +12 errors, 100% packet loss, time 12290ms
pipe 3
[user@personal-fedora-34 ~]$
38

Did you try to change your personal qube template to fedora-35?
Try to create new sys-net based on fedora-35 and see if itā€™ll have the same problem when you open gnome terminal.

39

Yes, but that didnā€™t work.

To be sure:

I use this commands and fedora-35-dvm as ā€œdisposable-Template-Nameā€, right?

40

Yes.
When you try to open gnome terminal in this new disp-sys-net you still have the same problem with it not opening?