VMs depending on a VPN ProxyVM cannot reach the DNS server

Since I upgraded my VPN ProxyVM (sys-vpn) from fedora-39-xfce to fedora-40-xfce, its depending VMs cannot reach the DNS server anymore.

Here is how to reproduce the problem:

  • dom0: Install fedora-39-xfce VM
  • dom0: Update fedora-39-xfce VM
  • dom0: Create a new VM:
    • Name: sys-vpn
    • Type: AppVM
    • Template: fedora-39-xfce
    • Networking: sys-firewall
    • provides network: true
    • Services:
      • network-manager
  • dom0: Create a new VM:
    • Name: test-vpn
    • Type: DispVM
    • Template: Some AppVM
    • Networking: sys-vpn
  • dom0: Start sys-vpn VM
  • sys-vpn: Setup VPN according to NetworkManager documentation:
    • In my case:
      • Left-click on the NetworkManager Systray Icon
      • VPN ConnectionsConfigure VPN...
      • Click on the + button
      • Choose Import a saved VPN configuration...
      • Add username and password to the configuration
  • sys-vpn: Check nft

Result:

$ sudo nft list chain ip qubes dnat-dns
table ip qubes {
	chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		ip daddr 10.139.1.1 udp dport 53 dnat to 10.139.1.1
		ip daddr 10.139.1.1 tcp dport 53 dnat to 10.139.1.1
		ip daddr 10.139.1.2 udp dport 53 dnat to 10.139.1.2
		ip daddr 10.139.1.2 tcp dport 53 dnat to 10.139.1.2
	}
}
  • sys-vpn: Start VPN
  • sys-vpn: Check nft

Result:

$ sudo nft list chain ip qubes dnat-dns
table ip qubes {
	chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		ip daddr 10.139.1.1 udp dport 53 dnat to 1.1.1.1
		ip daddr 10.139.1.1 tcp dport 53 dnat to 1.1.1.1
		ip daddr 10.139.1.2 udp dport 53 dnat to 8.8.8.8
		ip daddr 10.139.1.2 tcp dport 53 dnat to 8.8.8.8
	}
}

sys-vpn reroute correctly the DNS request to the proper DNS server

Result:

$ ping www.google.com
PING www.google.com (64.233.177.99) 56(84) bytes of data.
64 bytes from yx-in-f99.1e100.net (64.233.177.99): icmp_seq=2 ttl=55 time=36.1 ms
64 bytes from yx-in-f99.1e100.net (64.233.177.99): icmp_seq=3 ttl=55 time=35.4 ms
64 bytes from yx-in-f99.1e100.net (64.233.177.99): icmp_seq=4 ttl=55 time=35.9 ms
^C
--- www.google.com ping statistics ---
4 packets transmitted, 3 received, 25% packet loss, time 3033ms
rtt min/avg/max/mdev = 35.380/35.790/36.111/0.305 ms

test-vpn is able to reach the DNS server and Google.

  • dom0: Turn off test-vpn VM
  • dom0: Turn off sys-vpn VM
  • dom0: Install fedora-40-xfce VM
  • dom0: Update fedora-40-xfce VM
  • dom0: In the Settings of sys-vpn VM, change:
    • Template: fedora-39-xfce to fedora-40-xfce
  • dom0: Start sys-vpn VM
  • sys-vpn: Check nft

Result:

$ sudo nft list chain ip qubes dnat-dns
table ip qubes {
	chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		ip daddr 10.139.1.1 udp dport 53 dnat to 10.139.1.1
		ip daddr 10.139.1.1 tcp dport 53 dnat to 10.139.1.1
		ip daddr 10.139.1.2 udp dport 53 dnat to 10.139.1.2
		ip daddr 10.139.1.2 tcp dport 53 dnat to 10.139.1.2
	}
}
  • sys-vpn: Fix manually nft
nft flush chain ip qubes dnat-dns
nft 'add rule ip qubes dnat-dns ip daddr 10.139.1.1 udp dport 53 dnat to 1.1.1.1'
nft 'add rule ip qubes dnat-dns ip daddr 10.139.1.1 tcp dport 53 dnat to 1.1.1.1'
nft 'add rule ip qubes dnat-dns ip daddr 10.139.1.2 udp dport 53 dnat to 8.8.8.8'
nft 'add rule ip qubes dnat-dns ip daddr 10.139.1.2 tcp dport 53 dnat to 8.8.8.8'
  • sys-vpn: Check nft

Result:

$ sudo nft list chain ip qubes dnat-dns
table ip qubes {
	chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		ip daddr 10.139.1.1 udp dport 53 dnat to 1.1.1.1
		ip daddr 10.139.1.1 tcp dport 53 dnat to 1.1.1.1
		ip daddr 10.139.1.2 udp dport 53 dnat to 8.8.8.8
		ip daddr 10.139.1.2 tcp dport 53 dnat to 8.8.8.8
	}
}
  • sys-vpn: Start VPN
  • sys-vpn: Check nft

Result:

$ sudo nft list chain ip qubes dnat-dns
table ip qubes {
	chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		ip daddr 10.139.1.1 udp dport 53 dnat to 10.139.1.1
		ip daddr 10.139.1.1 tcp dport 53 dnat to 10.139.1.1
		ip daddr 10.139.1.2 udp dport 53 dnat to 10.139.1.2
		ip daddr 10.139.1.2 tcp dport 53 dnat to 10.139.1.2
	}
}

Even though I manually reroute correctly to the DNS server, when I start the VPN, NetworkManager reroutes the DNS in the wrong way.

Result:

$ ping www.google.com
<nothing>
  • sys-vpn: Fix manually nft
  • sys-vpn: Check nft

Result:

$ sudo nft list chain ip qubes dnat-dns
table ip qubes {
	chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		ip daddr 10.139.1.1 udp dport 53 dnat to 1.1.1.1
		ip daddr 10.139.1.1 tcp dport 53 dnat to 1.1.1.1
		ip daddr 10.139.1.2 udp dport 53 dnat to 8.8.8.8
		ip daddr 10.139.1.2 tcp dport 53 dnat to 8.8.8.8
	}
}

Result:

$ ping www.google.com
PING www.google.com (172.217.215.99) 56(84) bytes of data.
64 bytes from yo-in-f99.1e100.net (172.217.215.99): icmp_seq=1 ttl=56 time=33.8 ms
64 bytes from yo-in-f99.1e100.net (172.217.215.99): icmp_seq=2 ttl=56 time=34.3 ms
64 bytes from yo-in-f99.1e100.net (172.217.215.99): icmp_seq=3 ttl=56 time=34.4 ms
^C
--- www.google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 33.788/34.179/34.446/0.282 ms

Observations:

  • When using the fedora-39-xfce in sys-vpn, starting the VPN reroutes:
    10.139.1.1:53 → 1.1.1.1,
    10.139.1.2:53 → 8.8.8.8.
    The depending VM test-vpn can reach the DNS server.
  • When using the fedora-40-xfce in sys-vpn, starting the VPN reroutes:
    10.139.1.1:53 → 10.139.1.1,
    10.139.1.2:53 → 10.139.1.2,
    The depending VM test-vpn cannot reach the DNS server.

Questions:

  • How can I make NetworkManager reroute:
    10.139.1.1:53 → 1.1.1.1,
    10.139.1.2:53 → 8.8.8.8,
    when I start the VPN?

Do you use Wireguard, OpenVPN or some other VPN client?
Maybe you’re missing some package in the new template that was installed in the old template like openresolv.
Check the VPN client log, maybe there will be info about DNS servers configuration.

I use OpenVPN. I start the VPN from the NetworkManager systray icon.

Regarding missing packages, I did a diff on the packages list of both sys-vpn on fedora-39-xfce and sys-vpn2 on fedora-40-xfce:

sys-vpn$ sudo dnf -C list installed | awk '{ print $1 }' > pkg.old.txt
sys-vpn$ qvm-copy pkg.old.txt # to sys-vpn2
sys-vpn2$ sudo dnf -C list installed | awk '{ print $1 }' > pkg.new.txt
sys-vpn2$ diff -ua pkg.old.txt pkg.new.txt
--- pkg.old.txt	2024-11-15 10:05:22.803948685 -0500
+++ pkg.new.txt	2024-11-15 10:53:07.900019726 -0500
@@ -52,6 +52,7 @@
 adwaita-cursor-theme.noarch
 adwaita-gtk2-theme.x86_64
 adwaita-icon-theme.noarch
+adwaita-icon-theme-legacy.noarch
 alsa-lib.x86_64
 alsa-sof-firmware.noarch
 alsa-ucm.noarch
@@ -77,6 +78,7 @@
 augeas-libs.x86_64
 authselect.x86_64
 authselect-libs.x86_64
+avahi.x86_64
 avahi-glib.x86_64
 avahi-libs.x86_64
 avif-pixbuf-loader.x86_64
@@ -155,9 +157,73 @@
 dbus-tools.x86_64
 dbus-x11.x86_64
 dconf.x86_64
+default-fonts-am.noarch
+default-fonts-ar.noarch
+default-fonts-as.noarch
+default-fonts-ast.noarch
+default-fonts-be.noarch
+default-fonts-bg.noarch
+default-fonts-bn.noarch
+default-fonts-bo.noarch
+default-fonts-br.noarch
+default-fonts-chr.noarch
+default-fonts-cjk-mono.noarch
+default-fonts-cjk-sans.noarch
+default-fonts-cjk-serif.noarch
+default-fonts-core-emoji.noarch
+default-fonts-core-math.noarch
+default-fonts-core-mono.noarch
 default-fonts-core-sans.noarch
+default-fonts-core-serif.noarch
+default-fonts-dv.noarch
+default-fonts-dz.noarch
+default-fonts-el.noarch
+default-fonts-eo.noarch
+default-fonts-eu.noarch
+default-fonts-fa.noarch
+default-fonts-gu.noarch
+default-fonts-he.noarch
+default-fonts-hi.noarch
+default-fonts-hy.noarch
+default-fonts-ia.noarch
+default-fonts-iu.noarch
+default-fonts-ka.noarch
+default-fonts-km.noarch
+default-fonts-kn.noarch
+default-fonts-ku.noarch
+default-fonts-lo.noarch
+default-fonts-mai.noarch
+default-fonts-ml.noarch
+default-fonts-mni.noarch
+default-fonts-mr.noarch
+default-fonts-my.noarch
+default-fonts-nb.noarch
+default-fonts-ne.noarch
+default-fonts-nn.noarch
+default-fonts-nr.noarch
+default-fonts-nso.noarch
+default-fonts-or.noarch
+default-fonts-other-mono.noarch
+default-fonts-other-sans.noarch
+default-fonts-other-serif.noarch
+default-fonts-pa.noarch
+default-fonts-ru.noarch
+default-fonts-sat.noarch
+default-fonts-si.noarch
+default-fonts-ss.noarch
+default-fonts-ta.noarch
+default-fonts-te.noarch
+default-fonts-th.noarch
+default-fonts-tn.noarch
+default-fonts-ts.noarch
+default-fonts-uk.noarch
+default-fonts-ur.noarch
+default-fonts-ve.noarch
+default-fonts-vi.noarch
+default-fonts-xh.noarch
+default-fonts-yi.noarch
+default-fonts-zu.noarch
 dejavu-sans-mono-fonts.noarch
-deltarpm.x86_64
 desktop-file-utils.x86_64
 device-mapper.x86_64
 device-mapper-libs.x86_64
@@ -202,7 +268,7 @@
 exo.x86_64
 expat.x86_64
 f2fs-tools.x86_64
-f39-backgrounds-base.noarch
+f40-backgrounds-base.noarch
 fakeroot.x86_64
 fakeroot-libs.x86_64
 farstream02.x86_64
@@ -259,6 +325,7 @@
 gcr3.x86_64
 gcr3-base.x86_64
 gdb-headless.x86_64
+gdbm.x86_64
 gdbm-libs.x86_64
 gdk-pixbuf2.x86_64
 gdk-pixbuf2-modules.x86_64
@@ -301,10 +368,59 @@
 gobject-introspection.x86_64
 goffice.x86_64
 google-droid-sans-fonts.noarch
+google-noto-color-emoji-fonts.noarch
 google-noto-fonts-common.noarch
+google-noto-naskh-arabic-vf-fonts.noarch
+google-noto-sans-arabic-vf-fonts.noarch
+google-noto-sans-armenian-vf-fonts.noarch
+google-noto-sans-bengali-vf-fonts.noarch
+google-noto-sans-canadian-aboriginal-vf-fonts.noarch
+google-noto-sans-cherokee-vf-fonts.noarch
+google-noto-sans-cjk-vf-fonts.noarch
+google-noto-sans-devanagari-vf-fonts.noarch
+google-noto-sans-ethiopic-vf-fonts.noarch
+google-noto-sans-georgian-vf-fonts.noarch
+google-noto-sans-gujarati-vf-fonts.noarch
+google-noto-sans-gurmukhi-vf-fonts.noarch
+google-noto-sans-hebrew-vf-fonts.noarch
+google-noto-sans-kannada-vf-fonts.noarch
+google-noto-sans-khmer-vf-fonts.noarch
+google-noto-sans-lao-vf-fonts.noarch
+google-noto-sans-math-fonts.noarch
+google-noto-sans-meeteimayek-vf-fonts.noarch
+google-noto-sans-mono-cjk-vf-fonts.noarch
+google-noto-sans-mono-vf-fonts.noarch
+google-noto-sans-ol-chiki-vf-fonts.noarch
+google-noto-sans-oriya-vf-fonts.noarch
+google-noto-sans-sinhala-vf-fonts.noarch
+google-noto-sans-symbols-vf-fonts.noarch
+google-noto-sans-symbols2-fonts.noarch
+google-noto-sans-tamil-vf-fonts.noarch
+google-noto-sans-telugu-vf-fonts.noarch
+google-noto-sans-thaana-vf-fonts.noarch
+google-noto-sans-thai-vf-fonts.noarch
 google-noto-sans-vf-fonts.noarch
+google-noto-serif-armenian-vf-fonts.noarch
+google-noto-serif-bengali-vf-fonts.noarch
+google-noto-serif-cjk-vf-fonts.noarch
+google-noto-serif-devanagari-vf-fonts.noarch
+google-noto-serif-ethiopic-vf-fonts.noarch
+google-noto-serif-georgian-vf-fonts.noarch
+google-noto-serif-gujarati-vf-fonts.noarch
+google-noto-serif-gurmukhi-vf-fonts.noarch
+google-noto-serif-hebrew-vf-fonts.noarch
+google-noto-serif-kannada-vf-fonts.noarch
+google-noto-serif-khmer-vf-fonts.noarch
+google-noto-serif-lao-vf-fonts.noarch
+google-noto-serif-oriya-vf-fonts.noarch
+google-noto-serif-sinhala-vf-fonts.noarch
+google-noto-serif-tamil-vf-fonts.noarch
+google-noto-serif-telugu-vf-fonts.noarch
+google-noto-serif-thai-vf-fonts.noarch
+google-noto-serif-vf-fonts.noarch
 gparted.x86_64
 gpgme.x86_64
+gpgmepp.x86_64
 gpm-libs.x86_64
 graphene.x86_64
 graphite2.x86_64
@@ -321,6 +437,7 @@
 grub2-pc-modules.noarch
 grub2-tools.x86_64
 grub2-tools-minimal.x86_64
+grubby-dummy.noarch
 gsettings-desktop-schemas.x86_64
 gsm.x86_64
 gspell.x86_64
@@ -341,7 +458,7 @@
 gtkmm4.0.x86_64
 gtksourceview4.x86_64
 gtkspell.x86_64
-guile22.x86_64
+guile30.x86_64
 gupnp.x86_64
 gupnp-igd.x86_64
 gvfs.x86_64
@@ -383,34 +500,31 @@
 iwlegacy-firmware.noarch
 iwlwifi-dvm-firmware.noarch
 iwlwifi-mvm-firmware.noarch
+jack-audio-connection-kit.x86_64
 jansson.x86_64
 jasper-libs.x86_64
 javascriptcoregtk4.1.x86_64
 jbig2dec-libs.x86_64
 jbigkit-libs.x86_64
+jomolhari-fonts.noarch
 jq.x86_64
-js-jquery.noarch
 json-c.x86_64
 json-glib.x86_64
 jxl-pixbuf-loader.x86_64
 kbd.x86_64
 kbd-legacy.noarch
 kbd-misc.noarch
+kdump-utils.x86_64
 keepassxc.x86_64
 kernel.x86_64
 kernel.x86_64
-kernel.x86_64
-kernel-core.x86_64
 kernel-core.x86_64
 kernel-core.x86_64
 kernel-headers.x86_64
 kernel-modules.x86_64
 kernel-modules.x86_64
-kernel-modules.x86_64
 kernel-modules-core.x86_64
 kernel-modules-core.x86_64
-kernel-modules-core.x86_64
-kernel-modules-extra.x86_64
 kernel-modules-extra.x86_64
 kernel-modules-extra.x86_64
 kernel-srpm-macros.noarch
@@ -512,8 +626,10 @@
 libcloudproviders.x86_64
 libcom_err.x86_64
 libcomps.x86_64
+libconfig.x86_64
 libcue.x86_64
 libcurl.x86_64
+libdaemon.x86_64
 libdatrie.x86_64
 libdav1d.x86_64
 libdb.x86_64
@@ -537,6 +653,7 @@
 libexif.x86_64
 libfdisk.x86_64
 libfdt.x86_64
+libffado.x86_64
 libffi.x86_64
 libfido2.x86_64
 libfontenc.x86_64
@@ -677,7 +794,6 @@
 libshout.x86_64
 libsigc++20.x86_64
 libsigc++30.x86_64
-libsigsegv.x86_64
 libslirp.x86_64
 libsmartcols.x86_64
 libsmbclient.x86_64
@@ -700,6 +816,7 @@
 libtasn1.x86_64
 libtdb.x86_64
 libtevent.x86_64
+libtextstyle.x86_64
 libthai.x86_64
 libtheora.x86_64
 libtiff.x86_64
@@ -736,17 +853,18 @@
 libwebp.x86_64
 libwmf-lite.x86_64
 libwnck3.x86_64
-libwpe.x86_64
 libxcb.x86_64
 libxcrypt.x86_64
 libxcrypt-devel.x86_64
 libxdo.x86_64
+libxdp.x86_64
 libxfce4ui.x86_64
 libxfce4util.x86_64
 libxkbcommon.x86_64
 libxkbcommon-x11.x86_64
 libxkbfile.x86_64
 libxklavier.x86_64
+libxml++.x86_64
 libxml2.x86_64
 libxmlb.x86_64
 libxshmfence.x86_64
@@ -773,7 +891,9 @@
 lv2.x86_64
 lz4-libs.x86_64
 lzo.x86_64
+madan-fonts.noarch
 make.x86_64
+makedumpfile.x86_64
 man-db.x86_64
 man-pages.noarch
 mate-desktop-libs.x86_64
@@ -808,8 +928,7 @@
 mesa-va-drivers.x86_64
 mesa-vulkan-drivers.x86_64
 miniupnpc.x86_64
-minizip-compat.x86_64
-minizip-ng.x86_64
+minizip-ng-compat.x86_64
 mint-x-icons.noarch
 mint-y-icons.noarch
 mint-y-theme.noarch
@@ -870,7 +989,6 @@
 openssh-askpass.x86_64
 openssh-clients.x86_64
 openssl-libs.x86_64
-openssl-pkcs11.x86_64
 openvpn.x86_64
 opus.x86_64
 opus-tools.x86_64
@@ -883,6 +1001,7 @@
 p11-kit-trust.x86_64
 p7zip-plugins.x86_64
 package-notes-srpm-macros.noarch
+paktype-naskh-basic-fonts.noarch
 pam.x86_64
 pam-libs.x86_64
 pango.x86_64
@@ -890,6 +1009,8 @@
 pangomm2.48.x86_64
 parole.x86_64
 parted.x86_64
+passim.x86_64
+passim-libs.x86_64
 pavucontrol.x86_64
 pcaudiolib.x86_64
 pciutils.x86_64
@@ -935,7 +1056,6 @@
 perl-Digest.noarch
 perl-Digest-MD5.x86_64
 perl-Digest-SHA.x86_64
-perl-Digest-SHA1.x86_64
 perl-DirHandle.noarch
 perl-Dumpvalue.noarch
 perl-DynaLoader.x86_64
@@ -994,7 +1114,6 @@
 perl-IPC-Open3.noarch
 perl-IPC-SysV.x86_64
 perl-IPC-System-Simple.noarch
-perl-Importer.noarch
 perl-JSON-PP.noarch
 perl-Locale-Maketext.noarch
 perl-Locale-Maketext-Simple.noarch
@@ -1003,7 +1122,6 @@
 perl-MRO-Compat.noarch
 perl-Math-BigInt.noarch
 perl-Math-BigInt-FastCalc.x86_64
-perl-Math-BigRat.noarch
 perl-Math-Complex.noarch
 perl-Memoize.noarch
 perl-Module-Build.noarch
@@ -1138,13 +1256,12 @@
 pinentry-gnome3.x86_64
 pipewire.x86_64
 pipewire-alsa.x86_64
-pipewire-jack-audio-connection-kit.x86_64
-pipewire-jack-audio-connection-kit-libs.x86_64
 pipewire-libs.x86_64
 pipewire-pulseaudio.x86_64
 pipewire-qubes.x86_64
 pixman.x86_64
 pkcs11-helper.x86_64
+pkcs11-provider.x86_64
 pkgconf.x86_64
 pkgconf-m4.noarch
 pkgconf-pkg-config.x86_64
@@ -1171,7 +1288,6 @@
 pyproject-srpm-macros.noarch
 python-pip-wheel.noarch
 python-srpm-macros.noarch
-python-systemd-doc.x86_64
 python-unversioned-command.noarch
 python3.x86_64
 python3-abrt.x86_64
@@ -1185,7 +1301,6 @@
 python3-click.noarch
 python3-croniter.noarch
 python3-cups.x86_64
-python3-daemon.noarch
 python3-dateutil.noarch
 python3-dbus.x86_64
 python3-distro.noarch
@@ -1197,7 +1312,6 @@
 python3-gbulb.x86_64
 python3-gobject.x86_64
 python3-gobject-base.x86_64
-python3-gobject-base-noarch.noarch
 python3-hawkey.x86_64
 python3-humanize.noarch
 python3-idna.noarch
@@ -1210,7 +1324,6 @@
 python3-libs.x86_64
 python3-libselinux.x86_64
 python3-libsemanage.x86_64
-python3-lockfile.noarch
 python3-looseversion.noarch
 python3-lxml.x86_64
 python3-markupsafe.x86_64
@@ -1225,7 +1338,6 @@
 python3-psutil.x86_64
 python3-pycparser.noarch
 python3-pycryptodomex.x86_64
-python3-pycurl.x86_64
 python3-pyparsing.noarch
 python3-pysocks.noarch
 python3-pyxdg.noarch
@@ -1308,14 +1420,16 @@
 realtek-firmware.noarch
 redhat-rpm-config.noarch
 ristretto.x86_64
+rit-meera-new-fonts.noarch
+rit-rachana-fonts.noarch
 rpm.x86_64
 rpm-build-libs.x86_64
 rpm-libs.x86_64
+rpm-plugin-audit.x86_64
 rpm-plugin-selinux.x86_64
 rpm-plugin-systemd-inhibit.x86_64
 rpm-sequoia.x86_64
 rpm-sign-libs.x86_64
-rpmautospec-rpm-macros.noarch
 rpmfusion-free-release.noarch
 rpmfusion-nonfree-release.noarch
 rsvg-pixbuf-loader.x86_64
@@ -1329,7 +1443,6 @@
 samba-common.noarch
 samba-common-libs.x86_64
 satyr.x86_64
-sdubby.noarch
 seabios-bin.noarch
 seahorse.x86_64
 seavgabios-bin.noarch
@@ -1341,6 +1454,8 @@
 setxkbmap.x86_64
 shadow-utils.x86_64
 shared-mime-info.x86_64
+sil-nuosu-fonts.noarch
+sil-padauk-fonts.noarch
 slang.x86_64
 snappy.x86_64
 socat.x86_64
@@ -1352,6 +1467,7 @@
 soxr.x86_64
 speech-dispatcher.x86_64
 speech-dispatcher-espeak-ng.x86_64
+speech-dispatcher-libs.x86_64
 speech-dispatcher-utils.x86_64
 speex.x86_64
 spirv-tools-libs.x86_64
@@ -1362,6 +1478,7 @@
 sshpass.x86_64
 sstp-client.x86_64
 startup-notification.x86_64
+stix-fonts.noarch
 stoken-libs.x86_64
 strongswan.x86_64
 strongswan-charon-nm.x86_64
@@ -1371,7 +1488,6 @@
 system-config-printer.x86_64
 system-config-printer-libs.noarch
 systemd.x86_64
-systemd-boot-unsigned.x86_64
 systemd-libs.x86_64
 systemd-networkd.x86_64
 systemd-pam.x86_64
@@ -1402,7 +1518,6 @@
 transmission-common.x86_64
 transmission-gtk.x86_64
 tree.x86_64
-trousers-lib.x86_64
 ttmkfdir.x86_64
 tumbler.x86_64
 twolame-libs.x86_64
@@ -1430,12 +1545,14 @@
 urw-base35-z003-fonts.noarch
 usb_modeswitch.x86_64
 usb_modeswitch-data.noarch
+usbmuxd.x86_64
 usbutils.x86_64
 userspace-rcu.x86_64
 util-linux.x86_64
 util-linux-core.x86_64
 vamp-plugin-sdk.x86_64
 vapoursynth-libs.x86_64
+vazirmatn-vf-fonts.noarch
 vid.stab.x86_64
 vim-common.x86_64
 vim-data.noarch
@@ -1461,7 +1578,7 @@
 wireplumber-libs.x86_64
 woff2.x86_64
 wpa_supplicant.x86_64
-wpebackend-fdo.x86_64
+wsdd.noarch
 xarchiver.x86_64
 xcb-util.x86_64
 xcb-util-image.x86_64
@@ -1476,6 +1593,7 @@
 xdg-utils.noarch
 xdotool.x86_64
 xdpyinfo.x86_64
+xdriinfo.x86_64
 xen-hypervisor.x86_64
 xen-libs.x86_64
 xen-licenses.x86_64
@@ -1539,7 +1657,8 @@
 zchunk-libs.x86_64
 zenity.x86_64
 zeromq.x86_64
+zig-srpm-macros.noarch
 zimg.x86_64
 zip.x86_64
-zlib.x86_64
+zlib-ng-compat.x86_64
 zvbi.x86_64

openresolv was not installed in fedora-39-xfce, nor in fedora-40-xfce.

Regarding VPN client log, I am not sure where to look for them, so I did a journalctl in both sys-vpn and sys-vpn2. Here are the relevent parts:

  • sys-vpn:
NetworkManager[641]: <info>  [1731683860.6246] vpn[0x5e360718c930,0a5ff129-b04c-42b3-854a-bcdfb3300d3c,"BytzVPNAtlanta-NoAds"]: starting openvpn
NetworkManager[641]: <info>  [1731683860.6255] audit: op="connection-activate" uuid="0a5ff129-b04c-42b3-854a-bcdfb3300d3c" name="BytzVPNAtlanta-NoAds" pid=1018 uid=1000 result="success"
nm-openvpn[3394]: OpenVPN 2.6.9 x86_64-redhat-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
nm-openvpn[3394]: library versions: OpenSSL 3.1.4 24 Oct 2023, LZO 2.10
nm-openvpn[3394]: DCO version: N/A
nm-openvpn[3394]: NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
nm-openvpn[3394]: TCP/UDP: Preserving recently used remote address: [AF_INET]23.92.29.231:993
nm-openvpn[3394]: UDPv4 link local: (not bound)
nm-openvpn[3394]: UDPv4 link remote: [AF_INET]23.92.29.231:993
nm-openvpn[3394]: NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
nm-openvpn[3394]: [server] Peer Connection Initiated with [AF_INET]23.92.29.231:993
nm-openvpn[3394]: TUN/TAP device tun0 opened
nm-openvpn[3394]: /usr/libexec/nm-openvpn-service-openvpn-helper --debug 0 3384 --bus-name org.freedesktop.NetworkManager.openvpn.Connection_17 --tun -- tun0 1280 0 10.8.0.5 255.255.0.0 init
NetworkManager[641]: <info>  [1731683862.5483] manager: (tun0): new Tun device (/org/freedesktop/NetworkManager/Devices/14)
nm-openvpn[3394]: UID set to nm-openvpn
nm-openvpn[3394]: GID set to nm-openvpn
nm-openvpn[3394]: Capabilities retained: CAP_NET_ADMIN
nm-openvpn[3394]: Initialization Sequence Completed
NetworkManager[641]: <info>  [1731683862.5685] device (tun0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
NetworkManager[641]: <info>  [1731683862.5696] device (tun0): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
NetworkManager[641]: <info>  [1731683862.5701] device (tun0): Activation: starting connection 'tun0' (26ae60f2-c83f-4690-9b68-ae87fb1446eb)
NetworkManager[641]: <info>  [1731683862.5702] device (tun0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
NetworkManager[641]: <info>  [1731683862.5704] device (tun0): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
NetworkManager[641]: <info>  [1731683862.5706] device (tun0): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
NetworkManager[641]: <info>  [1731683862.5708] device (tun0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
systemd[1]: Starting NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service...
systemd[1]: Started NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service.
audit[1]: SERVICE_START pid=1 uid=0 auid=4294967296 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager-dispatcher comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
kernel: audit: type=1130 audit(1731683862.590:640): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager-dispatcher comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
NetworkManager[641]: <info>  [1731683862.5935] policy: set 'BytzVPNAtlanta-NoAds' (tun0) as default for IPv4 routing and DNS
systemd-resolved[429]: eth0: Bus client set default route setting: no
systemd-resolved[429]: eth0: Bus client reset DNS server list.
systemd-resolved[429]: tun0: Bus client set default route setting: yes
systemd-resolved[429]: tun0: Bus client set DNS server list to: 1.1.1.1, 8.8.8.8
NetworkManager[641]: <info>  [1731683862.6123] device (tun0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external')
NetworkManager[641]: <info>  [1731683862.6126] device (tun0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external')
NetworkManager[641]: <info>  [1731683862.6129] device (tun0): Activation: successful, device activated.
audit[3410]: NETFILTER_CFG table=qubes:76 family=2 entries=11 op=nft_register_chain pid=3410 subj=system_u:system_r:NetworkManager_dispatcher_custom_t:s0 comm="nft"
audit[3410]: SYSCALL arch=c000003e syscall=46 success=yes exit=1932 a0=3 a1=7ffe46b36680 a2=0 a3=736c0daf9c84 items=0 ppid=3409 pid=3410 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="nft" exe="/usr/sbin/nft" subj=system_u:system_r:NetworkManager_dispatcher_custom_t:s0 key=(null)
audit: PROCTITLE proctitle=6E6674002D2D00616464207461626C652069702071756265730A61646420636861696E20697020717562657320646E61742D646E730A64656C65746520636861696E20697020717562657320646E61742D646E730A7461626C65206970207175626573207B0A636861696E20646E61742D646E73207B0A74797065206E617420
kernel: audit: type=1325 audit(1731683862.672:641): table=qubes:76 family=2 entries=11 op=nft_register_chain pid=3410 subj=system_u:system_r:NetworkManager_dispatcher_custom_t:s0 comm="nft"
kernel: audit: type=1300 audit(1731683862.672:641): arch=c000003e syscall=46 success=yes exit=1932 a0=3 a1=7ffe46b36680 a2=0 a3=736c0daf9c84 items=0 ppid=3409 pid=3410 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="nft" exe="/usr/sbin/nft" subj=system_u:system_r:NetworkManager_dispatcher_custom_t:s0 key=(null)
  • sys-vpn2:
NetworkManager[638]: <info>  [1731683971.6431] vpn[0x5578ed446b00,d4ae74c7-cfa8-4925-b1ca-1aa0f73b0d61,"BytzVPNAtlanta-NoAds"]: starting openvpn
NetworkManager[638]: <info>  [1731683971.6438] audit: op="connection-activate" uuid="d4ae74c7-cfa8-4925-b1ca-1aa0f73b0d61" name="BytzVPNAtlanta-NoAds" pid=979 uid=1000 result="success"
nm-openvpn[2342]: DEPRECATED OPTION: --cipher set to 'AES-128-CBC' but missing in --data-ciphers (AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305). OpenVPN ignores --cipher for cipher negotiations.
nm-openvpn[2343]: OpenVPN 2.6.12 x86_64-redhat-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] [DCO]
nm-openvpn[2342]: library versions: OpenSSL 3.2.2 4 Jun 2024, LZO 2.10
nm-openvpn[2342]: DCO version: N/A
nm-openvpn[2342]: NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
nm-openvpn[2342]: TCP/UDP: Preserving recently used remote address: [AF_INET]23.92.29.231:993
nm-openvpn[2342]: UDPv4 link local: (not bound)
nm-openvpn[2342]: UDPv4 link remote: [AF_INET]23.92.29.231:993
nm-openvpn[2342]: NOTE: UID/GID downgrade will be delayed because of --client, --pull, or --up-delay
nm-openvpn[2342]: [server] Peer Connection Initiated with [AF_INET]23.92.29.231:993
nm-openvpn[2342]: TUN/TAP device tun0 opened
nm-openvpn[2342]: /usr/libexec/nm-openvpn-service-openvpn-helper --debug 0 2329 --bus-name org.freedesktop.NetworkManager.openvpn.Connection_12 --tun -- tun0 1280 0 10.8.0.9 255.255.0.0 init
NetworkManager[638]: <info>  [1731683973.6958] manager: (tun0): new Tun device (/org/freedesktop/NetworkManager/Devices/8)
nm-openvpn[2342]: UID set to nm-openvpn
nm-openvpn[2342]: GID set to nm-openvpn
nm-openvpn[2342]: Capabilities retained: CAP_NET_ADMIN
nm-openvpn[2342]: Initialization Sequence Completed
NetworkManager[638]: <info>  [1731683973.7373] policy: set 'BytzVPNAtlanta-NoAds' (tun0) as default for IPv4 routing and DNS
systemd-resolved[431]: eth0: Bus client set default route setting: no
systemd-resolved[431]: eth0: Bus client reset DNS server list.
systemd-resolved[431]: tun0: Bus client set default route setting: yes
systemd-resolved[431]: tun0: Bus client set DNS server list to: 1.1.1.1, 8.8.8.8
NetworkManager[638]: <info>  [1731683973.7630] device (tun0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
NetworkManager[638]: <info>  [1731683973.7633] device (tun0): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
NetworkManager[638]: <info>  [1731683973.7638] device (tun0): Activation: starting connection 'tun0' (96e3ab06-8460-4f0c-8567-ad87e63242a2)
NetworkManager[638]: <info>  [1731683973.7639] device (tun0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
NetworkManager[638]: <info>  [1731683973.7641] device (tun0): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
NetworkManager[638]: <info>  [1731683973.7642] device (tun0): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
NetworkManager[638]: <info>  [1731683973.7643] device (tun0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
NetworkManager[638]: <info>  [1731683973.7776] device (tun0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external')
NetworkManager[638]: <info>  [1731683973.7777] device (tun0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external')
NetworkManager[638]: <info>  [1731683973.7781] device (tun0): Activation: successful, device activated.
audit[2353]: NETFILTER_CFG table=qubes:27 family=2 entries=11 op=nft_register_chain pid=2353 subj=system_u:system_r:NetworkManager_dispatcher_custom_t:s0 comm="nft"
audit[2353]: SYSCALL arch=c000003e syscall=46 success=yes exit=1932 a0=3 a1=7ffe11d919c0 a2=0 a3=7ffe11d91b30 items=0 ppid=2352 pid=2353 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="nft" exe="/usr/sbin/nft" subj=system_u:system_r:NetworkManager_dispatcher_custom_t:s0 key=(null)
kernel: audit: type=1325 audit(1731683973.830:348): table=qubes:27 family=2 entries=11 op=nft_register_chain pid=2353 subj=system_u:system_r:NetworkManager_dispatcher_custom_t:s0 comm="nft"
kernel: audit: type=1300 audit(1731683973.830:348): arch=c000003e syscall=46 success=yes exit=1932 a0=3 a1=7ffe11d919c0 a2=0 a3=7ffe11d91b30 items=0 ppid=2352 pid=2353 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="nft" exe="/usr/sbin/nft" subj=system_u:system_r:NetworkManager_dispatcher_custom_t:s0 key=(null)

There are some minor difference between the logs, but I cannot tell how important they are. The only thing that stood out for me are the NetworkManager Script Dispatcher lines. I did this:

$ find /etc/NetworkManager/ -exec sha256sum {} \; >> hashes.txt
$ find /usr/lib/NetworkManager/dispatcher.d -exec sha256sum {} \; >> hashes.txt

on both sys-vpn and sys-vpn2. Both hashes.txt look like this:

b0b0effe09927299fb6a192f0d89eaf91ec19fc70f65c586f011614a3d8591b9  /etc/NetworkManager/NetworkManager.conf
aa9941fbf140d9d045f7e6a8cd654235e3c9800bb5c485f3873d6b9039b76c5a  /etc/NetworkManager/dispatcher.d/30-qubes-external-ip
b720c772abed4b9bd2ee6d5270c04307239613304a288d12fb31076137763310  /etc/NetworkManager/dispatcher.d/qubes-nmhook
a42ad7a25adb83fe57ee2d62d6d053bebb6fa1878f80291d6d12c177c651c52c  /usr/lib/NetworkManager/dispatcher.d/11-dhclient

The dnat-dns update works sometimes in fedora-40-xfce, but it’s unreliable.
There seems to be some problem with systemd-resolved, maybe some kind of a race condition or something else.
Restarting systemd-resolved before updating dnat-dns seems to fix the issue:

So if you add:

systemctl restart systemd-resolved

Before execution of:

/usr/lib/qubes/qubes-setup-dnat-to-ns

In the /etc/NetworkManager/dispatcher.d/qubes-nmhook file then it’ll work.
No idea what’s wrong with it, I guess it’s better to create an issue on github for this.

1 Like

I think the actual problem is that /usr/lib/qubes/qubes-setup-dnat-to-ns is getting the array of all available DNS servers for all links and not the ones that are used for default route link.
It’s getting an array of [10.139.1.1,10.139.1.1,1.1.1.1,8.8.8.8] and then it’s creating dnat-dns rules for first two DNS servers in the array which are Qubes OS Virtual DNS servers and not the ones used by VPN even though it’s a default route right now.
I think the correct way would be to get DNS only from the current default route link and use them.
More info:
Understanding systemd-resolved, Split DNS, and VPN Configuration – Michael Catanzaro's Blog
systemd-resolved and VPNs
org.freedesktop.resolve1

But maybe there are some cases where this would be a wrong approach, so I’m not sure.

I’ve edited the /usr/lib/qubes/qubes-setup-dnat-to-ns to make it work properly (based on the code from /usr/lib/python3.12/site-packages/blueman/main/DNSServerProvider.py), but I’m not sure if it’ll work properly in all cases, so I hope someone more experienced will review it.
Fixed file:

#!/usr/bin/python3
#
# The Qubes OS Project, http://www.qubes-os.org
#
# Copyright (C) 2022  Marek Marczykowski-Górecki
#                               <marmarek@invisiblethingslab.com>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#

from __future__ import annotations

import subprocess
import sys

import dbus
import qubesdb
from typing import List
from ipaddress import IPv4Address
import os

import re
import socket
from ipaddress import IPv4Address, ip_address
from typing import List, Generator

from gi.repository import GObject, Gio, GLib

class DNSServersDefaultRoute(GObject.GObject):
    RESOLVER_PATH = "/etc/resolv.conf"
    _RESOLVED_NAME = "org.freedesktop.resolve1"
    _RESOLVED_MANAGER_INTERFACE = "org.freedesktop.resolve1.Manager"

    def __init__(self) -> None:
        super().__init__()

    @classmethod
    def get_servers(cls) -> List[IPv4Address]:
        return list(set(cls._get_servers_from_systemd_resolved()) or cls._get_servers_from_resolver())

    @classmethod
    def _get_servers_from_systemd_resolved(cls) -> Generator[IPv4Address, None, None]:
        bus = Gio.bus_get_sync(Gio.BusType.SYSTEM)

        manager_proxy = Gio.DBusProxy.new_for_bus_sync(
            Gio.BusType.SYSTEM,
            Gio.DBusProxyFlags.DO_NOT_AUTO_START,
            None,
            cls._RESOLVED_NAME,
            "/org/freedesktop/resolve1",
            cls._RESOLVED_MANAGER_INTERFACE,
        )

        try:
            data = manager_proxy.call_sync(
                "org.freedesktop.DBus.Properties.Get",
                GLib.Variant("(ss)", (cls._RESOLVED_MANAGER_INTERFACE, "DNS")),
                Gio.DBusCallFlags.NONE,
                -1,
                None
            ).unpack()[0]
        except GLib.Error:
            return

        for (interface, address_family, address) in data:
            if address_family != socket.AF_INET.value:
                continue

            if interface != 0:
                object_path = manager_proxy.call_sync(
                    "GetLink",
                    GLib.Variant("(i)", (interface,)),
                    Gio.DBusCallFlags.NONE,
                    -1,
                    None
                ).unpack()[0]

                if bus.call_sync(
                    cls._RESOLVED_NAME,
                    object_path,
                    "org.freedesktop.DBus.Properties",
                    "Get",
                    GLib.Variant("(ss)", ("org.freedesktop.resolve1.Link", "DefaultRoute")),
                    None,
                    Gio.DBusCallFlags.NONE,
                    -1,
                    None
                ).unpack()[0]:
                    addr = ip_address('.'.join(str(p) for p in address))
                    assert isinstance(addr, IPv4Address)
                    yield addr

    @classmethod
    def _get_servers_from_resolver(cls) -> Generator[IPv4Address, None, None]:
        with open(cls.RESOLVER_PATH) as f:
            for line in f:
                match = re.search(r"^nameserver\s+((?:\d{1,3}\.){3}\d{1,3}$)", line)
                if match:
                    yield IPv4Address(match.group(1))


def get_dns_resolv_conf():
    nameservers = []
    try:
        resolv = open("/etc/resolv.conf", "r", encoding="UTF-8")
    except IOError:
        return nameservers
    with resolv:
        for line in resolv:
            tokens = line.split(None, 2)
            if len(tokens) < 2 or tokens[0] != "nameserver":
                continue
            try:
                nameservers.append(IPv4Address(tokens[1]))
            except ValueError:
                pass
    return nameservers

def get_dns_resolved():
    try:
        bus = dbus.SystemBus()
    except dbus.exceptions.DBusException as s:
        if s.get_dbus_name() == 'org.freedesktop.DBus.Error.NoReply':
            return get_dns_resolv_conf()
        raise
    try:
        resolve1 = bus.get_object('org.freedesktop.resolve1',
                                  '/org/freedesktop/resolve1')
        dns = resolve1.Get('org.freedesktop.resolve1.Manager',
                           'DNS',
                           dbus_interface='org.freedesktop.DBus.Properties')
    except dbus.exceptions.DBusException as s:
        error = s.get_dbus_name()
        if error in (
            'org.freedesktop.DBus.Error.ServiceUnknown',
            'org.freedesktop.DBus.Error.NameHasNoOwner',
            'org.freedesktop.DBus.Error.NoSuchUnit',
        ) or error.startswith('org.freedesktop.systemd1.'):
            return get_dns_resolv_conf()
        raise
    # Use global entries first
    dns.sort(key=lambda x: x[0] != 0)
    # Only keep IPv4 entries. systemd-resolved is trusted to return valid
    # addresses.
    # ToDo: We only need abridged IPv4 DNS entries for ifindex == 0.
    # to ensure static DNS of disconnected network interfaces are not added.
    return [IPv4Address(bytes(addr)) for ifindex, family, addr in dns
            if family == 2]

def install_firewall_rules(dns):
    qdb = qubesdb.QubesDB()
    qubesdb_dns = []
    for i in ('/qubes-netvm-primary-dns', '/qubes-netvm-secondary-dns'):
        ns_maybe = qdb.read(i)
        if ns_maybe is None:
            continue
        try:
            qubesdb_dns.append(IPv4Address(ns_maybe.decode("ascii", "strict")))
        except (UnicodeDecodeError, ValueError):
            pass
    preamble = [
        'add table ip qubes',
        # Add the chain so that the subsequent delete will work. If the chain already
        # exists this is a harmless no-op.
        'add chain ip qubes dnat-dns',
        # Delete the chain so that if the chain already exists, it will be removed.
        # The removal of the old chain and addition of the new one happen as a single
        # atomic operation, so there is no period where neither chain is present or
        # where both are present.
        'delete chain ip qubes dnat-dns',
    ]
    rules = [
        'table ip qubes {',
        'chain dnat-dns {',
        'type nat hook prerouting priority dstnat; policy accept;',
    ]
    if not dns:
        # User has no IPv4 DNS set in sys-net. Maybe IPv6 only environment.
        # Or maybe user wants to enforce DNS-Over-HTTPS.
        # Drop IPv4 DNS requests to qubesdb_dns addresses.
        for vm_nameserver in qubesdb_dns:
            rules += [
                f"ip daddr {vm_nameserver} udp dport 53 drop",
                f"ip daddr {vm_nameserver} tcp dport 53 drop",
            ]
    else:
        while len(qubesdb_dns) > len(dns):
            # Ensure that upstream DNS pool is larger than qubesdb_dns pool
            dns = dns + dns
        for vm_nameserver, dest in zip(qubesdb_dns, dns):
            dns_ = str(dest)
            rules += [
                f"ip daddr {vm_nameserver} udp dport 53 dnat to {dns_}",
                f"ip daddr {vm_nameserver} tcp dport 53 dnat to {dns_}",
            ]
    rules += ["}", "}"]

    # check if new rules are the same as the old ones - if so, don't reload
    # and return that info via exit code
    try:
        old_rules = subprocess.check_output(
            ["nft", "list", "chain", "ip", "qubes", "dnat-dns"]).decode().splitlines()
    except subprocess.CalledProcessError:
        old_rules = []
    old_rules = [line.strip() for line in old_rules]

    if old_rules == rules:
        sys.exit(100)

    os.execvp("nft", ("nft", "--", "\n".join(preamble + rules)))

if __name__ == '__main__':
    install_firewall_rules(DNSServersDefaultRoute.get_servers())
#    install_firewall_rules(get_dns_resolved())

Patch:

--- qubes-setup-dnat-to-ns.orig	2024-11-16 10:01:05.227860799 +0000
+++ qubes-setup-dnat-to-ns.fixed	2024-11-16 10:03:04.751864447 +0000
@@ -30,6 +30,86 @@
 from ipaddress import IPv4Address
 import os
 
+import re
+import socket
+from ipaddress import IPv4Address, ip_address
+from typing import List, Generator
+
+from gi.repository import GObject, Gio, GLib
+
+class DNSServersDefaultRoute(GObject.GObject):
+    RESOLVER_PATH = "/etc/resolv.conf"
+    _RESOLVED_NAME = "org.freedesktop.resolve1"
+    _RESOLVED_MANAGER_INTERFACE = "org.freedesktop.resolve1.Manager"
+
+    def __init__(self) -> None:
+        super().__init__()
+
+    @classmethod
+    def get_servers(cls) -> List[IPv4Address]:
+        return list(set(cls._get_servers_from_systemd_resolved()) or cls._get_servers_from_resolver())
+
+    @classmethod
+    def _get_servers_from_systemd_resolved(cls) -> Generator[IPv4Address, None, None]:
+        bus = Gio.bus_get_sync(Gio.BusType.SYSTEM)
+
+        manager_proxy = Gio.DBusProxy.new_for_bus_sync(
+            Gio.BusType.SYSTEM,
+            Gio.DBusProxyFlags.DO_NOT_AUTO_START,
+            None,
+            cls._RESOLVED_NAME,
+            "/org/freedesktop/resolve1",
+            cls._RESOLVED_MANAGER_INTERFACE,
+        )
+
+        try:
+            data = manager_proxy.call_sync(
+                "org.freedesktop.DBus.Properties.Get",
+                GLib.Variant("(ss)", (cls._RESOLVED_MANAGER_INTERFACE, "DNS")),
+                Gio.DBusCallFlags.NONE,
+                -1,
+                None
+            ).unpack()[0]
+        except GLib.Error:
+            return
+
+        for (interface, address_family, address) in data:
+            if address_family != socket.AF_INET.value:
+                continue
+
+            if interface != 0:
+                object_path = manager_proxy.call_sync(
+                    "GetLink",
+                    GLib.Variant("(i)", (interface,)),
+                    Gio.DBusCallFlags.NONE,
+                    -1,
+                    None
+                ).unpack()[0]
+
+                if bus.call_sync(
+                    cls._RESOLVED_NAME,
+                    object_path,
+                    "org.freedesktop.DBus.Properties",
+                    "Get",
+                    GLib.Variant("(ss)", ("org.freedesktop.resolve1.Link", "DefaultRoute")),
+                    None,
+                    Gio.DBusCallFlags.NONE,
+                    -1,
+                    None
+                ).unpack()[0]:
+                    addr = ip_address('.'.join(str(p) for p in address))
+                    assert isinstance(addr, IPv4Address)
+                    yield addr
+
+    @classmethod
+    def _get_servers_from_resolver(cls) -> Generator[IPv4Address, None, None]:
+        with open(cls.RESOLVER_PATH) as f:
+            for line in f:
+                match = re.search(r"^nameserver\s+((?:\d{1,3}\.){3}\d{1,3}$)", line)
+                if match:
+                    yield IPv4Address(match.group(1))
+
+
 def get_dns_resolv_conf():
     nameservers = []
     try:
@@ -105,8 +185,7 @@
         'chain dnat-dns {',
         'type nat hook prerouting priority dstnat; policy accept;',
     ]
-    dns_resolved = get_dns_resolved()
-    if not dns_resolved:
+    if not dns:
         # User has no IPv4 DNS set in sys-net. Maybe IPv6 only environment.
         # Or maybe user wants to enforce DNS-Over-HTTPS.
         # Drop IPv4 DNS requests to qubesdb_dns addresses.
@@ -116,10 +195,10 @@
                 f"ip daddr {vm_nameserver} tcp dport 53 drop",
             ]
     else:
-        while len(qubesdb_dns) > len(dns_resolved):
+        while len(qubesdb_dns) > len(dns):
             # Ensure that upstream DNS pool is larger than qubesdb_dns pool
-            dns_resolved = dns_resolved + dns_resolved
-        for vm_nameserver, dest in zip(qubesdb_dns, dns_resolved):
+            dns = dns + dns
+        for vm_nameserver, dest in zip(qubesdb_dns, dns):
             dns_ = str(dest)
             rules += [
                 f"ip daddr {vm_nameserver} udp dport 53 dnat to {dns_}",
@@ -142,4 +221,5 @@
     os.execvp("nft", ("nft", "--", "\n".join(preamble + rules)))
 
 if __name__ == '__main__':
-    install_firewall_rules(get_dns_resolved())
+    install_firewall_rules(DNSServersDefaultRoute.get_servers())
+#    install_firewall_rules(get_dns_resolved())
1 Like

I was about to do this:
/usr/lib/qubes/qubes-setup-dnat-to-ns:

...
if __name__ == '__main__':
    install_firewall_rules(get_dns_resolv_conf())
...

but your solution is more modern.

I posted an issue on GitHub about this here. You can submit your solution as pull request here, if I am correct.

1 Like

Thanks for opening the issue, I’ll check the fix some more before creating a pull request.

I’ve noticed one more issue with systemd-resolved, for some reason the DNS addresses are read in wrong order sometimes:

$ sudo nft list chain ip qubes dnat-dns
table ip qubes {
	chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		ip daddr 10.139.1.1 udp dport 53 dnat to 10.139.1.2
		ip daddr 10.139.1.1 tcp dport 53 dnat to 10.139.1.2
		ip daddr 10.139.1.2 udp dport 53 dnat to 10.139.1.1
		ip daddr 10.139.1.2 tcp dport 53 dnat to 10.139.1.1
	}
}

I’ll try to look into it first.

Observations:

  • If I start the VPN, restart systemd-resolved, 1.1.1.1 and 8.8.8.8 are part of Global.
  • If I simply start the VPN, 10.139.1.1 and 10.139.1.2 are the only entries in Global. If the dns entries are sorted like in the original code, 1.1.1.1 and 8.8.8.8 wont be in first 2 entries.

I made the following modifications to your code:
qubes-setup-dnat-to-ns:

#!/usr/bin/python3
#
# The Qubes OS Project, http://www.qubes-os.org
#
# Copyright (C) 2022  Marek Marczykowski-Górecki
#                               <marmarek@invisiblethingslab.com>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
#

from __future__ import annotations

import subprocess
import sys

import os
import re
import socket
from ipaddress import IPv4Address, ip_address
from typing import List, Generator

from gi.repository import GObject, Gio, GLib

import qubesdb

class DNSServers(GObject.GObject):
    RESOLVER_PATH = "/etc/resolv.conf"
    _RESOLVED_NAME = "org.freedesktop.resolve1"
    _RESOLVED_MANAGER_INTERFACE = "org.freedesktop.resolve1.Manager"

    def __init__(self) -> None:
        super().__init__()

    @classmethod
    def get_servers(cls) -> List[IPv4Address]:
        return list(cls._get_servers_from_systemd_resolved() or cls._get_servers_from_resolver())

    @classmethod
    def _get_servers_from_systemd_resolved(cls) -> Generator[IPv4Address, None, None]:
        try:
            bus = Gio.bus_get_sync(Gio.BusType.SYSTEM)
        except GLib.Error as e:
            if 'org.freedesktop.DBus.Error.NoReply' in e.message:
                return
            raise e

        try:
            manager_proxy = Gio.DBusProxy.new_for_bus_sync(
                Gio.BusType.SYSTEM,
                Gio.DBusProxyFlags.DO_NOT_AUTO_START,
                None,
                cls._RESOLVED_NAME,
                "/org/freedesktop/resolve1",
                cls._RESOLVED_MANAGER_INTERFACE,
            )
            data = manager_proxy.call_sync(
                "org.freedesktop.DBus.Properties.Get",
                GLib.Variant("(ss)", (cls._RESOLVED_MANAGER_INTERFACE, "DNS")),
                Gio.DBusCallFlags.NONE,
                -1,
                None
            ).unpack()[0]
        except GLib.Error as e:
            dbus_patterns = [
                'org.freedesktop.DBus.Error.ServiceUnknown',
                'org.freedesktop.DBus.Error.NameHasNoOwner',
                'org.freedesktop.DBus.Error.NoSuchUnit',
                'org.freedesktop.systemd1.',]
            for dbus_pattern in dbus_patterns:
                if dbus_pattern in e.message:
                    return
            raise e

        def get_is_not_global(dns_tuple):
            return dns_tuple[0] != 0

        def get_is_not_defaultroute(dns_tuple):
            try:
                object_path = manager_proxy.call_sync(
                    "GetLink",
                    GLib.Variant("(i)", (dns_tuple[0],)),
                    Gio.DBusCallFlags.NONE,
                    -1,
                    None
                ).unpack()[0]
            except GLib.GError as e:
                if 'org.freedesktop.DBus.Error.InvalidArgs' in e.message and dns_tuple[0] == 0:
                    return True
                raise e
            return not bus.call_sync(
                cls._RESOLVED_NAME,
                object_path,
                "org.freedesktop.DBus.Properties",
                "Get",
                GLib.Variant("(ss)", ("org.freedesktop.resolve1.Link", "DefaultRoute")),
                None,
                Gio.DBusCallFlags.NONE,
                -1,
                None
            ).unpack()[0]

        # Place DefaultRoute first, Global second
        data.sort(key=get_is_not_global) 
        data.sort(key=get_is_not_defaultroute)

        # Only keep IPv4 entries. systemd-resolved is trusted to return valid
        # addresses.
        # ToDo: We only need abridged IPv4 DNS entries for ifindex == 0.
        # to ensure static DNS of disconnected network interfaces are not added.
        for (interface, address_family, address) in data:
            if address_family == socket.AF_INET.value:
                addr = ip_address('.'.join(str(p) for p in address))
                assert isinstance(addr, IPv4Address)
                yield addr

    @classmethod
    def _get_servers_from_resolver(cls) -> Generator[IPv4Address, None, None]:
        try: 
            resolv = open(cls.RESOLVER_PATH)
        except IOError:
            return
        with resolv:
            for line in resolv:
                match = re.search(r"^nameserver\s+((?:\d{1,3}\.){3}\d{1,3}$)", line)
                if match:
                    yield IPv4Address(match.group(1))

def install_firewall_rules(dns):
    qdb = qubesdb.QubesDB()
    qubesdb_dns = []
    for i in ('/qubes-netvm-primary-dns', '/qubes-netvm-secondary-dns'):
        ns_maybe = qdb.read(i)
        if ns_maybe is None:
            continue
        try:
            qubesdb_dns.append(IPv4Address(ns_maybe.decode("ascii", "strict")))
        except (UnicodeDecodeError, ValueError):
            pass
    preamble = [
        'add table ip qubes',
        # Add the chain so that the subsequent delete will work. If the chain already
        # exists this is a harmless no-op.
        'add chain ip qubes dnat-dns',
        # Delete the chain so that if the chain already exists, it will be removed.
        # The removal of the old chain and addition of the new one happen as a single
        # atomic operation, so there is no period where neither chain is present or
        # where both are present.
        'delete chain ip qubes dnat-dns',
    ]
    rules = [
        'table ip qubes {',
        'chain dnat-dns {',
        'type nat hook prerouting priority dstnat; policy accept;',
    ]
    if not dns:
        # User has no IPv4 DNS set in sys-net. Maybe IPv6 only environment.
        # Or maybe user wants to enforce DNS-Over-HTTPS.
        # Drop IPv4 DNS requests to qubesdb_dns addresses.
        for vm_nameserver in qubesdb_dns:
            rules += [
                f"ip daddr {vm_nameserver} udp dport 53 drop",
                f"ip daddr {vm_nameserver} tcp dport 53 drop",
            ]
    else:
        while len(qubesdb_dns) > len(dns):
            # Ensure that upstream DNS pool is larger than qubesdb_dns pool
            dns = dns + dns
        for vm_nameserver, dest in zip(qubesdb_dns, dns):
            dns_ = str(dest)
            rules += [
                f"ip daddr {vm_nameserver} udp dport 53 dnat to {dns_}",
                f"ip daddr {vm_nameserver} tcp dport 53 dnat to {dns_}",
            ]
    rules += ["}", "}"]

    # check if new rules are the same as the old ones - if so, don't reload
    # and return that info via exit code
    try:
        old_rules = subprocess.check_output(
            ["nft", "list", "chain", "ip", "qubes", "dnat-dns"]).decode().splitlines()
    except subprocess.CalledProcessError:
        old_rules = []
    old_rules = [line.strip() for line in old_rules]

    if old_rules == rules:
        sys.exit(100)

    os.execvp("nft", ("nft", "--", "\n".join(preamble + rules)))

if __name__ == '__main__':
    install_firewall_rules(DNSServers.get_servers())

Patch:

--- qubes-setup-dnat-to-ns.apparatus	2024-11-17 04:05:41.954525914 -0500
+++ qubes-setup-dnat-to-ns.user11	2024-11-17 05:37:30.191662494 -0500
@@ -24,12 +24,7 @@
 import subprocess
 import sys
 
-import dbus
-import qubesdb
-from typing import List
-from ipaddress import IPv4Address
 import os
-
 import re
 import socket
 from ipaddress import IPv4Address, ip_address
@@ -37,7 +32,9 @@
 
 from gi.repository import GObject, Gio, GLib
 
-class DNSServersDefaultRoute(GObject.GObject):
+import qubesdb
+
+class DNSServers(GObject.GObject):
     RESOLVER_PATH = "/etc/resolv.conf"
     _RESOLVED_NAME = "org.freedesktop.resolve1"
     _RESOLVED_MANAGER_INTERFACE = "org.freedesktop.resolve1.Manager"
@@ -47,22 +44,26 @@
 
     @classmethod
     def get_servers(cls) -> List[IPv4Address]:
-        return list(set(cls._get_servers_from_systemd_resolved()) or cls._get_servers_from_resolver())
+        return list(cls._get_servers_from_systemd_resolved() or cls._get_servers_from_resolver())
 
     @classmethod
     def _get_servers_from_systemd_resolved(cls) -> Generator[IPv4Address, None, None]:
-        bus = Gio.bus_get_sync(Gio.BusType.SYSTEM)
-
-        manager_proxy = Gio.DBusProxy.new_for_bus_sync(
-            Gio.BusType.SYSTEM,
-            Gio.DBusProxyFlags.DO_NOT_AUTO_START,
-            None,
-            cls._RESOLVED_NAME,
-            "/org/freedesktop/resolve1",
-            cls._RESOLVED_MANAGER_INTERFACE,
-        )
+        try:
+            bus = Gio.bus_get_sync(Gio.BusType.SYSTEM)
+        except GLib.Error as e:
+            if 'org.freedesktop.DBus.Error.NoReply' in e.message:
+                return
+            raise e
 
         try:
+            manager_proxy = Gio.DBusProxy.new_for_bus_sync(
+                Gio.BusType.SYSTEM,
+                Gio.DBusProxyFlags.DO_NOT_AUTO_START,
+                None,
+                cls._RESOLVED_NAME,
+                "/org/freedesktop/resolve1",
+                cls._RESOLVED_MANAGER_INTERFACE,
+            )
             data = manager_proxy.call_sync(
                 "org.freedesktop.DBus.Properties.Get",
                 GLib.Variant("(ss)", (cls._RESOLVED_MANAGER_INTERFACE, "DNS")),
@@ -70,94 +71,71 @@
                 -1,
                 None
             ).unpack()[0]
-        except GLib.Error:
-            return
+        except GLib.Error as e:
+            dbus_patterns = [
+                'org.freedesktop.DBus.Error.ServiceUnknown',
+                'org.freedesktop.DBus.Error.NameHasNoOwner',
+                'org.freedesktop.DBus.Error.NoSuchUnit',
+                'org.freedesktop.systemd1.',]
+            for dbus_pattern in dbus_patterns:
+                if dbus_pattern in e.message:
+                    return
+            raise e
 
-        for (interface, address_family, address) in data:
-            if address_family != socket.AF_INET.value:
-                continue
+        def get_is_not_global(dns_tuple):
+            return dns_tuple[0] != 0
 
-            if interface != 0:
+        def get_is_not_defaultroute(dns_tuple):
+            try:
                 object_path = manager_proxy.call_sync(
                     "GetLink",
-                    GLib.Variant("(i)", (interface,)),
+                    GLib.Variant("(i)", (dns_tuple[0],)),
                     Gio.DBusCallFlags.NONE,
                     -1,
                     None
                 ).unpack()[0]
+            except GLib.GError as e:
+                if 'org.freedesktop.DBus.Error.InvalidArgs' in e.message and dns_tuple[0] == 0:
+                    return True
+                raise e
+            return not bus.call_sync(
+                cls._RESOLVED_NAME,
+                object_path,
+                "org.freedesktop.DBus.Properties",
+                "Get",
+                GLib.Variant("(ss)", ("org.freedesktop.resolve1.Link", "DefaultRoute")),
+                None,
+                Gio.DBusCallFlags.NONE,
+                -1,
+                None
+            ).unpack()[0]
 
-                if bus.call_sync(
-                    cls._RESOLVED_NAME,
-                    object_path,
-                    "org.freedesktop.DBus.Properties",
-                    "Get",
-                    GLib.Variant("(ss)", ("org.freedesktop.resolve1.Link", "DefaultRoute")),
-                    None,
-                    Gio.DBusCallFlags.NONE,
-                    -1,
-                    None
-                ).unpack()[0]:
-                    addr = ip_address('.'.join(str(p) for p in address))
-                    assert isinstance(addr, IPv4Address)
-                    yield addr
+        # Place DefaultRoute first, Global second
+        data.sort(key=get_is_not_global) 
+        data.sort(key=get_is_not_defaultroute)
+
+        # Only keep IPv4 entries. systemd-resolved is trusted to return valid
+        # addresses.
+        # ToDo: We only need abridged IPv4 DNS entries for ifindex == 0.
+        # to ensure static DNS of disconnected network interfaces are not added.
+        for (interface, address_family, address) in data:
+            if address_family == socket.AF_INET.value:
+                addr = ip_address('.'.join(str(p) for p in address))
+                assert isinstance(addr, IPv4Address)
+                yield addr
 
     @classmethod
     def _get_servers_from_resolver(cls) -> Generator[IPv4Address, None, None]:
-        with open(cls.RESOLVER_PATH) as f:
-            for line in f:
+        try: 
+            resolv = open(cls.RESOLVER_PATH)
+        except IOError:
+            return
+        with resolv:
+            for line in resolv:
                 match = re.search(r"^nameserver\s+((?:\d{1,3}\.){3}\d{1,3}$)", line)
                 if match:
                     yield IPv4Address(match.group(1))
 
-
-def get_dns_resolv_conf():
-    nameservers = []
-    try:
-        resolv = open("/etc/resolv.conf", "r", encoding="UTF-8")
-    except IOError:
-        return nameservers
-    with resolv:
-        for line in resolv:
-            tokens = line.split(None, 2)
-            if len(tokens) < 2 or tokens[0] != "nameserver":
-                continue
-            try:
-                nameservers.append(IPv4Address(tokens[1]))
-            except ValueError:
-                pass
-    return nameservers
-
-def get_dns_resolved():
-    try:
-        bus = dbus.SystemBus()
-    except dbus.exceptions.DBusException as s:
-        if s.get_dbus_name() == 'org.freedesktop.DBus.Error.NoReply':
-            return get_dns_resolv_conf()
-        raise
-    try:
-        resolve1 = bus.get_object('org.freedesktop.resolve1',
-                                  '/org/freedesktop/resolve1')
-        dns = resolve1.Get('org.freedesktop.resolve1.Manager',
-                           'DNS',
-                           dbus_interface='org.freedesktop.DBus.Properties')
-    except dbus.exceptions.DBusException as s:
-        error = s.get_dbus_name()
-        if error in (
-            'org.freedesktop.DBus.Error.ServiceUnknown',
-            'org.freedesktop.DBus.Error.NameHasNoOwner',
-            'org.freedesktop.DBus.Error.NoSuchUnit',
-        ) or error.startswith('org.freedesktop.systemd1.'):
-            return get_dns_resolv_conf()
-        raise
-    # Use global entries first
-    dns.sort(key=lambda x: x[0] != 0)
-    # Only keep IPv4 entries. systemd-resolved is trusted to return valid
-    # addresses.
-    # ToDo: We only need abridged IPv4 DNS entries for ifindex == 0.
-    # to ensure static DNS of disconnected network interfaces are not added.
-    return [IPv4Address(bytes(addr)) for ifindex, family, addr in dns
-            if family == 2]
-
 def install_firewall_rules(dns):
     qdb = qubesdb.QubesDB()
     qubesdb_dns = []
@@ -221,5 +199,4 @@
     os.execvp("nft", ("nft", "--", "\n".join(preamble + rules)))
 
 if __name__ == '__main__':
-    install_firewall_rules(DNSServersDefaultRoute.get_servers())
-#    install_firewall_rules(get_dns_resolved())
+    install_firewall_rules(DNSServers.get_servers())
  • Instead of _get_servers_from_systemd_resolved() returning only DefaultRoute DNS, I made it return all DNS including duplicates, but DefaultRoute ones are placed first, Global ones second. This should solve the ordering problem when the VPN is off.
  • I added the error handlings and the comments from the original functions to their respective methods in the class.

This will cause a problem if the DefaultRoute has only one DNS server.
You’ll get these dnat-dns rules:

$ sudo nft list chain ip qubes dnat-dns
table ip qubes {
	chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		ip daddr 10.139.1.1 udp dport 53 dnat to x.x.x.x
		ip daddr 10.139.1.1 tcp dport 53 dnat to x.x.x.x
		ip daddr 10.139.1.2 udp dport 53 dnat to 10.139.1.1
		ip daddr 10.139.1.2 tcp dport 53 dnat to 10.139.1.1
	}
}

Instead of:

$ sudo nft list chain ip qubes dnat-dns
table ip qubes {
	chain dnat-dns {
		type nat hook prerouting priority dstnat; policy accept;
		ip daddr 10.139.1.1 udp dport 53 dnat to x.x.x.x
		ip daddr 10.139.1.1 tcp dport 53 dnat to x.x.x.x
		ip daddr 10.139.1.2 udp dport 53 dnat to x.x.x.x
		ip daddr 10.139.1.2 tcp dport 53 dnat to x.x.x.x
	}
}

Maybe duplicate the DNS server in the list so it’ll have the same DNS server provided by DefaultRoute in first two entries.
And in case no DNS servers are provided by DefaultRoute then maybe just let the DNS requests bypass VPN and go to the Global DNS servers (e.g. Qubes OS Virtual DNS servers).

Also, @marmarek comment about this issue on matrix channel:

see also discussion at Fixing issue 9011: DNS leakage when only one DNS server is set by alimirjamali · Pull Request #505 · QubesOS/qubes-core-agent-linux · GitHub - there were some issues with getting global value, but maybe looking for just DefaultRoute would be better, but then we need to ensure it doesn’t break if somebody fills /etc/resolv.conf independently (systemd-resolved handles this case, but I think it doesn’t attach such DNS servers to any link then)

I’ll try to look into it to find a way to know if somebody filled /etc/resolv.conf independently and if so then use the DNS servers from Global instead of the DefaultRoute DNS servers.

Isn’t that supposed to be handled by this:

...
def install_firewall_rules(dns):
...
        while len(qubesdb_dns) > len(dns):
            # Ensure that upstream DNS pool is larger than qubesdb_dns pool
            dns = dns + dns
        for vm_nameserver, dest in zip(qubesdb_dns, dns):
...

?

That’s only if the dns list contains a single DNS entry.
Since you’ve:

Then the dns list will contain not only the single DNS server from DefaultRoute, but the Global ones as well so this code will be skipped.