Installing using the iso from http://qubes.notset.fr/iso/Qubes-4.2.202304291601-x86_64.iso, I’m having trouble upgrading whonix-gw and whonix-ws. The error messages are like “500 unable to reach tinyproxy 127.0.0.1:8082”. Reinstalling qubes-template-whonix-gw-16-4.1.0-202303181802.noarch.rpm didn’t solve this. So I think there’s probably something wrong with this template.
Restoring a whonix-gw-16 from R4.1 solved the above problem. However, the backup tool didn’t make necessary changes to /etc/apt/sources.list.d/qubes-r4.list. For other templates, the shifting of R4.1 repos to R4.2 repos happened automatically. But it’s not the case for whonix. I managed to manually editing qubes-r4.list and copying the keyring file to make apt happy.
Other templates work just fine. That’s why I think it’s specific to sys-whonix’s template ( whonix-gw-16 ).
In fact, after clean installing R4.2, I deleted the auto-created sys-whonix, and restored another sys-whonix from a R4.1 backup file. To rule out the possibility of the mismatch of something between “old” sys-whonix and 'new" whonix-gw-16, I tried fresh creating sys-whonix based on 20230318 whonix-gw-16 template (sudo qubesctl state.sls qvm.sys-whonix). But it didn’t help, either.
It’s possible that the first issue was introduced by myself. I’d like to wait and see if there will be more reports.
Really interesing. After doing what I described above, I can upgrade whonix templates through sys-whonix, but I have no connection in anon-whonix or whonix-16-dvm. System check said that eth0 is not up, and I restarted network service, but it didn’t help.
So I updated dom0 to testing latest, removed whonix gw and ws and sys-whonix, and used qvm-template to reinstall those two templates. I also ran sudo qubesctl state.sls qvm.sys-whonix. Now I fell into the same pit a second time. I have internet connection in anon-whonix and whonix-16-dvm, but I cannot upgrade the whonix templates.
Ign:1 tor+https://fasttrack.debian.net/debian bullseye-fasttrack InRelease
Ign:2 tor+https://deb.whonix.org bullseye InRelease
Ign:3 https://deb.qubes-os.org/r4.2/vm bullseye InRelease
Ign:4 tor+https://deb.debian.org/debian bullseye InRelease
Ign:5 tor+https://deb.kicksecure.com bullseye InRelease
Ign:1 tor+https://fasttrack.debian.net/debian bullseye-fasttrack InRelease
Ign:2 tor+https://deb.whonix.org bullseye InRelease
Ign:3 https://deb.qubes-os.org/r4.2/vm bullseye InRelease
Ign:6 tor+https://deb.debian.org/debian bullseye-updates InRelease
Ign:5 tor+https://deb.kicksecure.com bullseye InRelease
Ign:1 tor+https://fasttrack.debian.net/debian bullseye-fasttrack InRelease
Ign:2 tor+https://deb.whonix.org bullseye InRelease
Ign:3 https://deb.qubes-os.org/r4.2/vm bullseye InRelease
Ign:7 tor+https://deb.debian.org/debian-security bullseye-security InRelease
Ign:5 tor+https://deb.kicksecure.com bullseye InRelease
Err:1 tor+https://fasttrack.debian.net/debian bullseye-fasttrack InRelease
Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
Err:2 tor+https://deb.whonix.org bullseye InRelease
Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
Err:3 https://deb.qubes-os.org/r4.2/vm bullseye InRelease
Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
Ign:8 tor+https://deb.debian.org/debian bullseye-backports InRelease
Err:5 tor+https://deb.kicksecure.com bullseye InRelease
Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
Ign:4 tor+https://deb.debian.org/debian bullseye InRelease
Ign:6 tor+https://deb.debian.org/debian bullseye-updates InRelease
Ign:7 tor+https://deb.debian.org/debian-security bullseye-security InRelease
Ign:8 tor+https://deb.debian.org/debian bullseye-backports InRelease
Ign:4 tor+https://deb.debian.org/debian bullseye InRelease
Ign:6 tor+https://deb.debian.org/debian bullseye-updates InRelease
Ign:7 tor+https://deb.debian.org/debian-security bullseye-security InRelease
Ign:8 tor+https://deb.debian.org/debian bullseye-backports InRelease
Err:4 tor+https://deb.debian.org/debian bullseye InRelease
Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
Err:6 tor+https://deb.debian.org/debian bullseye-updates InRelease
Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
Err:7 tor+https://deb.debian.org/debian-security bullseye-security InRelease
Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
Err:8 tor+https://deb.debian.org/debian bullseye-backports InRelease
Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
Reading package lists... Done
E: Failed to fetch tor+https://deb.debian.org/debian/dists/bullseye/InRelease Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
E: Failed to fetch tor+https://deb.debian.org/debian/dists/bullseye-updates/InRelease Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
E: Failed to fetch tor+https://deb.debian.org/debian-security/dists/bullseye-security/InRelease Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
E: Failed to fetch tor+https://deb.debian.org/debian/dists/bullseye-backports/InRelease Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
E: Failed to fetch tor+https://fasttrack.debian.net/debian/dists/bullseye-fasttrack/InRelease Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
E: Failed to fetch tor+https://deb.kicksecure.com/dists/bullseye/InRelease Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
E: Failed to fetch tor+https://deb.whonix.org/dists/bullseye/InRelease Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
E: Failed to fetch https://deb.qubes-os.org/r4.2/vm/dists/bullseye/InRelease Invalid response from proxy: HTTP/1.0 500 Unable to connect Server: tinyproxy/1.10.0 Content-Type: text/html Connection: close [IP: 127.0.0.1 8082]
E: Some index files failed to download. They have been ignored, or old ones used instead.
And the journal says:
May 02 14:56:01 host sudo[1245]: user : TTY=pts/0 ; PWD=/home/user ; USER=root ; COMMAND=/usr/bin/apt update
May 02 14:56:01 host sudo[1245]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1000)
May 02 14:56:01 host systemd[1]: Started Forward connection to updates proxy over Qubes RPC (127.0.0.1:43396).
May 02 14:56:01 host systemd[1]: Started Forward connection to updates proxy over Qubes RPC (127.0.0.1:43408).
May 02 14:56:01 host systemd[1]: Started Forward connection to updates proxy over Qubes RPC (127.0.0.1:43418).
May 02 14:56:01 host systemd[1]: Started Forward connection to updates proxy over Qubes RPC (127.0.0.1:43420).
May 02 14:56:01 host systemd[1]: Started Forward connection to updates proxy over Qubes RPC (127.0.0.1:43434).
May 02 14:56:02 host systemd[1]: qubes-updates-proxy-forwarder@46-127.0.0.1:8082-127.0.0.1:43396.service: Succeeded.
May 02 14:56:02 host systemd[1]: Started Forward connection to updates proxy over Qubes RPC (127.0.0.1:43438).
May 02 14:56:02 host systemd[1]: qubes-updates-proxy-forwarder@47-127.0.0.1:8082-127.0.0.1:43408.service: Succeeded.
May 02 14:56:02 host systemd[1]: Started Forward connection to updates proxy over Qubes RPC (127.0.0.1:43450).
May 02 14:56:02 host systemd[1]: qubes-updates-proxy-forwarder@49-127.0.0.1:8082-127.0.0.1:43420.service: Succeeded.
May 02 14:56:02 host systemd[1]: Started Forward connection to updates proxy over Qubes RPC (127.0.0.1:43454).
May 02 14:56:03 host systemd[1]: qubes-updates-proxy-forwarder@48-127.0.0.1:8082-127.0.0.1:43418.service: Succeeded.
May 02 14:56:03 host systemd[1]: qubes-updates-proxy-forwarder@50-127.0.0.1:8082-127.0.0.1:43434.service: Succeeded.
......
Dom0 journal says:
dom0 qrexec-policy-daemon[2901]: qrexec: qubes.UpdatesProxy+: whonix-ws-16 -> @default: allowed to sys-whonix
qubes-updates-proxy.service is up and running, but there are tinyproxy error logs in the journal:
May 02 15:34:55 host tinyproxy[19756]: Proxying refused on filtered domain "127.0.0.1"
May 02 15:35:01 host tinyproxy[19757]: opensock: Could not establish a connection to fasttrack.debian.net
May 02 15:35:01 host tinyproxy[19756]: opensock: Could not establish a connection to deb.debian.org
May 02 15:35:01 host tinyproxy[19757]: opensock: Could not establish a connection to deb.whonix.org
May 02 15:35:01 host tinyproxy[19756]: opensock: Could not establish a connection to deb.qubes-os.org
May 02 15:35:01 host tinyproxy[19756]: Error reading readable client_fd 9
May 02 15:35:01 host tinyproxy[19756]: Could not retrieve request entity
May 02 15:35:02 host tinyproxy[19757]: opensock: Could not establish a connection to deb.kicksecure.com
May 02 15:35:02 host tinyproxy[19757]: Error reading readable client_fd 9
May 02 15:35:02 host tinyproxy[19757]: Could not retrieve request entity
@fepitre I think I succeeded in finding the most relevant package.
In whonix-gw-16, if I install xen-utils-guest 4.14.5-20+deb11u1 from R4.1 repo and restart sys-whonix, the uplink service will succeed, thus update will work; but xendriverdomain.service will fail, resulting those AppVMs that have sys-whonix as their netvm cannot find their route to the internet.
Logs from such an AppVM:
host sdwdate[827]: __ Tor Bootstrap Result: Tor's Control Port could not be reached.Did you start Gateway beforehand? Please run systemcheck on Gateway.
...
host systemd-socket-proxyd[1335]: Failed to connect to remote host: No route to host
...
Logs from sys-whonix:
● xendriverdomain.service - Xen driver domain device daemon
Loaded: loaded (/etc/systemd/system/xendriverdomain.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2023-05-03 08:05:54 UTC; 6s ago
Process: 9522 ExecStart=/usr/sbin/xl devd (code=exited, status=203/EXEC)
CPU: 410us
May 03 08:05:54 host systemd[9522]: xendriverdomain.service: Failed to locate executable /usr/sbin/xl: No such file or directory
May 03 08:05:54 host systemd[9522]: xendriverdomain.service: Failed at step EXEC spawning /usr/sbin/xl: No such file or directory
May 03 08:05:54 host systemd[1]: Starting Xen driver domain device daemon...
May 03 08:05:54 host systemd[1]: xendriverdomain.service: Control process exited, code=exited, status=203/EXEC
May 03 08:05:54 host systemd[1]: xendriverdomain.service: Failed with result 'exit-code'.
May 03 08:05:54 host systemd[1]: Failed to start Xen driver domain device daemon.
In whonix-gw-16, if I install xen-utils-guest 1.1-1+deb11u1 from R4.2 repo and restart sys-whonix, the uplink service will fail, thus update won’t work; but xendriverdomain.service will succeed, so I can get internet access through sys-whonix.
Update: the above experiment about xen-utils-guest was performed on an whonix-gw-16 directly restored from R4.1, and I didn’t update it since restoration. Switching repos and updating that template didn’t update xen-utils-guest, because the version number of the older package is greater. This is however unimportant as I believe there will be another whonix release to coordinate with R4.2.
What’s more, updating that restored template introduces such an inability to perform updates, which is a regression. If necessary, I can provide with the list of packages that get upgraded.
I’ve noticed that there are rather recent R4.2 openqa runs that perform system check on all whonix qubes, and their states were all fine. I’m not sure why a clean reinstall of whonix-gw-16 can’t solve the problem for me.
May 03 06:59:13 host setup-ip[792]: Error: IPv6 is disabled on nexthop device.
May 03 06:59:13 host setup-ip[798]: Error: IPv6 is disabled on nexthop device.
It’s numb of me that I neglected these lines on those “seemingly good” situations.
The truth is that I enabled ipv6 on my sys-net, which means almost all of my networking qubes are ipv6-enabled.
qvm-features sys-whonix ipv6 '' fixed this for me.
But this also revealed a issue that whonix-gw-16 is having difficulties handling ipv6.
Import of key(s) didn’t help, wrong key(s)?
Public key for qubes-template-whonix-gw-16-4.1.0-202303181802.noarch.rpm is not installed. Failing package is: qubes
-template-whonix-gw-16-4.1.0-202303181802.noarch
GPG Keys are configured as: file:///tmp/qubes-installer/qubes-release/RPM-GPG-KEY-qubes-4.2-templates-community
still persist, i think this problem is happen since 1-2 months ago?