Hi,
My Dom0 update proxy is sys-whonix. Dom0 and template updates fly.
However, Qubes Template Manager’s refresh takes a very long time. Sometimes even 30 minutes to show anything at all.
How can I diagnose/fix this?
I’m uncertain what your issue is.
I know that broadband here is the best in the world, and I dont use
Whonix, but my refresh over Tor is in seconds.
The fact that your updates through Whonix “fly” suggests there’s nothing
intrinsically wrong. I would start by setting the updatevm to sys-firewall
- that should tell you if the issue is with Template-Manager or with Whonix.
Also compare CLI qvm-template and GUI manager.
I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.
Thanks for your feedback @unman.
I prefer not to deanonymize the connection through the sys-firewall route.
The CLI version is just as slow:
user@dom0:~ > time qvm-template list
Installed Templates
debian-12-minimal 0:4.2.0-202308031621 qubes-templates-itl
...
real 6m51.054s
user 0m0.158s
sys 0m0.066s
This was actually pretty fast (compared to the regular). During that time there was no other network activity.
What I notice is that there is just zero network traffic while I am waiting for a refresh. Then suddenly at some point in time some traffic shows up and the list updates instantly.
What URLs does qvm-template connect to?
You can see the repos with qvm-template repolist, and the source URLs
in the files in /etc/qubes/repo-templates
I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.
Thanks.
I really don’t understand what is going on:
In a Whonix workstation VM:
user@host:/tmp/download > time curl https://yum.qubes-os.org/r4.2/templates-itl/repodata/repomd.xml.metalink -fo /dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2659 100 2659 0 0 2654 0 0:00:01 0:00:01 --:--:-- 2656
real 0m1.090s
user 0m0.059s
sys 0m0.044s
Yet, I still get minutes for qvm-template list.
Any ideas?
The .metalink file is a list of mirrors, look inside of it.
It’s possible to comment out the metalink = line in qubes-templates.repo and uncomment a baseurl = line (or define a new one) to use a specific server.
2 Likes
This has been an issue for a while with sys-whonix, but more generally with Tor.
When you open “Onion Circuits” in sys-whonix, you will likely see an IP address get stuck on “Sentconnect” as it moves from circuit to circuit until it finds a working exit node.
As @rustybird suggested, you can comment out the metalink part and switch to the clearnet or onion baseurl in /etc/qubes/repo-templates/qubes-templates.repo.
1 Like
Switched to the onion baseurls.
Result:
user@dom0:~ > time qvm-template list
...
real 0m7.025s
user 0m0.129s
sys 0m0.058s
Thank you!
I guess this begs the obvious questions: is there any security risk to commenting and uncommenting, and if not, then why is this the default if so many users clearly suffer as a result?
If you use baseurl to select the .onion mirror (or another specific mirror) it could occasionally be out of sync for a while. With metalink you’ll sooner or later get an up-to-date server from the rotation. You could select the non-onion master server using baseurl, but it might not be sustainable to have everyone use that.
BTW maybe it’s not Tor causing this problem with the mirror rotation but rather using a Whonix Gateway (e.g. sys-whonix) directly. I have a Fedora-based named DisposableVM configured as the system updatevm, with sys-whonix as its netvm, and I never experienced the slowdown on qvm-template list. Just now I tried with updatevm switched to sys-whonix, and indeed there it is… Edit: And it’s gone again :\
1 Like
I’m sure you are right.
This is an issue that should be reported at Whonix Forums or their bug
tracker.
I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.
1 Like
I’m sure you are right.
I am not so sure. I have always used sys-whonix as a gateway and continue to do so. After switching as @rustybird explained, the result is that it works very fast. If it was a Whonix issue, it would still be slow, no?
1 Like
That’s still consistent with it being a hypothetical Whonix or Debian bug affecting the initial .metalink download, or certain mirrors, or HTTPS.
But I’m also not sure and I haven’t really tried to reproduce it.
1 Like
Hi,
I’ve been following this discussion and since the discussion concluded this is likely a Whonix-specific issue, I’ve taken the initiative to report it on the Whonix Forum to get their input.
Here’s the Whonix Forum post for reference: Slow Template List Loading in Qubes Template Manager with Whonix Gateway
I wasn’t part of this discussion but have been actively observing it because I’ve experienced the same slow loading since I started using Qubes with Whonix. At first, I assumed it was just “expected behavior” for Qubes-Whonix setups, but this thread made me realize it’s a fixable issue, and I’m really eager to see it resolved.
If anyone has additional insights, please chime in! Thanks for raising this issue.
1 Like
How to debug:
To investigate this further, it’s required to look what’s happening on the command line.
dom0 command qvm-template list must result in some action being in inside UpdateVM (sys-whonix).
So before running qvm-template list.
ps aux > a
(Write processes to file a)
After running qvm-template list.
ps aux > b
Then use File Comparison Tools to compare files a and b.
Through that, I found out, the command that command is running:
/usr/bin/dnf-3.anondist-orig repoquery --releasever=4.2 -y --setopt=reposdir=/tmp/user/1000/tmp.pOvpwWJjmj --quiet --setopt=pluginpath=/usr/lib/qubes/dnf-plugins --qf="%{name}|%{epoch}|%{version}|%{release}|%{repoid}|%{downloadsize}|%{buildtime}|%{license}|%{url}|%{summary}|%{description}|" "*"
dnf-3 --version shows DNF version 4 something.
Fedora dnf --version shows DNF version 5 something.
That might make a difference.
Analysis:
This indicates that there is nothing that Whonix can do about this and that this is based on a dnf version difference. This most likely cannot be translated into “here’s a bug in Whonix’s source code - and here’s how to fix it”.
(This might get better once Debian trixie gets blessed stable and Whonix gets ported to that Debian version.)
The method used by @rustybird is the workaround I would recommend.
For me:
time qvm-template list
...
real 1m30
This might indeed be slower depending on geographical location, ISP and Tor network censorship.
If you’re into that sort of analysis, you might like this:
2 Likes