Sys-apt-cacher-ng always fails on specific qubes

User Support General
1

Since a few days, sys-apt-cacher-ng fails on specific qubes.
I use apt-cacher-ng together with privoxy because I use onionized repositories.

debian-13-minimal always fails (another template that is based on debian-13-minimal works fine):

Updating debian-13-minimal
Refreshing package info
Ign:1 http://HTTPS///deb.debian.org/debian trixie InRelease
Ign:2 http://HTTPS///deb.debian.org/debian-security trixie-security InRelease
Ign:3 http://HTTPS///deb.qubes-os.org/r4.2/vm trixie InRelease
Ign:1 http://HTTPS///deb.debian.org/debian trixie InRelease
Ign:2 http://HTTPS///deb.debian.org/debian-security trixie-security InRelease
Ign:3 http://HTTPS///deb.qubes-os.org/r4.2/vm trixie InRelease
Ign:1 http://HTTPS///deb.debian.org/debian trixie InRelease
Ign:2 http://HTTPS///deb.debian.org/debian-security trixie-security InRelease
Ign:3 http://HTTPS///deb.qubes-os.org/r4.2/vm trixie InRelease
Err:1 http://HTTPS///deb.debian.org/debian trixie InRelease
  500  Unable to connect [IP: 127.0.0.1 8082]
Err:2 http://HTTPS///deb.debian.org/debian-security trixie-security InRelease
  500  Unable to connect [IP: 127.0.0.1 8082]
Err:3 http://HTTPS///deb.qubes-os.org/r4.2/vm trixie InRelease
  500  Unable to connect [IP: 127.0.0.1 8082]
Reading package lists...
E: Failed to fetch http://HTTPS///deb.debian.org/debian/dists/trixie/InRelease  500  Unable to connect [IP: 127.0.0.1 8082]
E: Failed to fetch http://HTTPS///deb.debian.org/debian-security/dists/trixie-security/InRelease  500  Unable to connect [IP: 127.0.0.1 8082]
E: Failed to fetch http://HTTPS///deb.qubes-os.org/r4.2/vm/dists/trixie/InRelease  500  Unable to connect [IP: 127.0.0.1 8082]
E: Some index files failed to download. They have been ignored, or old ones used instead.
Ign:1 http://HTTPS///deb.debian.org/debian trixie InRelease
Ign:2 http://HTTPS///deb.debian.org/debian-security trixie-security InRelease
Ign:3 http://HTTPS///deb.qubes-os.org/r4.2/vm trixie InRelease
Ign:1 http://HTTPS///deb.debian.org/debian trixie InRelease
Ign:2 http://HTTPS///deb.debian.org/debian-security trixie-security InRelease
Ign:3 http://HTTPS///deb.qubes-os.org/r4.2/vm trixie InRelease
Ign:1 http://HTTPS///deb.debian.org/debian trixie InRelease
Ign:2 http://HTTPS///deb.debian.org/debian-security trixie-security InRelease
Ign:3 http://HTTPS///deb.qubes-os.org/r4.2/vm trixie InRelease
Err:1 http://HTTPS///deb.debian.org/debian trixie InRelease
  500  Unable to connect [IP: 127.0.0.1 8082]
Err:2 http://HTTPS///deb.debian.org/debian-security trixie-security InRelease
  500  Unable to connect [IP: 127.0.0.1 8082]
Err:3 http://HTTPS///deb.qubes-os.org/r4.2/vm trixie InRelease
  500  Unable to connect [IP: 127.0.0.1 8082]
Reading package lists...
E: Failed to fetch http://HTTPS///deb.debian.org/debian/dists/trixie/InRelease  500  Unable to connect [IP: 127.0.0.1 8082]
E: Failed to fetch http://HTTPS///deb.debian.org/debian-security/dists/trixie-security/InRelease  500  Unable to connect [IP: 127.0.0.1 8082]
E: Failed to fetch http://HTTPS///deb.qubes-os.org/r4.2/vm/dists/trixie/InRelease  500  Unable to connect [IP: 127.0.0.1 8082]
E: Some index files failed to download. They have been ignored, or old ones used instead.

Two debian-12-minimal based templates also always fail (debian-12-minimal itself not):

Updating tpl-signal
Refreshing package info
Hit:1 tor+http://deb.qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion/r4.2/vm bookworm InRelease
Ign:2 http://HTTPS///updates.signal.org/desktop/apt xenial InRelease
Hit:3 tor+http://5ajw6aqf3ep7sijnscdzw77t7xq4xjpsy335yb2wiwgouo7yfxtjlmid.onion bookworm-security InRelease
Hit:4 tor+http://2s4yqjx5ul6okpp3f2gaunr2syex5jgbfpfvhxxbbjwnrsvbk5v3qbid.onion/debian bookworm InRelease
Ign:2 http://HTTPS///updates.signal.org/desktop/apt xenial InRelease
Get:5 tor+http://2s4yqjx5ul6okpp3f2gaunr2syex5jgbfpfvhxxbbjwnrsvbk5v3qbid.onion/debian bookworm-backports InRelease [59.4 kB]
Ign:2 http://HTTPS///updates.signal.org/desktop/apt xenial InRelease
Err:2 http://HTTPS///updates.signal.org/desktop/apt xenial InRelease
  500  Unable to connect [IP: 127.0.0.1 8082]
Fetched 59.4 kB in 10s (5890 B/s)
Reading package lists...
E: Failed to fetch http://HTTPS///updates.signal.org/desktop/apt/dists/xenial/InRelease  500  Unable to connect [IP: 127.0.0.1 8082]
E: Some index files failed to download. They have been ignored, or old ones used instead.
Ign:1 http://HTTPS///updates.signal.org/desktop/apt xenial InRelease
Hit:2 tor+http://5ajw6aqf3ep7sijnscdzw77t7xq4xjpsy335yb2wiwgouo7yfxtjlmid.onion bookworm-security InRelease
Ign:1 http://HTTPS///updates.signal.org/desktop/apt xenial InRelease
Hit:3 tor+http://deb.qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion/r4.2/vm bookworm InRelease
Hit:4 tor+http://2s4yqjx5ul6okpp3f2gaunr2syex5jgbfpfvhxxbbjwnrsvbk5v3qbid.onion/debian bookworm InRelease
Ign:1 http://HTTPS///updates.signal.org/desktop/apt xenial InRelease
Hit:5 tor+http://2s4yqjx5ul6okpp3f2gaunr2syex5jgbfpfvhxxbbjwnrsvbk5v3qbid.onion/debian bookworm-backports InRelease
Err:1 http://HTTPS///updates.signal.org/desktop/apt xenial InRelease
  500  Unable to connect [IP: 127.0.0.1 8082]
Reading package lists...
E: Failed to fetch http://HTTPS///updates.signal.org/desktop/apt/dists/xenial/InRelease  500  Unable to connect [IP: 127.0.0.1 8082]
E: Some index files failed to download. They have been ignored, or old ones used instead.

i have two fedora-41-minimal based vms and they also fail:

Updating tpl-default-mgmt-dvm
Refreshing package info
Errors during downloading metadata for repository 'fedora':
  - Status code: 500 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=fedora-41&arch=x86_64&protocol=http (IP: 127.0.0.1)
Failed to download metadata for repo 'fedora': Cannot prepare internal mirrorlist: Status code: 500 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=fedora-41&arch=x86_64&protocol=http (IP: 127.0.0.1)
Error checking if metalink "http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f41&arch=x86_64&protocol=http" is in sync for repository "updates"
 Librepo error: Cannot prepare internal mirrorlist: Status code: 500 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f41&arch=x86_64&protocol=http (IP: 127.0.0.1)

All my other qubes are based on debian-12-minimal and they don’t fail.
The qubes I mentioned always fail, no matter if I update all Qubes at once or each independently.

apt-cacher.err contains only this:

Fri May 23 17:11:52 2025|Binding localhost:/run/apt-cacher-ng/socket
Fri May 23 17:11:52 2025|Binding on host: (null), port: 8082
Fri May 23 17:11:52 2025|Binding 0.0.0.0:8082
Fri May 23 17:11:52 2025|Binding [::]:8082
Fri May 23 17:11:53 2025|Binding localhost:/run/apt-cacher-ng/socket
Fri May 23 17:11:53 2025|Binding on host: (null), port: 8082
Fri May 23 17:11:53 2025|Binding 0.0.0.0:8082
Fri May 23 17:11:53 2025|Binding [::]:8082
Fri May 23 17:14:39 2025|Binding localhost:/run/apt-cacher-ng/socket
Fri May 23 17:14:39 2025|Binding on host: (null), port: 8082
Fri May 23 17:14:39 2025|Binding 0.0.0.0:8082
Fri May 23 17:14:39 2025|Binding [::]:8082
Fri May 23 17:15:15 2025|Binding localhost:/run/apt-cacher-ng/socket
Fri May 23 17:15:15 2025|Binding on host: (null), port: 8082
Fri May 23 17:15:15 2025|Binding 0.0.0.0:8082
Fri May 23 17:15:15 2025|Binding [::]:8082
2

Can you compare the contents of /etc/apt/sources between these templates
and report back.

Again, compare the contents of the repo definitions and report back.

It might be helpful if you could increase the debug level
(/etc/apt-cacher-ng/acng.conf) -set Debug level to 7
and try to update one of them.

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

3

I compared the repo definitions but found nothing interesting. I even reinstalled debian-13-minimal but the error still occured.

I already set the Debug level to the max level.

I regret doing this now (because I would be very interested in why this error occured out of nowhere?) but I deleted sys-apt-cacher-ng out of frustration and set it up again with my saltstack states (so its 1:1 the same) and now it works again.

The only error that still occurs and that I get quite often is when updating debian-13-minimal, I got it even after re-creating my sys-apt-cacher-ng, but as of writing this post I couldn’t replicate it again; I will create another post when it occurs again.