3isec-qubes-cacher-1.10-1 breaks Debian/Fedora TemplateVM updates

User Support
, ,
1

I installed the cacher package according to the steps outlined in https://qubes.3isec.org/tasks.html on my Qubes setup running 4.1.2

The installation of 3isec-qubes-cacher-1.10-1.fc32.x86_64.rpm via dom0 went well, but afterward, none of my Debian or Fedora TemplateVMs could update via apt/dnf update.

Debian 12

E: Failed to fetch http://HTTPS///deb.debian.org/debian/dists/bookworm/InRelease Connection failed [IP: 127.0.0.1 8082]
E: Failed to fetch http://HTTPS///deb.debian.org/debian-security/dists/bookworm-security/InRelease Connection failed [IP: 127.0.0.1 8082]
E: Failed to fetch http://HTTPS///deb.qubes-os.org/r4.1/vm/dists/bookworm/InRelease Connection failed [IP: 127.0.0.1 8082]
E: Some index files failed to download. They have been ignored, or old ones used instead.

Fedora 38

Errors during downloading metadata for repository ā€˜fedoraā€™:
Curl error (52): Server returned nothing (no headers, no data) for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=fedora-38&arch=x86_64&protocol=http [Empty reply from server]
Error: Failed to download metadata for repo ā€˜fedoraā€™: Cannot prepare internal mirrorlist: Curl error (52): Server returned nothing (no headers, no data) for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=fedora-38&arch=x86_64&protocol=http [Empty reply from server]

My cacher AppVM is connected to the same netVM which is successfully providing network to other AppVMs. However, when I try to ping out from the cacher AppVM I get

Ping: connect: Network is unreachable

ip link indicates eth0 is down. I brought it up via ip link set eth0 up but still canā€™t ping out from the cacher AppVM.

What am I missing here?

2

Usually when I see messages like that in a VM, I find that I have the VM set up to try to use the proxy, but the UpdateProxy policy is pointing to sys-firewall not to sys-cacher. (or vice versa, if it seems to be having trouble finding https://mirrorsā€¦ without HTTPS in it.)

In other words, you may need to fix your policy to match what your VM is trying to do (thatā€™s easier than fixing the VM to conform to the policy).

Iā€™ll assume the former case here since it would be consistent with what you report.

It works a bit differently in 4.2 versus 4.1. In 4.2 you want to look at the file /etc/qubes/policy.d/50-config-updates.policy and there should be a line at the end that contains @type:TemplateVM and ends with target=sys-cacher. If it says something like target=sys-firewall, you may be able to fix your issue by editing that line to read sys-cacher.

In 4.1 the line will appear in a different file (grep for UpdatesProxy); Iā€™m not sure which one because my 4.1 install has been tinkered with a lot.

On the other hand your sys-cacher doesnā€™t seem to have network connectivity, so this probably isnā€™t the problem, or at least itā€™s not the ONLY problem.

3

On 4.1, I havenā€™t encountered this problem.
It seems to me that you need to address the networking problem first.
Why is the cacher offline when you have a netvm set? This looks
seriously wrong.
Check that you have qubes networking set up with apt list installed qubes-core-agent-networking

systemctl status apt-cacher-ng will tell you if the service is running,
but until you resolve the network problem thatā€™s no help.

4

Thanks @unman.

It looks like the template-cacher TemplateVM was created based on my Global default template (fedora-38-minimal) and therefore did not install apt-cacher-ng (obviously) or qubes-core-agent-networking (less obvious).

Even after manually installing those packages, my Fedora TemplateVMs error out when updating over clearnet (not TOR or VPN) connections.

Errors during downloading metadata for repository ā€˜updatesā€™:
Status code: 403 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f38&arch=x86_64&protocol=http (IP: 127.0.0.1)
Error: Failed to download metadata for repo ā€˜updatesā€™: Cannot prepare internal mirrorlist: Status code: 403 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f38&arch=x86_64&protocol=http (IP: 127.0.0.1)

I then tried to uninstall 3isec-qubes-cacher-1.10-1 from dom0 via sudo dnf remove 3isec-qubes-cacher-1.10-1.fc32.x86_64 to try to reinstall it with debian-11 set as the Global default template. The PRERUN scriptlet appeared to run, but also fail:

error: %preun(3isec-qubes-cacher-1.10-1.fc32.x86_64) scriptlet failed, exit status 20
Error in PREUN scriptlet in rpm package 3isec-qubes-cacher
Verifying : 3isec-qubes-cacher-1.10-1.fc32.x86_64 1/1
Failed:
3isec-qubes-cacher-1.10-1.fc32.x86_64

Is the 403 error still a known issue? And if so, how do we ā€œclear the cacher before trying another updateā€?

How should cacher be uninstalled? I found this unanswered forum post while trying to troubleshoot my unsuccessful uninstall and reinstall of cacher.

5

Update: I was able to uninstall 3isec-qubes-cacher from dom0 by running:

sudo dnf remove 3isec-qubes-cacher-1.10-1
sudo rpm --noscripts -e 3isec-qubes-cacher

And then reinstalled it after setting my Global template to debian-12 and running the following from dom0:

sudo dnf install ./3isec-qubes-cacher-1.10-1.fc32.x86_64.rpm -y

The reinstallation threw these errors:

whonix-ws-16: ERROR (exit code 20, details in /var/log/qubes/mgmt-whonix-ws-16.log)
whonix-gw-16: ERROR (exit code 20, details in /var/log/qubes/mgmt-whonix-gw-16.log)
warning: %post(3isec-qubes-cacher-1.10-1.fc32.x86_64) scriptlet failed, exit status 20
Error in POSTIN scriptlet in rpm package 3isec-qubes-cacher
Verifying : 3isec-qubes-cacher-1.10-1.fc32.x86_64 1/1
Installed:
3isec-qubes-cacher-1.10-1.fc32.x86_64

I noticed that the template-cacher TemplateVMā€™s sudo systemctl status apt-cacher-ng indicates apt-cacher-ng is installed as a masked service (expected), but qubes-core-agent and qubes-core-agent-networking were not installed (unexpected).

I noticed that the cacher AppVM doesnā€™t seem to have the /etc/apt-cacher-ng/ directory or the apt-cacher-ng service as per systemctl status apt-cacher-ngā€™s output:

Unit apt-cacher-ng.service could not be found.

I verified cacher has network connectivity via ping. However, now neither debian nor fedora TemplateVMs can update through cacher.

debian-12

E: Failed to fetch http://HTTPS///deb.debian.org/debian/dists/bookworm/InRelease Connection failed [IP: 127.0.0.1 8082]
E: Failed to fetch http://HTTPS///deb.debian.org/debian-security/dists/bookworm-security/InRelease Connection failed [IP: 127.0.0.1 8082]
E: Failed to fetch http://HTTPS///deb.qubes-os.org/r4.1/vm/dists/bookworm/InRelease Connection failed [IP: 127.0.0.1 8082]
E: Some index files failed to download. They have been ignored, or old ones used instead.

fedora-38

Fedora 38 - x86_64 0.0 B/s | 0 B 00:01
Errors during downloading metadata for repository ā€˜fedoraā€™:
Curl error (1): Unsupported protocol for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=fedora-38&arch=x86_64&protocol=http [Received HTTP/0.9 when not allowed]
Error: Failed to download metadata for repo ā€˜fedoraā€™: Cannot prepare internal mirrorlist: Curl error (1): Unsupported protocol for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=fedora-38&arch=x86_64&protocol=http [Received HTTP/0.9 when not allowed]

It is still not clear to me how these cacher VMs are being built (in general, and why they are missing some of these packages in particular) or how this package should be uninstalled and reinstalled.

6

I dont see how this could happen - the clone.sls should ensure that a
debian template is installed, and that the template used by cacher is
cloned from debian-X-minimal.
But apparently it has happened. (Feel the need for a baffled emoji here)

How do you remove the package? Usually I would just remove the package
with dnf remove 3isec-qubes-cacher, but as your instal is completely
borked do it manually.

First roll back any changes made to the templates - thereā€™s a salt state at
/srv/salt/cacher/restore_templates.sls that should do this:
sudo qubesctl --skip-dom0 --show-output --targets=templates state.apply cacher.restore_templates

Then edit /etc/qubes/policy.d/30-user.policy and remove any reference
to qubes.UpdatesProxy .

Finally:

sudo dnf remove 3isec-qubes-cacher`
sudo rpm --noscripts -e 3isec-qubes-cacher

should clean everything up on the rpm side.

Check that /srv/salt/cacher is deleted.

7

I attempted to remove the package from dom0 via

sudo dnf remove 3isec-qubes-cacher-1.10-1

But it errored out toward the end:

error: %preun(3isec-qubes-cacher-1.10-1.fc32.x86_64) scriptlet failed, exit status 20
Error in PREUN scriptlet in rpm package 3isec-qubes-cacher
Verifying : 3isec-qubes-cacher-1.10-1.fc32.x86_64 1/1
Failed:
3isec-qubes-cacher-1.10-1.fc32.x86_64
Error: Transaction failed

Thatā€™s why/how I came across having to also run the following from dom0:

sudo rpm --noscripts -e 3isec-qubes-cacher

I reverted the changes to dom0ā€™s /etc/qubes/policy.d/30-user.policy and deleted /srv/salt/cacher via

sudo rm -r /srv/salt/cacher

This command did not return any output (I did not run this command before):

sudo qubesctl --skip-dom0 --show-output --targets=templates state.apply cacher.restore_templates

Re-installing cacher via dom0 (sudo dnf install ./3isec-qubes-cacher-1.10-1.fc32.x86_64.rpm) results in this error (I donā€™t recall any errors the first time I installed this):

warning: %post(3isec-qubes-cacher-1.10-1.fc32.x86_64) scriptlet failed, exit status 20
Error in POSTIN scriptlet in rpm package 3isec-qubes-cacher
Verifying : 3isec-qubes-cacher-1.10-1.fc32.x86_64 1/1
Installed:
3isec-qubes-cacher-1.10-1.fc32.x86_64

Interestingly, this time I was able to update debian-12, but fedora-38 continues to error out with the 403 error as before:

Errors during downloading metadata for repository ā€˜updatesā€™:
Status code: 403 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f38&arch=x86_64&protocol=http (IP: 127.0.0.1)
Error: Failed to download metadata for repo ā€˜updatesā€™: Cannot prepare internal mirrorlist: Status code: 403 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f38&arch=x86_64&protocol=http (IP: 127.0.0.1)

Also, I noticed this time the cacher AppVM shows the apt-cacher-ng service and has more files in its /etc/apt-cacher-ng/ directory. Not sure if that qubesctl command did something different.

8

UPDATE: It seems like thereā€™s some sort of conflict between apt-cacher-ng 3.6.4-1 and tinyproxy 1.10.0-5 on Debian 11 where the apt-cacher-ng service canā€™t bind to the 8082 socket.

  • After upgrading to Qubes 4.2.1 and installing the 3isec-qubes-cacher.x86_64 v1.16-1.fc37 package, which created a debian-11 TemplateVM running apt-cacher-ng 3.6.4-1 and tinyproxy 1.10.0-5, I found that my Debian TemplateVMs throw the following error:

    E: Failed to fetch http://HTTPS///deb.debian.org/debian/dists/bookworm/InRelease 500 Unable to connect [IP: 127.0.0.1 8082]

  • I inspected cacherā€™s /rw/config/rc.local and noticed a useless

    • iptables command (because /sbin/iptables didnā€™t exist) and a similarly-useless
    • nft command (because systemctl status nftables indicated the nftables service wasnā€™t running)
      • I temporarily enabled the nftables service on cacher by enabling it on template-cacher, but Debian TemplateVMs continue to throw the same error
      • I inspected the apt-cacher-ng service on cacher via systemctl status apt-cacher-ng and saw

        Couldnā€™t bind socket: Address already in use
        Port 8082 is busy, see the manual (Troubleshooting chapter) for details.

        • Troubleshooting indicated we could ferret out the conflicting process via fuser -4 -v -n tcp 8082 which revealed there were 3 instances of tinyproxy running
          • I uninstalled tinyproxy from cacher and restarted the apt-cacher-ng service
          • Now both the cacher AppVM and my debian TemplateVMs throw DNS errors
  • A related curiosity: What in the fedora update process in particular uses JSON files?
9

Thereā€™s something bizarre here.

  1. cacher v1.16-1 clones a debian-12-minimal template, not a debian-11 template
  2. The package does not install any iptables command in /rw/config/rc.local
  3. The nft command in /rw/config/rc.local runs as expected to open port 8082 to incoming traffic, for network connected qubes.
  4. There is no need to enable the nftables service - in fact you should not do this.
  5. tinyproxy is masked to avoid any conflict with apt-cacher-ng

None of the issues you report match 3isec-qubes-cacher.x86_64 v1.16-1.

You dont say what sort of upgrade you performed - did you upgrade in
place and try to use an existing cacher qube? And then try to install
v1.16-1 over an existing installation? That;s the best I can come up
with.

I cant account for your networking problem.

I never presume to speak for the Qubes team. When I comment in the Forum I speak for myself.
10

Thanks for getting back to me @unman.

I had performed an in-place upgrade of Qubes 4.1.2 to 4.2.
I uninstalled cacher during Stage 4 of the in-place upgrade by running this from Dom0

sudo dnf remove 3isec-qubes-cacher

After the in-place upgrade completed (and after I updated /etc/yum.repos.d/3isec-dom0.repoā€™s baseurl with fc37) I re-installed cacher by running this from Dom0

sudo qubes-dom0-update 3isec-qubes-cacher

I just now uninstalled and re-installed cacher again, using these command in Dom0

sudo dnf remove 3isec-qubes-cacher
sudo rpm --noscripts -e 3isec-qubes-cacher
sudo qubes-dom0-update 3isec-qubes-cacher

Observed my

  • Debian-12-minimal TemplateVMā€™s apt update continues to error out:

    E: Failed to fetch http://HTTPS///deb.debian.org/debian/dists/bookworm/InRelease Connection failed [IP: 127.0.0.1 8082]

  • template-cacher TemplateVMā€™s qvm-features indicates it is now based on debian-12-minimal

  • cacherā€™s nftables service is disabled (as per systemctl status nftables)

  • cacherā€™s tinyproxy service could not be found (as per systemctl status tinyproxy; sounds like this is expected because of the masking)

  • cacherā€™s apt-cacher-ng service could not be found as per systemctl status apt-cacher-ng OR apt list --installed|grep apt-cacher

    • I also found that I could not run apt update from cacher (https://ā€¦) or template-cacher (http://HTTPS///)ā€“both throw the same error as above after ignoring all sources or

      E: Failed to fetch https://deb.qubes-os.org/r4.2/vm/dists/bookworm/InRelease Reading from proxy failed - select (115: Operation now in progress) [IP: 127.0.0.1 8082]

      • I cloned template-cacher to see if toggling the source proxy prefix (https:// vs. http://HTTPS///) would make a difference

        sed -i 's^http://HTTPS///^https://^' /etc/apt/sources.list
        sed -i 's^http://HTTPS///^https://^' /etc/apt/sources.list.d/*.list

        • It did not; it ignored all of the URLs and errored out as before.

Interestingly, before I uninstalled and re-installed cacher tonight, I could update my TemplateVMs if I prefixed my sources with https://. The cacher AppVM would boot, but apt-cacher-ng wouldnā€™t log any activity. Now I canā€™t update my TemplateVMs at all. I confirmed Dom0ā€™s /etc/qubes/policy.d/30-user.policy and /etc/qubes/policy.d/50-config-updates.policy files had the expected configurations.

11

This is your problem. My guess is that you had a broken proxy, so
that when you were trying to install in to template-cacher, the install
failed.
The fact that using ā€œhttpsā€ worked shows that you were not using
apt-cacher-ng.

Hereā€™s what you could do.

  1. Remove any reference to qubes.UpdatesProxy from /etc/qubes/policy.d/30-user.policy
  2. Change the entry in /etc/qubes/policy.d/50-config-updates.policy from cacher to sys-net
  3. sudo dnf remove 3isec-qubes-cacher
  4. sudo qubes-dom0-update 3isec-qubes-cacher

Itā€™s important that your caching proxy has network access. If the
package install failed then it will not, because it is based on a minimal
template which does not have qubes-core-agent-networking installed.
Similarly, if you have set the default netvm to none, cacher will not
have network access, and you will need to assign a netvm.

Things to check:

  1. The install in 4 completes without error.
  2. cacher has network access.
  3. The apt-cacher-ng service is running in cacher
  4. /etc/qubes/policy.d/50-config-updates.policy contains reference to cacher
  5. The repository definitions in a sample template do not contain https:// entries

When you update in a template you will see an entry in cacher
under /var/log/apt-cacher-ng/apt-cacher-ng.log - once the system
is installed you rarely need to attend to this.

I never presume to speak for the Qubes team. When I comment in the Forum I speak for myself.
12

Thank you for your guidance @unman

I followed the steps and apt-cacher-ng got installed in template-cacher, however I now get this error when I try to update my debian 12 TemplateVMs:

E: Failed to fetch http://HTTPS///deb.debian.org/debian/dists/bookworm/InRelease 503 Resource temporarily unavailable [IP: 127.0.0.1 8082]

systemctl status apt-cacher-ng on template-cacher seems to indicate it is ok now:

Jul 08 10:25:23 cacher systemd[1]: Starting apt-cacher-ng.service - Apt-Cacher NG software download proxyā€¦
Jul 08 10:25:23 cacher systemd[1]: Started apt-cacher-ng.service - Apt-Cacher NG software download proxyā€¦

To your point, the Dom0 sudo qubes-dom0-update 3isec-qubes-cacher command throws this error every time (I included the line before and after for context and bolded the error):

template-cacher: OK
whonix-gateway-17: ERROR (exit code 20, details in /var/log/qubes/mgmt-whonix-gateway-17.log)
whonix-workstation-17: ERROR (exit code 20, details in /var/log/qubes/mgmt-whonix-workstation-17.log)
warning: %post(3isec-qubes-cacher-1.16-1.fc37.x86_64) scriptlet failed, exit status 20
Error in POSTIN scriptlet in rpm package 3isec-qubes-cacher
Verifying : 3isec-qubes-cacher-1.16-1.fc37.x86_64 1/1
Installed:
3isec-qubes-cacher-1.16-1.fc37.x86_64

How can I view the POSTIN scriptlet? Iā€™d like to try to run it line-by-line to see whatā€™s breaking.

Thank you.

13

Iā€™m not a Whonix user.
What does it say in the log files that are cited?

You can see the source here - the
scripts are in cacher.spec, but these ERRORS come in the section that
attempts to update the repository definitions, using
/srv/salt/cacher/change_templates.sls - thereā€™s a test in that file
that prevented changes in Whonix templates by testing for nodename of
host - has that changed?

You could run this with:
sudo qubesctl --skip-dom0 --show-output --targets=whonix-workstation-17 state.apply cacher.change_templates

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

14

Neither of the cited logs (/var/log/qubes/mgmt-whonix-gateway-17.log or var/log/qubes/mgmt-whonix-workstation-17.log) contained any errors, Result: False, or exit code 20.

However, /var/log/qubes/mgmt-cacher.log DOES have a Result: False

ID: /rw/config/qubes-bind-dirs.d/50_user.conf
Function: file.managed
Result: False
Comment: Unable to manage file: none of the specified sources were found
Started: 21:54:12.347846
Duration: 5.814 ms
Changes:
Summary for cacher
Succeeded: 2 (changed=2)
Failed: 1
Total states run: 3
Total run time: 44.849 ms
exit code: 20

While researching these errors, I found forum threads like these, which seemed to capture the general sentiment: PSA: apt-cacher-ng is a buggy pile of shit and this nearly 12-year old thread which makes me wonder whether there are bigger problems here.

My Dom0ā€™s /srv/salt/cacher/change_templates.sls still has the test line

{% if grains['nodename']|lower != 'host' %}

Running sudo qubesctl --skip-dom0 --show-output --targets=whonix-workstation-17 state.apply cacher.change_templates from Dom0 doesnā€™t return any problems:

whonix-workstation-17:
Summary for whonix-workstation-17
Succeeded: 0
Failed: 0
Total states run: 0
Total run time: 0.000 ms

What should I try next?

XKCD "Automation"

1 Like
15

Yet you reported that there was an error.

How did you not see this error on the original install?
You should not need to poke in GitHub - the file is present in
/srv/salt/cacher - if it isnt you have more serious undiagnosed
issues.
What template are you using for default-mgmt-dvm ?

Are you able to update any template?
What does the cacher qube have as netvm?

There are thousands of people using apt-cacher-ng, and many Qubes users
running it without issue, but it is your choice.

This is not a test line - itā€™s a marker to exclude whonix templates
which AFAIK are the only templates where the nodename is host

16

Yet you reported that there was an error.

  • Yes. I also thought that was weird. The output from the installation and uninstallation processes indicated whonix-gateway-17 errored out with an exit code 20, but the referenced logs in Dom0 didnā€™t reflect that. Whereas, the /var/log/qubes/mgmt-cacher.log DOES reflect an exit code 20 as it relates to /rw/config/qubes-bind-dirs.d/50_user.conf

How did you not see this error on the original install?

  • Honest mistake. It didnā€™t stand out to me and I didnā€™t think to scroll through all of the output. This was my fault.

the file is present in /srv/salt/cacher - if it isnt you have more serious undiagnosed issues.

  • There is a copy of that file there, however, I noticed it was missing the last two lines from the version on GitHub

    binds+=( '/etc/apt-cacher-ng/blackarch_mirror-list' )
    binds+=( '/etc/apt-cacher-ng/Qubes_mirrors' )

What template are you using for default-mgmt-dvm ?

  • debian-12. I re-ran the uninstallation and reinstallation with fedora-39 set as the default-mgmt-dvm but ran into the same issues, except this time, I did catch an error in /var/log/qubes/mgmt-whonix-workstation-17.log

    /usr/lib/python3.12/site-packages/salt/utils/jid.py:19: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    return datetime.datetime.utcnow()
    exit code: 20

  • Also, interestingly, thereā€™s no default-mgmt-dvm in Qube Manager, even when itā€™s running (via qvm-run -u root default-mgmt-dvm xterm &). The only way I could find details about it was to view its settings in the Template Manager. Should it appear in Qube Manager? I found another of your forum posts and ran sudo qubesctl state.apply qvm.default-mgmt-dvm, which completed successfully, but didnā€™t change anything in Qube Manager.

Are you able to update any template?

  • No

What does the cacher qube have as netvm?

  • sys-net

This is not a test line - itā€™s a marker to exclude whonix templates

  • My mistake. I couldnā€™t get the grains command to run or test whether Whonix uses host as its node name. Let me know if thereā€™s something else I should be looking for in my Dom0ā€™s /srv/salt/cacher/change_templates.sls file.
17

We seem to be travelling in circles.

Some of the issues arise because you are using debian-12 as template
for default-mgmt-dvm. This is a known bug for Salt, which has been
covered in the Forum before. If you use a Debian base, then files will
not be copied to the target. You should use a Fedora template based
default-mgmt-dvm.

The entry in the Whonix log is not an error - it is (as stated) a
warning.

default-mgmt-dvm is an internal qube - to see it in Qube Manager, you
must select View->ā€œShow Internal Qubesā€

There is no importance in the difference between the cacher files in
GitHub and those you have in /srv/salt/cacher - it reflects the fact
that the latest files on GitHub havent yet been tested and packaged.

Letā€™s try some troubleshooting in a root terminal in cacher qube:

  1. systemctl status apt-cacher-ng should show apt-cacher-ng is running.
  2. ss -ntpl should show apt-cacher-ng listening on 0.0.0.0:8082
  3. The cacher qube must have network access - you can test this as
    follows:
cd
sed -i s^http://HTTPS///^https://^ /etc/apt/sources.list
apt update
apt install wget
wget www.qubes-os.org

The wget command should show successful resolution, connection and
saving of the Qubes home page.

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

18

@unman,

Thank you for your patience.

Thank you for pointing out the known issues with Debian as the default-mgmt-dvm AND that it is an internal qubeā€“I wasnā€™t aware of this. Unfortunately, the issues persisted even after I set it to use a fedora-39 base.

Your comments about the Whonix log and the delta between the 50_user.conf files are also noted.

The apt-cacher-ng servie was running as expected.
Apt-cacher-ng was listening on 0.0.0.0:8082 as expected
The apt update failed thusly

Ign:1 Index of /debian bookworm InRelease
Ign:2 https://deb.debian.org/debian-security bookworm-security InRelease
Ign:3 Index of /r4.2/vm/ bookworm InRelease
Ign:4 Index of /r4.2/vm/ bookworm-testing InRelease
Ign:1 Index of /debian bookworm InRelease
Ign:3 Index of /r4.2/vm/ bookworm InRelease
Ign:4 Index of /r4.2/vm/ bookworm-testing InRelease
Ign:2 https://deb.debian.org/debian-security bookworm-security InRelease
Ign:1 Index of /debian bookworm InRelease
Ign:2 https://deb.debian.org/debian-security bookworm-security InRelease
Ign:3 Index of /r4.2/vm/ bookworm InRelease
Ign:4 Index of /r4.2/vm/ bookworm-testing InRelease
Err:1 Index of /debian bookworm InRelease
Temporary failure resolving ā€˜deb.debian.orgā€™
Err:2 https://deb.debian.org/debian-security bookworm-security InRelease
Temporary failure resolving ā€˜deb.debian.orgā€™
Err:3 Index of /r4.2/vm/ bookworm InRelease
Temporary failure resolving ā€˜deb.qubes-os.orgā€™
Err:4 Index of /r4.2/vm/ bookworm-testing InRelease
Temporary failure resolving ā€˜deb.qubes-os.orgā€™
Reading package listsā€¦ Done
E: Failed to fetch https://deb.debian.org/debian/dists/bookworm/InRelease Temporary failure resolving ā€˜deb.debian.orgā€™
E: Failed to fetch https://deb.debian.org/debian-security/dists/bookworm-security/InRelease Temporary failure resolving ā€˜deb.debian.orgā€™
E: Failed to fetch https://deb.qubes-os.org/r4.2/vm/dists/bookworm/InRelease Temporary failure resolving ā€˜deb.qubes-os.orgā€™
E: Failed to fetch https://deb.qubes-os.org/r4.2/vm/dists/bookworm-testing/InRelease Temporary failure resolving ā€˜deb.qubes-os.orgā€™
E: Some index files failed to download. They have been ignored, or old ones used instead.

Needless to say I couldnā€™t install wget and didnā€™t have curl. Running ping www.qubes-os.org failed thusly

ping: www.qubes-os.org: Temporary failure in name resolution

Other AppVMs (sys-net and personal) are able to ping www.qubes-os.org and run apt update. Only the TemplateVMs cannot.

19

I wanted you to run these commands in the cacher qube

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

20

I should have been clearer. I did run those tests from the cacher AppVM. I ran the same tests from other AppVMs as a way to verify other other AppVMs could connect, whereas the cacher AppVM could not. This was poor phrasing on my part.