Cacher (Apt-cacher-ng): issues with fedora updates

User Support
21

You should be able to run it within any qube. I use dispVMs to access cacher’s webui.

22

no, it doesn’t work…
except from cacher I have a “vanilla” setup and the only way to access that page is from localhost of cacher…

23

Hi there,
yesterday new “locking” on updates.
So I did same procedure: remove the metalink old repo index (“updates” was the culprit) and relaunch of updates.
All working now, but I think I found a new problem:

  • in one of my fedora templates there are rpmfusion-* repo, that I cannot find in present metalinks…

Is there a limitation in cacher that skip all non standard repo?
Thanks.
M.

24

There isn’t such a limitation.
I’m not clear what you mean: can you post an example of part of the repo
definition from /etc/yum.repos.d - just the lines for metalink= will do

25

Not helpful.

You need to have a qube with netvm set to cacher, and then open browser
in that qube to http://:8082

26

Hi there,
I’ve created a clone of tetmplate and installed firefox onto it.
I’ve just expired / cleaned all cache and relaunched update.
Still fails.
Following logs.
PLease let me know if You need anything else.
Thanks,
M.

27

Hi there.
Following generated log.

I’ve modded repo in this way:

  • in rpmfusion-free.repo and rpmfusion-nonfree.repo
    I’ve removed the http forwarder added by cacher, to see if there’s any differences with other ones
    (You can see in next repo list that is there a backup of original ones)

This is the complete list of repo in /etc/yum.repos.d of my fedora modded template:
adobe-linux-x86_64.repo
fedora-cisco-openh264.repo
fedora-updates-testing.repo
fedora-updates.repo
fedora.repo
google-chrome.repo
qubes-r4.repo
rpmfusion-free-updates-testing.repo
rpmfusion-free-updates.repo
rpmfusion-free.repo
rpmfusion-free.repo.bck
rpmfusion-nonfree-updates-testing.repo
rpmfusion-nonfree-updates.repo
rpmfusion-nonfree.repo
rpmfusion-nonfree.repo.bck

Then that’s the update log session.
Thanks,
M.

dnf_update.log (3.3 KB)

28

I access it over qrexec without a problem.

29

I can tell you that this isnt going to work.
That means that dnf will try to access the repository using https.
When that request gets to cacher, cacher has no idea what it means
because it is encrypted.
as there is no special treatment for https traffic, the request will fail

Can you please give me the line I asked for before?
The metalink data in the repo definitions.

Also, are you updating over Tor?

30

Hi unman.
I’m sorry: I tought it was a better evidence…
Yes I’m updating by sys-whonix.
And from previous log:

“modded”:
RPM Fusion for Fedora 36 - Free 0.0 B/s | 0 B 00:02
Errors during downloading metadata for repository ‘rpmfusion-free’:

“unmodded”:
RPM Fusion for Fedora 36 - Free - Updates 582 B/s | 512 B 00:00
Errors during downloading metadata for repository ‘rpmfusion-free-updates’:

If You need internal ones too:
for first one

  • metalink=HTTPS://mirrors.rpmfusion.org/metalink?repo=free-fedora-$releasever&arch=$basearch&protocol=http

for second one:

  • metalink=http://HTTPS///mirrors.rpmfusion.org/metalink?repo=free-fedora-updates-released-$releasever&arch=$basearch&protocol=http

I hope it’s everything You need…
Otherwise, just ask.
Thanks,
M.

31

TLDR - problem with caching fedora updates repository may be fixed by
setting in acng.conf:
VfilePatternEx: .*fedora.*updateinfo.*xml.zck$
DontCache: .*fedora.*updates.*updateinfo.xml.zck

This post covers problems using cacher with fedora updates are reported, e.g by
@weyoun.six:

* At least one of the zchunk checksums doesn???t match in
http://mirror.init7.net/fedora/fedora/linux/updates/36/Everything/x86_64/repodata/73c60d6746840b97edeca71d4a9b31
3aaf88e648ead74abbf7d23fe3747a002c-primary.xml.zck
* At least one of the zchunk checksums doesn???t match in
http://mirror.init7.net/fedora/fedora/linux/updates/36/Everything/x86_64/repodata/afdb89258121bb4cd97ead414e06ed
3b561e270aab025cfccc0d5166960d2bd5-filelists.xml.zck

I believe that this affects the updates repository because of the effect
of mirroring. If you hit the right mirrors at the right time, then the
cached data matches, and the update will continue.

apt-cacher-ng categorises data between static data, that should not
change, and volatile data. These are set using configuration entities,
PFilePattern and VFilePattern.
You can use the acngtool utility at /usr/lib/apt-cacher-ng/acngtool to
see the full configuration, with acngtool cfgdump

You can use “Ex” versions of these patterns to override the defaults.
I have changed the patterns so that .*fedora.*updateinfo.xml.zck is
set in VFilePatternEx, so that updateinfo is treated as volatile.
Because of the issues with mirror synchronising, I found it necessary
not to cache updateinfo.xml.zck data from the fedora update
repositories:
DontCache: .*fedora.*updates.*updateinfo.xml.zck

This combination seems to fix the issue, at minor cost. The updateinfo
files are no longer cached, but the packages still are.
I have tested it with combination of cacher and Tor/clearnet, and will
push an updated package that will update the acng.conf file idc.

It’s important to realise that even with no caching proxy and clearnet
access, dnf update can still throw errors. With Tor, the errors are more
frequent. All one can do is wait and retry.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.
2 Likes
32

Thanks unman.
I’m glad to hear it’s going to be fixed, I think cacher is too much useful!!!
I’ll let You know if any other problem show up.
Cheers,
M.

33

looks like there is already a VFilePatternX line in acng.conf
should the line be appended, or replace what is currently there?

34

I don’t see why and how it could be replaced, but maybe I’m wrong.

35

So this means each time for each fedora template we would have to download tens of MB updateinfo files from the repositories? And, when I have 30 fedora templates and update them in a row, 30 times those files will be downloaded?

I don’t get it how it works. Can you please describe?

36

This post relates to use of rpmfusion repositories with apt-cacher-ng.

There is an error, as reported by @mila -

Errors during downloading metadata for repository
'rpmfusion-free-updates':
  - Status code: 403 for
http://HTTPS///mirrors.rpmfusion.org/metalink?repo=free-fedora-updates-released-36&arch=x86_64&protocol=http
(IP: 127.0.0.1)
Error: Failed to download metadata for repo 'rpmfusion-free-updates':
Cannot prepare internal mirrorlist: Status code: 403 for
http://HTTPS///mirrors.rpmfusion.org/metalink?repo=free-fedora-updates-released-36&arch=x86_64&protocol=http
(IP: 127.0.0.1)

Status code 403 is “access denied” - the server has received the request
but forbids access.
This is not related to Tor - the same error occurs if cacher is
connected to sys-firewall.
It seems to be specific to apt-cacher-ng - a normal browser is able to access the
same URL without any problem. wget also downloads the mirror list.

There are two quick possibilities: One is to set “DontCache” for
mirrors.rpmfusion.org. The other is to check the
[FAQ](FAQ - RPM FusionCannot_retrieve_repository_metadata.28repomd.xml.29_for_repository:_rpmfu
+sion-foo._Please_verify_its_path_and_try_again) and follow the “temporary” advice
This involves commenting out the mirrorlist line(s) and enabling baseurl.

"sed -i 's|^#baseurl|baseurl| ; s|^mirrorlist|#mirrorlist|' /etc/yum.repos.d/rpmfusion*free*repo"

This allows caching at the cost of using http rather than https
connections.
I will make this change in next version of the cacher package.

37

@unman I am getting the same error when upgrading a cloned fedora-37-minimal from a fedora-36-minimal

[root@fedora-37-minimal ~]# sudo dnf clean all && sudo dnf --releasever=37 distro-sync --best --allowerasing
14 files removed
Fedora 37 - x86_64                               30 MB/s |  64 MB     00:02    
Fedora 37 openh264 (From Cisco) - x86_64        1.7 kB/s | 2.5 kB     00:01    
Fedora 37 - x86_64 - Updates                    757  B/s | 512  B     00:00    
Errors during downloading metadata for repository 'updates':
  - Status code: 403 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f37&arch=x86_64&protocol=http (IP: 127.0.0.1)
Error: Failed to download metadata for repo 'updates': Cannot prepare internal mirrorlist: Status code: 403 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f37&arch=x86_64&protocol=http (IP: 127.0.0.1)

I tried to commenting out the mirrorlist line(s) as you suggested (but for the fedora-updates*.repo) but no avail:

"sed -i 's|^#baseurl|baseurl| ; s|^mirrorlist|#mirrorlist|' /etc/yum.repos.d/fedora-updates*.repo"
38

fedora-updates and fedora-updates-testing repos never worked for me, actually, cacher or not.

39

Really? They work without cacher.

40

Then, sorry. I’ll re-check. But since I’m using cacher only, I probably got such an impression. Nevertheless they don’t work for me via cahcer ever, so I disabled them.

then maybe their links to leave without HTTPS///?