Cacher (Apt-cacher-ng): issues with fedora updates

This post relates to use of rpmfusion repositories with apt-cacher-ng.

There is an error, as reported by @mila -

Errors during downloading metadata for repository
'rpmfusion-free-updates':
  - Status code: 403 for
http://HTTPS///mirrors.rpmfusion.org/metalink?repo=free-fedora-updates-released-36&arch=x86_64&protocol=http
(IP: 127.0.0.1)
Error: Failed to download metadata for repo 'rpmfusion-free-updates':
Cannot prepare internal mirrorlist: Status code: 403 for
http://HTTPS///mirrors.rpmfusion.org/metalink?repo=free-fedora-updates-released-36&arch=x86_64&protocol=http
(IP: 127.0.0.1)

Status code 403 is “access denied” - the server has received the request
but forbids access.
This is not related to Tor - the same error occurs if cacher is
connected to sys-firewall.
It seems to be specific to apt-cacher-ng - a normal browser is able to access the
same URL without any problem. wget also downloads the mirror list.

There are two quick possibilities: One is to set “DontCache” for
mirrors.rpmfusion.org. The other is to check the
[FAQ](FAQ - RPM FusionCannot_retrieve_repository_metadata.28repomd.xml.29_for_repository:_rpmfu
+sion-foo._Please_verify_its_path_and_try_again) and follow the “temporary” advice
This involves commenting out the mirrorlist line(s) and enabling baseurl.

"sed -i 's|^#baseurl|baseurl| ; s|^mirrorlist|#mirrorlist|' /etc/yum.repos.d/rpmfusion*free*repo"

This allows caching at the cost of using http rather than https
connections.
I will make this change in next version of the cacher package.

@unman I am getting the same error when upgrading a cloned fedora-37-minimal from a fedora-36-minimal

[root@fedora-37-minimal ~]# sudo dnf clean all && sudo dnf --releasever=37 distro-sync --best --allowerasing
14 files removed
Fedora 37 - x86_64                               30 MB/s |  64 MB     00:02    
Fedora 37 openh264 (From Cisco) - x86_64        1.7 kB/s | 2.5 kB     00:01    
Fedora 37 - x86_64 - Updates                    757  B/s | 512  B     00:00    
Errors during downloading metadata for repository 'updates':
  - Status code: 403 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f37&arch=x86_64&protocol=http (IP: 127.0.0.1)
Error: Failed to download metadata for repo 'updates': Cannot prepare internal mirrorlist: Status code: 403 for http://HTTPS///mirrors.fedoraproject.org/metalink?repo=updates-released-f37&arch=x86_64&protocol=http (IP: 127.0.0.1)

I tried to commenting out the mirrorlist line(s) as you suggested (but for the fedora-updates*.repo) but no avail:

"sed -i 's|^#baseurl|baseurl| ; s|^mirrorlist|#mirrorlist|' /etc/yum.repos.d/fedora-updates*.repo"

fedora-updates and fedora-updates-testing repos never worked for me, actually, cacher or not.

Really? They work without cacher.

Then, sorry. I’ll re-check. But since I’m using cacher only, I probably got such an impression. Nevertheless they don’t work for me via cahcer ever, so I disabled them.

then maybe their links to leave without HTTPS///?

You were right. For some reason these two repos (updates and updates-testing) don’t like cacher, regardless of http://HTTPS///. When routed through cacher, even http:// only won’t make it to these two repos (base url or meta link).
@unman, any idea?

What server did you use for baseurl ?
What error message did you get?

I’ve already given you ideas.
It is true that these repositories are hard to work with.
It isn’t true that they cant be made to work.

I exclude the 403 error.

At least part of the time other issue can be traced to trying to cache
data which differs between mirrors. This is particularly noticeable when
the mirror list hands out servers that aren’t covered in the apt-cacher
fedora-mirrors file.
You will see the effect of this in /var/cache/apt-cacher-ng if there are
entries for distinct servers.

Another issue is that the mirrors have various paths to the fedora
update files. If these are not correctly specified in the mirror list
then there will be multiple entries for the “same” config files.
Errors have to be manually fixed.

I am not surprised that the author of apt-cacher-ng gave up on Fedora.
I currently get Fedora 37 updated correctly most of the time.

But then, updates can be seen to fail with no cacher, no Tor.

I am not sure I understand at first what to do. But as usual I’ll re-read your post few times more in order to try to comprehed what you meant on. Will be free to ask for further clarification if (this time not “when”) I can’t get it. Thanks.

Well, for this you were right.
Putting specific mirror for baseURL, for example

http ://fedora-mirror01.rbc.ru/pub/fedora/linux/updates/37/Everything/x86_64/

would work.

So mystery is why cacher interfere with updates metalinks (and default BaseURL) and not with “stable”…

For the reference in cases like this, this wouldn’t work since old release packages cannot be found this way, since they’re moved to archive repositories.

Indeed - if you specify a repository that does not have any packages
or metadata, then you will not be able to update from it.

If you manually download the metalink data you will see repositories
that still hold old release packages. Use one of them.

I currently have almost 100% success with updating Fedora templates
through apt-cacher-ng, updates repositories included.
I have modified acng.conf as I have already suggested, and made sure
that the Fedora mirrors list is updated and showing the correct path
for each repository. There are still some errors where mirrors are not
in sync, which particularly affects “updates” obviously.

As I have said, it’s easy to find complaints about the update process,
and issues with wrong checksums on the metadata. This is not a cacher
specific issue but it is exacerbated.

Another change I made, (referenced on the dnf Bugzilla), was to disable
zchunk in dnf.conf - set zchunk=False to that file in the template.

I’ll push an updated package which will incorporate these changes soon.

2 Likes

But it’s exactly 403 error that we get with these 2 repositories, so I really couldn’t comprehend what your suggestion towards this was.

Have you followed all the suggestions I made in this thread?
Have you ensured that the server paths in “fedora_mirrors” are correct?

To the extent I was able to follow you. I wasn’t the one mentioning this issue, I was just responding with my experience and workarounds and so far I can live with that.
I am sorry if I made an impression that I expected solution for this specific issue.

My suggestion is that if you follow the advice in this thread, and
ensure that the server paths in “fedora_mirrors” are correct, you will
minimise the risk of update errors with Fedora mirrors.

Note I say “minimise” - if you look, you will see many non-Tor,
non-Qubes users complaining about update errors.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.

I have done this initially when setting up cacher for the first time. But checking 127 mirrors each time update fails looks too cumbersome to me and time-consuming then to set one which works at the moment.

And I am not surprised too that including the reason above, most probably I will give up on using apt-cacher-ng. Deleting cacher will leave my system debian-free and indeed tradeoff of using it looks like too high for me at the moment.

Simple script of running update of all the templates and restarting them every night during sleep seems like a real, not virtual caS(c)h(e).

That is, of course, your right.
Since I have apt-cacher-ng working fine with all available templates,
including Fedora, I will continue to use it.
For me, the gain of minimising downloads, and speeding updates, is too
good to give up on.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.
1 Like