Qubes OS Main repo/iso Mirroring Hints

Community Guides
1

Background

I did some research on the Qubes main repository/iso download mirror (ftp.qubes-os.org). There is a brief short page on Qubes OS site with little information on hosting a Qubes OS mirror. Since I used to host an unofficial Qubes mirror in the past, I decided to post this guideline for people who might be interested on hosting a Qubes OS mirror.

Brief Technical Details

Mirroring of Distro mirrors is usually perfored via rsync(1). Qubes source mirror does not provide rsync protocol to the public (or I am not aware of it). But some of its mirrors provide it which makes it perfect for analysis or Tier-2/Personal mirroring. Recommended syncing frequency is not mentioned. This is usually 3 or 4 times a day for release based Distros and is scheduled via a systemd timer/service pair (by comparison, rolling Distros might do it every few minutes via some special trigger!).

Mirror size

Total hard drive space required to host a Qubes OS mirror at the moment is 822.75GB (822,748,334,768 bytes). This info can be obtained prior to the main sync via:

rsync -rtlHnvh --stats rsync://mirrors.kernel.org/qubes

For personal/corporate usage, user are most probably interested only on parts of the mirror. The mirror could be broken to:

  • 150.79G for ISO files of R4.0 to R4.2.1 including beta and release candidates.
    Considering R4.0 is EOL, maybe its ISO should be axed from main mirror (there might be one for keeping historic releases).
  • 671.94G repo directory could be broken to:
  • 145.71M Archlinux repository
  • 8.98G Debian repository also used by Whonix templates.
    (Maybe R2.0, 3.1, 3.2, 4.0 related ones should be axed)
  • 265MB debu directory (possible Debian Unstable Builds?)
  • 657.82G yum directory contains repositories for both Fedora templates as well as dom0. Also the template images. It could be broken to:
  • R4.0: 91.78G - Again, maybe this should be axed from the main mirror.
  • R4.1: 343.49G
  • R4.2: 219.04G.
    If anyone wants only the stable repositories: 21.46GB and stable templates: 29.29GB.
  • R4.3: 3.52G

Debian and Fedora mirrors of each release might contain packages for EOL templates. But axing them is most probably not a good idea.

Required Bandwidth

Required bandwidth for source mirrors (not personal) is not documented. This is usually at least 1Gb/s for most Distros.

1 Like
2

Moved this to the Community Guides category.

1 Like
3

For people who are interested on mirroring the archival historic content, the details are available in this post:

The total size for archival content is around 750GB at the time of this post.

2 Likes
4

And … I only see iso images for R4.2
All the older .iso’s are missing from that archive.

1 Like
5

There is an archive of Qubes versions 1 through 4.1 located in the archive/ repository. All other versions are weekly builds, which I routinely delete to keep only the latest ones. No archive is maintained for these weekly builds. If you need them, you might be able to find some on openqa.qubes-os.org. Unfortunately, I don’t have the resources to store weekly builds for more than a few weeks. With two ISOs generated per week, the storage requirements grow very quickly.

2 Likes