New Gentoo templates and maintenance infrastructure

This is the third article in the “What’s new in Qubes 4.1?” series. Previously: The New Qrexec Policy System and The GUI Domain.

New Gentoo templates

The work I’ve been doing on Gentoo templates is finally ready to be released! The corresponding issue is #4412, where you can find almost every related piece of work. I would like to highlight that this has been a great opportunity to collaborate with the Gentoo core team, and multiple improvements have been implemented on the Gentoo side thanks to the help of Gentoo devs mgorny and zmedico. When I encountered issues, I appreciated the quick response from several maintainers of different Gentoo components.

Why a Gentoo template? Gentoo is designed for users who want to customize a Linux distribution to fit their specific needs. The benefit in Qubes is that it allows one to create highly customized and hardened TemplateVMs (or StandaloneVMs). For example, one could customize the Gentoo Qubes Builder to create a ClipOS build.

The new Gentoo templates are available in three flavors. The default (Gnome), minimal, and XFCE. Currently, they are available in the qubes-templates-community-testing repo, and they’ll soon be in the qubes-templates-community repo. (Update: As of 2020-10-20, they are now available in the stable qubes-templates-community repo.)

Maintenance infrastructure

In order to keep the new Gentoo template in good working condition, we need a set of automated tests. The bare minimum is continually testing whether just building the template still works. Due to the nature of Gentoo, such tests require far more resources (time, CPU power) than are available for open-source projects on Travis CI (which we use for testing other templates). We use Travis CI as a bare minimum for validating every pull request, then it generally goes to openQA. Here, the issue with Gentoo is that, by definition, it’s a source distribution, so everything needs to be rebuilt. Travis allows a maximum timeout of something like 50 minutes for jobs. Simply rebuilding a Qubes component for Gentoo takes several hours (and more than half a day for each template). So, Travis is out.

I’ve set up in our pull request pipeline the use of a feature of Gentoo that allows us to get pre-built binaries from a mirror. For that, when I build a full template, I push a fresh repository with every package built on my mirror. But still, even when doing this, hours are needed for jobs. I didn’t want to give up on this, so, after evaluating several options, I decided to set up my own self-hosted GitLab CI instance. For that, I’ve developed a service that I call qubes-g2g-continuous-integration, forwarding selected GitHub pull requests to my GitLab CI instance.

In consequence, I can manage the allocated resources for Gentoo builds, and we now have the means to validate every pull request for each component that has been integrated into Gentoo. A check status appears on GitHub side-by-side with Travis checks for other distros where everyone can access the build logs too. Here is an example. By the way, this is also what we use for automatically checking kernel pull requests. Here is a recent pull request. When viewing the checks, you can see the results for the kernel builds. Once again, it’s because the build time is superior to what Travis allows.

Conclusion

All this infrastructure is intended not only for the kernel and Gentoo, but also to help Arch Linux users. Depending on the needed resources, we could also add the longer Arch Linux jobs into my GitLab CI instance, because we currently don’t properly validate the template itself.

In general, this new infrastructure allows us to create and test pipelines that require a lot of resources. Combined with Travis and openQA, we have another layer to rely on for validating and automating the building of cutting edge templates like Gentoo and Arch Linux. While the same kinds of features are available on paid GitLab plans, I preferred to do it myself with a free self-hosted instance, which allows me to provide as many workers as I have available.

Our overarching goal here is to broaden the scope of things in Qubes that we test automatically. This helps not only with increasing the quality of software we deliver, but also with saving developers’ time, since more automated testing means less need for time-consuming manual testing.

Thank you to Marek Marczykowski-Górecki for contributing to this article.


This is a companion discussion topic for the original entry at https://www.qubes-os.org/news/2020/10/05/new-gentoo-templates-and-maintenance-infrastructure/
4 Likes

Not enough. I need about treefiddy

Jokes aside, this is great news–Gentoo was the first Linux distro I heard of when I asked a CS student which OS was the most secure (in 2012). Chrome OS is based on Gentoo and it’s considered one of the most locked-down systems out there (but a large part of that seems to be because of the verified boot, which cannot be replicated in a VM AFAIK).

I never got around to learning it since just installing it was far too daunting for a point-and-click Windows/OSX user. Looks like I finally have a reason to learn Gentoo and spend all my time compiling everything. Thank you!

2 Likes

Quick question–I’ve already installed the tree flavors in R4.1 but I want to install them on my R4.0.3. However, Gentoo isn’t in qubes-templates-community-testing for that release. Is there a way for me to install it, or is it limited to R4.1?

Yes it’s limited to R4.1 and there is no specific plan to make it available for R4.0. That would require too much work and time.

BTW, just a tip: https://www.qubes-os.org/doc/templates/gentoo/#after-installing. Take care of allocated resources of your Gentoo template. If not, you would certainly scratch your head for non-sense reasons.

2 Likes

I’ve been trying Gentoo for a while years ago but wasn’t ready for the use-flags (or patient enough) and maybe for Linux in general back then. I stayed with Arch for several years before switching to Qubes.
Looking forward to trying out something new!

1 Like

Good to know for when I start poking around

And on that upbeat note, we might need to set up a support group for the muggles (like myself) who will be traumatized by Gentoo

Thanks for noting that. Pushed typo fix.

The overlay has now release4.1 branch. A new template will be built with correct overlay branch selected. In order to prevent any next R4.2 possible compatibility breakage like with qrexec or gui-agent, for user already having Gentoo installed please do:

$ echo "sync-git-clone-extra-opts = --branch release4.1" >> /etc/portage/repos.conf/qubes.conf

and remove the existing git repository located at /var/db/repos/qubes in order to force emerge to clone the appropriate branch.

Is it just me seeing error messages like eutils: EAPI 7 not supported for all the qubes-specific ebuilds?

No it’s something else (so I also opened a new issue and a PR):