Sys-update gentoo system

General Discussion
1

I think I have made some relevant comments about this in the past but I am thinking of creating a system where every qube is gentoo compiled, I think it can be achieved in the same time it would take to compile a bloated gentoo system which is actually quite fast if you only update the @security system set bi-daily and the world set just about monthly, honestly with todays 24 threads in an average computer it might be as fast at it takes to download updates from tor for every single template.

Some differences have to be taken into account when attempting something like this in qubes or a local network, to make the process of updating the system simpler I think ignoring portage and using paludis is the right choice.

Lets break some things down, to achieve head-ache less updating and managing of multiple machines some unification has to be made in the configurations, we can group settings together and distrubute them to each system using .d/*.conf magic as well as overlayfs, it is quite simple in execution:
A) create an /overlay/global/etc/ folder stracture and place the necesarry settings in the lowest priority like (for portage) /etc/portage/make.conf/00-global.conf.
B)IUsing crossdev you create a new environment
C) You overlayfs mount /overlay/global/ to a chroot the crossdev environment

Now your to-be-compiled qube has your global use flags pain-free(overlayfs might need remounting if global settings change)

Now the choice of paludis
A)SYSROOT, ROOT and other such env-vars in portage are never working how you wold expect them to work, portage generally makes the compilation of software for another device a pain unless you are using both devices with distcc or you are using crossdev, I have lived dependency hell on loop for weeks in the past trying a bunch of different setups involving that exact scenario, paludis on the other hand makes this almost pain free as it allows you choosing a “compiling root” and a destination, here portage would use both the /etc/portage and the /usr/x86_64-crossdev-glibc/etc/portage folders, no matter what value ROOT, SYSROOT or CONFIG_ROOT have, unless you want to start every vm’s useflag with a -* then dont bother with portage at all, or strangle with headaches trying to find waldo in between 2 or more config_roots(overlayfs remember?)
B) It is unfortunate that paludis’ dependency resolution is mutch, mutch smarter then portage’s, at this point it is almost a joke since 90%+ of gentoo user’s dont even know paludis exists and 5% of the ones that do don’t want to bother learning a new package manager
C) paludis offers suggestions, that means you can create a suggestion like “www-client/* bluetooth pulseaudio”, now in general this wild yield no difference in what flags are used to compile www-client packages, unless for some reason any of these clients require pulseaudio or bluetooth to solve a dependency, in which case it will automatically enable required flags for that single dependency resolution, do you get it? Normal portage allows you to choose yes or no, hot or cold, enable or disable flags, paludis allows you to say “oh well if I have to, use that flag, just dont tell me about it I just want to install mah open source software”
D) It is match easier to create hooks during any point of the installation or compilation process, useful for communicating with dom0 or another domU during an update, running custom setup scripts etc
E) Lastly strictness, paludis unlike portage does not allow the creation on directories if not specified bh the ebuild, which is something that could create pretty serious security holes for someone with just enough knowledge to find them

so here is the theory

  1. create /overlay/network-manager /overlay/global /overlay/xorg /overlay/service-vm
  2. mount /overlay/network-manage and /overlay/qubes/sys-net to /usr/x86_64-musl
  3. either boot sys-net-template and NFS into it to /chroot/sys-net or mount it by creating a script that asks dom0 to get access to the disk, then mount that disk to /chroot/sys-net
  4. update the vm using paludi’s config_dir and root variables appropriate for sys-net
  5. unmount, ask dom0 to shutdown sys-net-template(or revoke access to the disk and then ask dom0 to start and then stop sys-net-template)
    5.5) if eselect is needed for anything or any other portage command execute cave exec -e sys-net(environment name) eselect locale set . . .
  6. repeat for the next qube

This whole process saves time because paludis can save compiled packages together with their metadata (like cc flags or use flags), if you used portage you might overwrite a package you compiled earlier because you changed a few flags or so

There are only a few things to iron out in this stracture:
A)The user must watch out for ghost flags, things like that can easily break because overlayfs is a very simple filesystem with very complex consiquences when missused
B)I am still not sure if NFS is the right way to do this or not
C) I am still trying to figure out a stracture to have catalyst included in this whole stracture or if I can use Exherbo’s project-builder as an alternative to catalyst, , most likely the latter
D) I am not sure if I am ever going to finish this by the and of the year if at all by next year since ram prices are still skyscraper high, mostly because I am not ready to use gentoo again untill I am ready to compile just about everything compilable with filcc (rust’s clang and clang++ cousin), but the filcc overhead is 4x at this moment

Anyway I am just giving out ideas to the community, if anyone tries this out then I ourge you to not underuse overlayfs, it is a very useful tool for such cases and if you are a tinkerer who wants full control but smooth updates then dont let suggestions.conf from paludis out of your eyes either, imagination is your limit yall

Edit: last time i tested musl as dom0 on a gentoo workstation it worked just fine, there where a few missing libraries a year ago but everythng should work now, so if someone ports dom0’s software stack to a gentoo overlay then even dom0 can be updated that way, its a shame that openbsd/gentoo has been dropped year ago, it would have been the perfect playground for kernel tinkerers to add full qubesos support to an openbsd sys-net.
On another note keep in mind that you can create an overlay, copy the base profile and edit the copy to block portage from being installed on domU’s, you can go as far as to make the whole update process happen inside sys-update, package it and sign it to create immutable qubes before updating the qube and applying the key, but that needs a little more research on my part

2

Some have the practice of comiling whatever operating systems are used on their private machines. The compile jobs are rarely done on the machines that have a keyboard attached to them.

One of these operating systems could be Gentoo. Whenever I read a remark about the overhead of compiling Gentoo I think “huh, you never figured out that compile jobs should happen on a server”.

Like Fedora and Debian, the packages compiled by a Gentoo system for Gentoo systems that may run in qubes should be for the most part Qubes agnostic. Also like Fedora and Debian, there should be some qubes that are run for tests at some build job steps that run on Qubes OS with nested virtualization on a server.

There may be some cases where emerge must run in a qube on a full Qubes OS instance to complete, but those should be rare and when found should be possible to eliminate.

Running Gentoo in a qube on Qubes OS is necessary for running tests for proper Qubes OS integration after build jobs. But running Gentoo in a qube for most (if not all) runs of emerge should not be necessariy and is likely not necessary at all.

Maybe we can collaborate on supportable Gentoo templates for Qubes if you would like.

I am interested in both OpenBSD and Gentoo on Qubes. We can, all of us can enjoy OpenBSD and Gentoo on Qubes, along with all of the benefits that come with those. No one can stop us.

3

I am quite sure that paludis can perform post install scripts better then portage when dealing with an alternative ROOT or SYSROOT, but that is an area where I need to research so do not quote me on this one.

Both these cases require coordination with the qube’s overlay maintainer (which I believe is a community member not a qubes dev or team member), otherwise a separate overlay should be hosted in such cases, either way there must be someone to version the qube’s overlay packages’ version keywords to amd64, ~amd64 and 9999, I am also not to keen on creating tests for a qubesos system that works in a nested environment like inside a VPS or otherwise, mostly because of my currently low power setup making me working on such project extreamly slow (think 2010 laptop), or I would have to rent a deticated server to perform such tests

Maybe, I can imagine multiple templates for such a system, but at the same time I think creating more then 2 templates is ill advised, contrary to my wishes a systemd+glibc and a systemd+musl template are the best ways to keep the community’s focus concentrated to 2 sub projects rather then 10+(as based on the profiles), openrc can be worked later by forking the alpine template’s openrc scripts, lastly the only real focus such a system requires is security audits, basically what is the best way to connect sys-update with the rest of the system, should sys-update grub updates directly from sys-whonix or should an intermidiate qube be there to download the package’s source code and then pass the /var/lib/portage folder to sys-update, should we solve package dependandies on sys-update, save the state(/var/lib/portage or /var/lib/paludis) and pass it to the template to update itself by pulling the already compiled packages from inside sys-update? Extending on that question should templates be given enough cores to compile missing packages themselves or should they be only allowed to install “binaries” from sys-update?

The only answer I have regarding package install safety is this:
To avoid system infection sys-update should never have any interactions with any templates, I would install paludis or portage inside the /rw folder of all templates, allow the templates to pull updates from disp-repository and have sys-update pull and push packages to disp-repository

A minor note I could make that will most possibly cause the gentoo community chase me with pitchforks:
I think a sys-ai could intergrate a local ai chat with the sys-update to “semi” automate the update process like

"AI: Warrning, updates on template: personal-template have been paused due to a dependency conflict which can be solved by enabling the +X flag on or disabling the +wayland flag on and giving it an +X flag instead

Flags:
X: The X flag enables the Xorg window manager, an archaic but trusted and beloved window manager by the community, considered less secure then wayland
wayland: The new slick boy window manager, considered more secure and efficient by the community

Would you like to hear more or to change would you like for me to make the change and continue compiling the packages?

You may hate me for this but I think this could bring a lot of fresh blood to gentoo and initiate them to the configuration colture.
Edit2: I dont mean such an ai should wholey configure a gentoo system let alone a whole fleet of them, that would be the staff of nightmares until we can run something like deepseek-v8-pro-max-agi in a laptop, so around 2035-2050, honestly I get shivers thinking about an ai breaking my system every 5 minuites, no, an ai can act as an assistant for an already build an set-up system where during an update or while adding a new package a small dependency conflict comes up, this solves the time from the user typing away to find out about a package and it’s flags or what will change, which again he can then type away and re-confirm themselves, this has a point to assit the user to manage a fleet like a commander, the user should be an admin not a todler.