Building templates without building anything...in 60 seconds

Here are short instructions to build an up to date Fedora template the absolute quickest way and minimal headache. Maybe not 60 seconds…but this is a high level overview.

Advantages and disadvantages are covered after.

# Get the qubes builder
git clone https://github.com/QubesOS/qubes-builder
cd qubes-builder
# Use the base config already in the repo
cp example-configs/templates.conf builder.conf

# Append custom settings to builder.conf
# Better to update values in-place, but this is the 60 second version
cat >>builder.conf <<EOF

# Build Fedora 33; fc33+minimal or other flavors could be given
DISTS_VM = fc33

# Use the Qubes online repo for Qubes packages since we are not building them
USE_QUBES_REPO_VERSION = \$(RELEASE)

# Use the 'current' repo; not 'current-testing'
USE_QUBES_REPO_TESTING = 0

# Don't build dom0 packages
TEMPLATE_ONLY = 1

# Since this is TEMPLATE_ONLY, list the only components needed
TEMPLATE = builder-rpm linux-template-builder

EOF

# install build dependencies
make install-deps

# Download and verify sources (only Qubes git repos)
make get-sources-git

# Build the template
make template

The template will be at qubes-src/linux-template-builder/rpm/noarch/*.rpm

To give the template a different name so that it can be installed alongside the official Fedora templates and is not seen as an update,

make TEMPLATE_NAME=fedorabuilt template

Advantages:

  • No build chroot is setup, no code is actually compiled saving download and time compared to a normal build
  • Uses official Qubes repo for qubes packages
  • Downloads most up-to-date Fedora packages at build time, so no need to download 2GB of templates and an additional 1GB of updates
  • Moves more download bandwidth onto Fedora mirrors as opposed to Qubes/Qubes mirrors
  • Learn how the Qubes build process works

Disadvantages:

  • Doesn’t build Qubes packages from source? Check your threat model first
  • The template RPM will be unsigned, which makes installing it more difficult in Qubes. This is meant to be installed on the same computer; distribution is not recommended.
  • Depending on internet speed and computer performance, it may take more time
  • If the build fails, debugging is not very fun

This same process should in theory work on other templates (Debian, Archlinux), but may require some COMPONENTS to be built first.

This means: add more components to the TEMPLATE variable, make get-sources, make qubes-vm, then make template.

3 Likes

I tried to create a debian template and had success when I applied the following changes to the script:
DISTS_VM = buster+gnome
TEMPLATE = builder-rpm builder-debian linux-template-builder
and adding the following lines after make get-sources-git:
make get-sources
make install-deps
make qubes-vm
I am not sure, however, if all three lines are necessary.

So the method works not only for fedora templates.

One more remark: Installaition will fail, if the file /etc/dnf/dnf.conf contains the line localpkg_gpgcheck=1, as recommended to squelsh the error message on updating fedora templates.

That is good to hear it works seamlessly for Debian, I had not tested Debian.

For a Debian build, the input you’ve provided is sufficient. The primary change to builder.conf is to ensure DISTS_VM contains the right distribution specification, and that TEMPLATE contains the right build-[distro] QubesOS repos (which have rules for how to build the corresponding distribution).

For those interested,

TEMPLATE_ONLY=1
TEMPLATE=builder-rpm component1 component2 ...

is equivalent to:

COMPONENTS=build-rpm component1 component2 ...
DIST_DOM0=<empty>

- but I like seeing TEMPLATE_ONLY as a helpful indicator of what the intent of the build is. You can always override all of these variables when invoking make.

With a Debian build that doesn’t need to build code, the minimum sequence should be:

  1. Set right values in builder.conf, below is if we were editing builder.conf, not doing cat ...:

    DISTS_VM = buster+gnome
    USE_QUBES_REPO_VERSION = $(RELEASE)
    USE_QUBES_REPO_TESTING = 0
    TEMPLATE_ONLY = 1
    TEMPLATE = builder-rpm linux-template-builder builder-debian
    
  2. make get-sources

  3. make install-deps

  4. make template

In my original post for a Fedora template, install-deps followed by get-sources was sufficient because we are building an rpm on an rpm-based distribution. For a Debian template or others, yes, the order should be get-sources (so that Qubes-builder sees that builder-debian needs more stuff) followed by install-deps. This is the better approach overall. So anytime new source code is downloaded and placed in qubes-src, there might be a need to run install-deps if the repo needs anything special to build.

The reason you don’t need make qubes-vm is because there are no COMPONENTS (or TEMPLATE entries) that define a rule saying “compile me for installation into a VM”. If there were, a build chroot would need to be downloaded and installed and the code for the component would be built inside of the chroot. That then generates an rpm (or deb) which is placed in a local repo - being available for installation during template creation.

If anything is being created when doing make qubes-vm, it means something is being built locally instead of using the official signed packages. What I earlier described was for the case when something must be built locally before the official Qubes repos can be used (as is the case for Archlinux currently).

The key is simply that there is a COMPONENTS variable that defines stuff that should be built for installation into a VM and stuff that should be built for installation into dom0. Then there are COMPONENTS like builder-rpm, builder-debian, linux-template-builder that are somewhere in the middle - they technically also create stuff that is installed in dom0, but no code should be running in dom0[see note].

This process is to create an up-to-date template when needed, using official Fedora- (or Debian/Arch/etc) and Qubes-signed packages. Creation of a template is largely the installation of rpm or deb packages - we are choosing to get those packages online instead of building them ourself.

note: aside from rpm scriptlets executed at installation/uninstallation

Unfortunately that is a result of QSB-67. To install unsigned packages one must use:

rpm --define '_pkgverify_level digest' -i /path/to/the/package.rpm

That is why this process is only recommended if you are building a template on your machine and installing it on the same computer. All packages will still pull updates from the official Fedora/Qubes repos.