Derivative templates

General Discussion
1

Hi Everyone!

While setting up my Qubes systems sometimes I found myself in the situation where it would be nice to have an AppVM that has an extra program installed which is not installed in the original template. As far as I can see, there are two solutions right now:

  1. Use some solution to install the program under /home/user, so it is just in the AppVM and persists.
  2. Clone the template, install the package in it, and switch the AppVM template to the new clone.

Solution 1 is Ok, if a bit “inelegant”, and solution 2 may lead (if taken to an extreme) to a lot of wasted space in templates.

I was wondering if it would be technically possible to create derivative templates (or “templated templates” :-)), where a read-only copy of a template is used as a base, and then template overlay that only records the differences with the base one. This could make having many slightly different templates quite space-efficient.

When thinking about this a bit more, I realized that as long as you only add things to the overlay things should not get too complicated, but the moment when you try to install into the overlay something that conflicts with a package in the base template it could get hairy quite quickly.

So I don’t have anything very concrete to say, except that it would seem to be a nifty feature if it were possible to implement without too much hassle but I am not sure if that is realistic. So I am posting this to hear opinions from more informed people. :slight_smile:

1 Like
2

“inelegant”? Are you saying that I’m inelegant?! :slight_smile:

What do you mean? I like this solution and if the user home is “inelegant” you can always use /usr/local…

1 Like
3

Overall I prefer to install software from repositories using system tools (apt, dnf) when possible so I get automatic updates. But I do not think you can install something with apt under /usr/local (or /home/user) without some hacks.

And I would find installing something available in the repositories by hand just to install it under /usr/local definitely inelegant. But that may be just a personal quirk. :grin:

1 Like
4

You are right. I totally forgot about dnf and apt!

1 Like
5

While technically possible, it’s terrible in practice as our software packaging is not designed to work this way.

A solution could be to install programs from flatpak, in a template install big runtimes, they will be common to all AppVMs, and install the applications with flatpak as user (so it’s stored in /home) and it will inherits the big runtimes from the template installed system wide.

4 Likes
6

AppImages are an alternative solution that do not require managing another TemplateVM or StandaloneVM.

3 Likes
7

Hi Pablob

This could be an interesting feature, and its been raised a number of
times before. I think that cloning templates remains the best option.
If you use minimal templates, the storage overhead is relatively small.
Using a caching proxy will keep a check on network traffic. I provide a
packaged cacher here

I never presume to speak for the Qubes team. When I comment in the Forum I speak for myself.
1 Like
8

I came into this when installing a proprietary VPN I need for work. To keep it isolated (since it is a heavily invasive piece of software), I wanted its own template. But a minimal template is still about 1 GB, so having to use 1 GB of space to install a single piece of software (which is not 1 GB) rubbed me the wrong way. If I could have a overlaid template that only occupied the amount of storage of the extra program (plus some small overhead) that would have been nice in this case.

When I started thinking more about it I realized that with this possibility template updates would become more complicated, and conflicts would be a real pain.

My personal solution, in the end, is to have a single template where I install all proprietary invasive software together.

The discussion has been interesting, thanks everyone!

1 Like
9

For what it’s worth, I have a LOT of templates because I do the clone-and-install routinely.

I don’t install ANYTHING directly on the provided template (be it “minimal” or the full xfce template). I clone to create a sys-base, which has very little on it (stuff to set up my terminal environment mostly); I make sys-net, sys-firewall and so on by cloning that.

Qubes I’m actually likely to do things on descend from a different clone of sys-base which I probably should call app-base, but I just call base. And that gets cloned dozens of times (including clones of the clones) with different apps installed. (However in many cases the only difference between two templates is what it accesses; e.g, usb devices versus the internet–I am trying to get the differences down to stuff that can be done in AppVMs; which will eliminate at least a dozen templates. Templates take up space and take forever to build; just the clone of the (originally minimal) template can take five minutes apiece.)

Something as nasty as your proprietary VPN sounds, I would definitely have in its own template.

Background: /usr/local/bin on the AppVm gets brought over from /usr/local.orig/bin the first time you run the AppVm. So if you are installing something on the template you want to appear in /usr/local/bin (or any other /usr/local/* directory) you must put in /usr/local.orig/. If you are intending to install to /usr/local/bin on the AppVM then install to /usr/local/bin

One thing I have sometimes done is to install a small package I don’t really need everywhere in a template, but then I do one thing to “cripple” it (like moving a file it can’t work without into /usr/local.orig/bin, linking to it (in /usr/local/bin; which is where it will be after the appvm is created) from /usr/bin and then I rename the file). Note that that file will appear in AppVMs but can be modified by them. Then when I create the AppVM I either delete the file (permanently disabling the package), or change it back to the original name (re-enabling it). If that verbal description is a bit bewildering, here is pseudo-code:

(while building the template)
install package X
move critical-file to /usr/local.orig/bin
rename critical-file to critical-file-save
link /usr/local/bin/critical-file /usr/bin/critical-file
[if the template were to run, the package wouldn’t work because the file it depends on appears to be a link to nowhere]

(while building an appvm that uses package X)
rename /usr/local/bin/critical-file-save to /usr/local/bin/critical-file
[now when running that appvm the link in /usr/bin points to somewhere useful]

(while building an appvm that does NOT use package x)
rm /usr/local/bin/critical-file-save
[This package can never run on that appvm.]

Another possibility I also use is to do the install in the appvm where it’s needed, but then copy files from /usr/bin to /usr/local/bin (which will exist once you start the appvm); generally things in /usr/share can be relocated to the corresponding path in /usr/local/share; e.g. /usr/share/icon/my_icon copied to /usr/local/share/icon/my_icon. When you shut down the appvm the installs to /usr/bin, etc., disappear but the ones in /usr/local/bin, etc. remain, and /usr/local/bin should be in your path (and the software that reads desktop files and looks for icons knows about /usr/local/share). The tricky part here is to ensure you find the files, but at least with apt, a file is written [you’ll have to google this] listing every file that is installed; read that and write some script to do those copies. [Or if using salt, write a state to do the copies.]

Note that if the installation package creates a systemd service this might not work well. Systemd as far as I know (someone correct me if I am wrong) won’t look in /usr/local for anything it needs.

2 Likes