Automate debian-minimal based template creation

OK, a few things I’ve noticed about the Arkenfox profiles setup:

  1. It reinstalls your plugins every time you open the window. This is very annoying because download helper always pops up a tab to congratulate you for installing it (after taking 5 to 10 seconds to re-download for the nth time). It generally pops up halfway through my typing the URL I am interested in, and of course it seizes keyboard focus mid-URL. I was trying to avoid gratuitous stupid tabs like this.

  2. It seems like most of the web pages I visit just present me with a blank page. I then have to go into settings and re-enable everything that got disabled, though I never found a switch to (re) enable javascript (is it disabled?), so oftentimes that isn’t enough to do the job. (The only thing that seems to be under my control at runtime is cookie/tracking preferences.)

If this is a disposable template, why are we disabling that stuff? It supposedly CANNOT do us any lasting harm.

  1. even though the settings clearly say google search, etc. aren’t being installed they still show up in the running window as choices. In fact, I haven’t figured out how to COMPLETELY DISABLE search from the address bar. [To be sure there may be no such way. Damn Firefox for putting that “feature” in and replacing autocompleting of an incomplete URL.] Oh, you can put a SEPARATE search bar in, but that doesn’t make the URL field NOT a search field. In fact, right now my address field (in a window I just opened) shows google, and everything else that’s supposedly disabled, as an option at the bottom of the dropdown when I put the cursor in the URL-and-dammit-search bar.

Part of the problem with this whole schema is if you don’t know what Firefox named the settings you can’t adjust the ones not already referenced in the template. That’s on Firefox for being obscure, though, not on anyone here.

So in sum, I wonder why I bothered with this. It gives me a non-functional page most of the time (blank when I visit a website) so I end up having to use a regular window to go visit, and it STILL isn’t clean by my lights.

1 Like

You might want to give LibreWolf a try, instead of constantly fiddling with arkenfox.js and about:config settings of Firefox.

1 Like

I will set LibreWolf as my new default Firefox for all my minimal templates. It is how Firefox should be by default.

1 Like

Looks like it’s one of those doggone things you need “wget” to install.

wget or curl both do the same, both are no problem.

Just open the template terminal and follow the install routine. Make sure you have wget or curl installed (apt install -y curl wget).

http://127.0.0.1:8082/ is doing the magic.

curl should also work:

curl --proxy http://127.0.0.1:8082/ -s https://deb.librewolf.net/keyring.gpg | apt-key add -
1 Like

As I recall there was some issue with exposing dom0 to the internet with that stuff.

You don’t install LIbreWolf to the dom0.

You don’t, but in order to install it ANYWHERE, don’t you have to expose dom0 for that stuff?

@Sven
I was reading your blog post about automating the debian templates.

I have a question regarding this line:
“[We set] the maxmem property to zero, which signals we do not want memory balancing.”

Why is this? From reading the
$ qvm-prefs --help-properties debian-11
and
$ man qvm-prefs
I see that maxmem allows the template (and the app-qubes based on that template) to dynamically adjust the memory allocated to them (starts with 400 MB and goes up to 4 GB).

Why do you suggest switching off this behavior for a deb-11-minimal template?

Nope, but you should know what you are doing. Best is to first clone your template (or even better using a minimal template) and install with http://127.0.0.1:8082/

1 Like

Oh, the irony. That thread you pointed to? That’s the one where I remembered it being a big heaping smelly issue…and the second to the last post (#95) was me posting my (partial) solution to the issue. asking if anyone could critique it from a security standpoint. (I have installed brave and veracrypt this way.) I had completely forgotten about it.

OK high up on the to-do list is trying out LibreWolf. Thanks!

2 Likes

Why do you suggest switching off this behavior for a deb-11-minimal template?

I don’t have a good/technical reason. My impression (which could be wrong) is this:

  • Linux seems to always grab as much memory as you give it
  • memory balancing seems to come at some performance cost
  • since I carefully measured and experimented to determine the maximum memory needed for my qubes to work the way I want, there isn’t really a need to allow them to take any more

An exception to this may be qubes running web browsers. I’ll spare you my rant about web browsers and the amount of memory they consume. Google’s Chrome seems to work well with 1GB memory in my scenarios, but I am compartmentalizing heavily so you won’t see me use more than 2-3 tabs tops.

2 Likes

I’ve been playing with minimal templates a lot. (In fact, as of now the only thing not based on minimal on my system is sys-usb…because I have no idea what to do with that one.)

One thing that comes up, a lot when trying to regenerate the templates is the system refusing to do so because such-and-such template (call it deb11a-net, just for instance) is “in use” because it’s a template for some other qube, or (worse) it’s the minimal dvm for some other qube (and that’s usually a lot of them).

Certainly, in a script you can temporarily re-base the affected qubes, rebuild the one you’re interested in, then switch them back…but in that case you have to hard write those qubes into the script, and if you add new qubes [and I gather you do, a lot, because you’ve compartmentalized with a very fine grain], you have to maintain the script.

I don’t imagine you actually do it that way. I can imagine you’ve written a script to do a qvm-ls, hunt for references to the template (or dvm template) you want to re-build, switch them temporarily to a dummy template, rebuild, then switch them back (by hunting for references to the dummy, which should be a qube you never actually use). I’ve been thinking about doing this, actually. For now I have things hard coded, most places, with commands referencing the dependent VMs by name. But I want to get to fully automated, like you are.

Certainly you have to do something about this or you’d never be able to walk away from your system while you were maintaining it!

Is one of these basically what you did, or do you have some other solution?

My scripts are arranged in an hierarchy. The top-most one is called build-all-templates and it currently looks like this:

#!/bin/bash

ORG_TEMPLATE_NAME=debian-11-minimal
NEW_TEMPLATE_PREFIX=debian-11

build-deb-min     $ORG_TEMPLATE_NAME    $NEW_TEMPLATE_PREFIX

build-deb-usb     $NEW_TEMPLATE_PREFIX
build-deb-net     $NEW_TEMPLATE_PREFIX
build-deb-cacher  $NEW_TEMPLATE_PREFIX
build-deb-salt    $NEW_TEMPLATE_PREFIX
build-deb-web     $NEW_TEMPLATE_PREFIX
build-deb-brain   $NEW_TEMPLATE_PREFIX
build-deb-jekyll  $NEW_TEMPLATE_PREFIX
build-deb-heads   $NEW_TEMPLATE_PREFIX
build-deb-lib     $NEW_TEMPLATE_PREFIX
build-deb-mail    $NEW_TEMPLATE_PREFIX
build-deb-office  $NEW_TEMPLATE_PREFIX
build-deb-signal  $NEW_TEMPLATE_PREFIX
build-deb-element $NEW_TEMPLATE_PREFIX
build-deb-teams   $NEW_TEMPLATE_PREFIX
build-deb-vault   $NEW_TEMPLATE_PREFIX
build-deb-rdp     $NEW_TEMPLATE_PREFIX

As you can see this script has two variables at the start: ORG_TEMPLATE_NAME is the name of the Qubes OS provided template that build-deb-min will install, clone and subsequently remove again. NEW_TEMPLATE_PREFIX is how the name of all my templates start.

As outlined in my note these build-deb-xxx in turn clone from the -min template just created and then call add-feature-xxx scripts to install specific capabilities (net, audio, usb, qt, …) or programs (libreoffice, vlc, calibre, …). But that’s just a side note and not relevant to your question.

Of course I have encountered your issue. In a nutshell what I do is this:

  • always build the whole set of templates by calling build-all-templates
  • if the templates that are currently in use start with debian-11 then I change the prefix in the script to deb-11 or vice versa.
  • I then use qvm-ls --field=name, template > tmp to list all qubes with their templates
  • in that tmp file I then remove all lines that are actual template qubes, named disposables, standalones and dom0 proper
  • next I add qvm-prefs at the beginning of each line
  • next I search and replace the debian-11 that is part of all template names with template deb-11
  • finally I add qvm-shutdown --all --wait at the beginning and qvm-start sys-usb at the end
  • then I make the script executable via chmod +x tmp and run it

Because all qubes are shutdown while you do the change you won’t run into ‘in use’ errors. Don’t worry about your disposables either, since they are based on other qubes that in turn are based on templates (which will be changed in the script). For extra points you add qubes-prefs default_template deb-11-min at the end of tmp and qvm-remove -f debian-11-xxx calls for all the templates that are now unused / unneeded.

I actually described this before and another forum member posted an improved generic version you might want to check out.

Finally I feel obliged to point out once more that the thing I am doing and describing here is a poor mans solution to a problem that is far more elegantly solved using salt. However the strong response it is getting seems to indicate that it may be a bit more accessible.

1 Like

Yep, that’s another way of doing it, and quite feasible for me, actually.

My big script defines two prefixes now: deb11m and deb11a. m stands for minimal, and a for app. Now the 11a templates ultimately spring from the minimal template but they all start with so much stuff it didn’t seem right to me to continue to call it 11m. (Basically 11a is the bare minimum I’d want to deal with as a user.) Also, they alphabetize cleanly into two groups: templates for system level VMs, and templates for “user level” VMs–handy in menus and the manager (which I do use a lot since it shows me multiple VMs at once and it’s easy to do a side-by-side comparison (even if it’s actually over-and-under).

I could always just change these two to something different (then next time change them back) and I’d be close to what you’re saying at least in concept.

I thought I had read this entire thread, but I must have got overloaded because I didn’t remember the prior description you referenced. Sorry about that!

I agree about salt. It looks like it could be wonderful, but I simply can’t understand the documentation because it talks about entities (columns, etc.) without relating them to what it is I am actually trying to configure (and if it’s not something made for Qubes that is quite understandable, but doesn’t make it easier). What corresponds to a VM Template I am trying to create? I dunno.

Lots of food for thought here; thanks!

I finally got to this

Unfortunately, no matter how many hoops I jump through the damn thing is unfindeable.

When I try to issue wget, it first tells me I already have the key ring then it says it can’f find ‘deb.librewolf.net

I wish people would just use a damn regular install sequence instead of overcomplicating it with this crap.

1 Like

I thought it is not worth to open a new topic on this and since it is part of my auto deb-mini tpl script I place it here.

I tried to make a generic function for gpg key download and apt update. Beside the well known Signal case, I picked Sublime and Librewolf. Now, I wonder about their installation instructions.

Librewolf

(Installation on Debian based systems – LibreWolf)

wants to save the gpg key to
/usr/share/keyrings/

and for the apt it is suggesting a .sources file
/etc/apt/sources.list.d/librewolf.sources
with a Signed-By: /usr/share/keyrings/librewolf.gpg link.

Sublime

(Linux Package Manager Repositories)

gpg key instructions points to
/etc/apt/trusted.gpg.d/

and the apt wants a .list
/etc/apt/sources.list.d/sublime-text.list

My question what are the differences? Why, historical reasons, more generic, more robust…? Any pro and cons?

Any pro cons related to Qubes OS, or minimal templates?

Thanks

1 Like

ok, found it:
https://manpages.debian.org/bullseye/apt/sources.list.5.en.html

side note: apt-key(8) will last be available in Debian 11 and Ubuntu 22.04.

1 Like

Thank you for doing this @Sven. Already recreated most of my qubes using this method. Very easy to modify and create templates as needed. Hopefully this method works into the foreseeable future.

3 Likes