Idea for a Program

User Support
1

Hi,

I have an idea for a program for dom0 and don’t know how to do it. It really should just be a script.

The way it works is it would be migrate-template template-1 template-2

and then the script would run dpkg -l (or the equivalent in the template)

then, somehow, it would find the dpkg -l for the template before anything was installed (to do this, it might have to rename template-1 to template-1-bak and then download template-1 again)

then it would take the difference in the two lists to determine all programs that were installed after the template was downloaded

then it takes new template-2 and downloads all the program using the appropriate commands (sudo dnf or sudo apt etc)

the script would also need to test to make sure template-1 and template-2 were the same distro

Does anyone think this idea would work and would anyone be willing to help me with the idea? I think it would make it easier to move from one template to another without wasting time.

I think it could be written as a bash script, posted on github with the hash included, and then it might be using Qubes easier.

2

I’m not quite sure why you want to just recreate the exact same template from scratch. What would be your point of doing this other than to know what you previously installed? You can just back up the template and restore it later. You can create a list of packages, copy it offline, and diff/compare it later.

But if you need to do this you can just list the packages in the existing template and redirect that output to a file, download the same template again with another name, catalog that new template to a file, qvm-copy the text file from old to new, and then diff the two lists in place. Anything extra in the first list is what you need to install in the newly downloaded template to make them the same.

But this sounds very similar to the problem of upgrading to a new vanilla template OS version. When downloading the very latest version of a template it would make sense to catalog the old template version and then install all the previous apps in the bran new template (e.g fedora-41-xfce => fedora-42-xfce). That is something I have played with but never created anything that I thought was worthy of being published. Too many things can change between upgraded versions and you need to handle the repo configuration files first before trying to install anything. And then contend with printer configs and all the other things. Half the time its just easier to upgrade in place if things are working well in that template.

3

Is there a way to do this -to install a template with a name different from its packaged name - if the original template is in use?
I end up renaming all my base templates, just in case I need to obtain a pristine copy.
I have a feeling I’m going to feel quite stupid, although I’ve done a fair bit of manpage reading, hunting for that elusive option…

4

Get list of manually installed package (manually = not as dependency)

apt list --manual-installed 2>/dev/null | awk -F '/' 'NF > 1 { printf $1" " } END { print "" }'

Copy-paste the output as parameter to apt install

5

As far as I know, template creation has already been sufficiently automated. Here is a take on how to achieve it with salt:

(Btw, you can find some further reading on using salt with QubesOS in the official documentation.)

But as for me, template management is not something I do a lot of, so I find it perfectly sufficient to keep a record of my installed packages in a text file.

If you don’t know what you’ve installed since the template was created, check the logs:

sudo zcat /var/log/apt/history.log.*.gz | cat - /var/log/apt/history.log | grep ^Commandline:
1 Like
6

If I created a script migrate-template template-1 template-2, is there a way to query template-1 using dom0 to determine whether it uses apt, dnf, or neither?

I would need to be able to execute a terminal from dom0 based on a script and then type into the terminal based on the script and then somehow have dom0 be aware of that outcome, probably by coping to dom0 in a text file using cat and then querying that text file?

also, is there a way for dom0 to open up template-1 and type in

apt list --manual-installed 2>/dev/null | awk -F '/' 'NF > 1 { printf $1" " } END { print "" }'

and then copy the result back to a file?

the file could be called “migrate-template.txt”

7

i read the salt documentation but it was hard

i don’t understand what salt does still or what unman said

8

I’ve never tried to learn salt, but yeah, from what little I’ve seen it doesn’t look very approachable. They say it’s easy once you get the hang of it tho; Maybe someone on the forum has written a good guide for it. But tbh, I have very little motivation to get into it, since automation in future versions of Qubes will be moving to Ansible, allegedly.

9

If you don’t want to go through salt automation, you could have a simple shell script running apt install with a very long line of packages, and copy it from template to template when upgrade to a newer version. This could also have an associated README file or comments for specific instructions that you changed

This way, it’s easier to apply the same changes when you want to switch to a newer template.

10

Salt and bash scripts are helpful in building and rebuilding the
templates.
In salt, building the list is as simple as this -
As root in dom0 create a file in /srv/salt/my_install.sls:

Install_my_packages:
  pkg.installed:
    - pkgs:
        - PACKAGE_NAME_1
        - PACKAGE_NAME_2
        - PACKAGE_NAME_3
        - PACKAGE_NAME_4

Then apply it like this:
sudo qubesctl --skip-dom0 --targets=TEMPLATE_NAME --show-output state.apply my_install

If you maintain that list as you want to have new packages in the
template, then you can recreate (on this or another machine), using:
sudo qubesctl --skip-dom0 --targets=NEW_TEMPLATE_NAME --show-output state.apply my_install

That’s the simplest use of salt. Slightly more complex is using variables
to allow for cases where package names differ between Fedora and
Debian, but you dont need to do that.

However, @wmrom2, if I understand them right, wants to do this after the fact.
The simplest solution in Debian is to use native tools - apt-clone
will do exactly what is wanted, but is not available in bookworm,
although it is in sid and bullseye-backports, and can be used in
bookworm with some fiddling…

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

2 Likes
11

The easiest way is to rename your templates that you are using so that you can just download the default template without worrying about name conflicts.

For instance I use fedora-xx-xfce and I rename it to fedora-xx-ex (extended) before I start installing and modifying it. This way I can just download the default template if anything goes wrong.

1 Like
12

I also had this idea and did it with salt (instead of comparing package lists). Unfortunately, this isn’t particularly useful.

Every major distro release introduces plenty of changes, some of them to package names and some - to the way things should be installed (which packages do I install to achieve x? how do I configure?). I ended up reviewing my salt configuration every release just to make sure that everything will work the way I want once I demand it. So far every fedora release led to changes in my package lists.

Maybe I just use some weird programs, or am doing it wrong, idk :grin:

So, you might simplify the process by dropping the old template download part. Also instead of blindly installing manually installed packages, you could print yourself a list of changes for review.

Still, config management is better here, because it has comments. It really helps you remember the rationale behind more obscure changes.

@wmrom2, what do you think about a script(s) that turns current list of manually installed packages in a template into a salt formula? (or a bash script?)

1 Like
13

Perhaps,

I think the OP wants to start with a fresh, unused Template. Install some specfic software inside the Qube, and then at any time, renew to that Original Template after using if for awhile, which leaves off the User data inside Qube.

Template Qubes can be Cloned, and any of them can be “re-named”.

Settings for the Template Qube has commands. Or use the blue Qube, upper left, right click, “Open Qubes Manager” One can do the “Clone Command” by right clicking on the Template one wants, Leaving, how to move the personal data over to the correct software related to it.

Which brings up the question, how much hard drive is used up by this. And since there are some common areas of data Pools, can one Clone of a Template – infect another with Malware, or — surprises?

14

For completeness, in Fedora and other dnf-based Linux, I think it needs (from memory):

sudo dnf repoquery --userinstalled --qf "%{name}"

Which should just give a simple list of package names, usable as arguments to “sudo dnf install”.

This is what I use during fedora upgrades… but it often needs some manual adjustments to the list.
[Edit: I never tried apt in fedora, I am just assuming that Solène’s method will not work]

15

Indeed, I only tried for Debian based systems, your command should work to get a package list manually installed without version numbers, so that it can be reused in dnf install command line :+1:

1 Like
16

is there a typo in this?

i tried using it in an older fedora template, but changed apt to dnf and the output was nothing.

17

it’s not so much about having a fresh template

it’s that if fedora 42 has a new template, i want to type in a dom0 terminal

template-migrate fedora-41-xfce fedora-42-xfce

and then have all the software from fedora-41-xfce installed in 42 if it’s in the repo automatically, with commands done from a dom0 script.

18

My command won’t work for fedora, by the way, don’t type commands without understand what they do :wink:

19

This would apply whatever method was used - even if one simply kept a
paper record of packages.
That said, in my case, I almost never have faced this problem - it’s
just a matter of updating the base template to latest release. But I
work mainly with Debian templates.

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

1 Like
20

This is trivially done in salt.

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.