[Salt Help] How to Abstract Base Template Version in State (.sls) Files?

User Support General
,
1
To mods

This is purely a salt question applied to Qubes, and not specific to Qubes. I asked here because I’m configuring a Qubes system, but feel free to move or close as you see fit.

Currently, when using Salt to create a new template based on another or to manage that template like Fedora 43, I have several mentions to Fedora 43 specifically, and it would be nice if I only had to write the version once instead of multiple times.

Currently, I include a sls that includes the name of the template, which I would also like to generalize, and I clone the template. But I have over twenty templates that I have to manually or batch rename, and it’s error-prone and redundant. I also have to manage the templates themselves, as each has its own .sls and .top file to contend with as well.

Has anyone faced this problem before and found an elegant solution, ideally a .sls I can place in a ‘common’ directory and be done with it, updating that one file to update everything? I hate to be a beggar and chooser, and I’d love to hear any solution that I could improve, but the emphasis here is on elegant. I don’t want to trade one debt for another.

2

I put the base in a pillar - then I can pull that in to any state as
needed. Single point to update and universally available - you can use
this either to clone to new templates, or to run in place upgrades.

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

3

Thanks, I’ll look into that again. I’m currently not familiar with pillars. Are they like global variables? I’ve tried to start using them before, but I wasn’t able to get a clear picture from the salt docs. Are there examples in your unman/shaker GitHub repo?

4

That’s probably my best bet from what I can see, but do you know of any other solutions?

Please excuse my rant.

I’m not ungrateful for your help, but I am passing from frustration and into anger with Salt. It’s one claim to fame is that it abstracts configuration management to make it better and easier, but I’m increasingly frustrated with the feeling that I only traded one problem for another (worse?) problem. I get that Salt is complex by necessity since abstracting all these systems while keeping Salt’s feature set is bound to have complications, but as soon as you graduate from the most basic of configurations, salt is downright ugly, and in every way the YAML is worse and less readable than code because they’ve basically built a coding language on top of YAML. With Bash or Python, I could just make a variable however I want and import it. With pillar, I need to use an entirely different root structure, make sure to add it to the .top file, and write a novel to retrieve it.

My frustration with Salt is growing at an exponential rate compared to the amount of ‘features’ I use, and I’m almost ready to kiss the mouth of the man who invents something that’s not trading one issue for another of equal (or worse) trouble. The only reason I don’t leave Salt right now is because I’m holding out for the hope that I use it long enough to be good enough to negate the issue (i.e. ‘beware the man with one gun’). Maybe the problem is just me, but I’m still upset that something that’s supposed to make this simple has ended up costing me more time and energy now than every other part of this project combined.

Maybe Ansible is better? The readability still looks terrible, so I’m not too hopeful, but maybe that’s just because I’m used to Salt.

5

I find Salt to be simple to use, and extremely readable. Maybe I’m
blind to the issues (Ha)

You can use salt['pillar.get']('codename:latest') and apply that
across all your templates - that seems pretty straightforward to me, and
a very short story.

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

6

That’s better than I thought. I was reading that I had to make a file under /srv/pillar and include that file in the .top file under base, and then call it with {{ salt['pillar.get']('varname', 'fallback') }} according to the salt docs on pillar. To me, that’s insanely ugly for something I should just be able to include a file for at the top and have ‘varname’. Something like:

...
include:
 - path.to.variables # (.sls)

{{ variables.fedora }}: # The file and this prefix would probably be templates, i.e. {{ templates.fedora }}
  qvm.template_installed: []
...

Maybe I’m just getting overly frustrated because I can’t make it work the way I want it. You know how it is with programming sometimes, when you have a complex problem you just can’t make work. Maybe I’m just not being flexible enough.

Another example, if I want to clone a template I need to confirm that the source exists or install it (qvm.template_installed). I should just be able to do this:

...
clone-<name-of-template>:
  qvm.template_installed:
    - source-template
  qvm-clone:
    - name: name-of-template
    - source: source-template
...

instead, it fails because I can’t use the same module (qvm.) twice in the same id (‘clone-<name-of-template>’). because of this, I can’t have a file (say /srv/salt/common/require-templates.sls) with a qvm.template_installed for each base template, and just include it and require the id of the specific template I need.

...
include:
  - path.to.sls
...

The above executes all states in the included sls, which is contrary to expectation. It should wait for a state to be called, such as in a require statement.

Lastly, and one of my biggest issues, is that when using qvm.vm to manage/create qubes, you have to use - present: in order to create a missing qube, but if the qube already exists, it will not configure a qube. So, if you want a qube to be present or created and in a certain configuration, you must use duplicate information in present to create it and prefs to modify a created qube.

I’m pretty sure I’ve had other issues, but these are just the ones fresh in my memory. My frustration isn’t so much with the software not working per se, just what I see as not doing its job well by being too different/complicated/verbose/constraining.

1 Like
7

Can you give a particular example? I tried salt['pillar.get']('fedora:latest'), but that didn’t work.

8

Have you set that pillar item? If not, it wont work.

Set the pillar like this in os_versions/init.sls:

debian:
  stable: "13"
  oldstable: "12"

os_versions/init.top:

base:
  '*':
    - os_versions

Import in to the pillar, and refresh.
Then you access in a state file:

{% set debian_stable = salt['pillar.get']('debian:stable') %}

new_template:
  qvm.clone:
    - name: template-test
    - source: debian-{{ debian_stable }}-minimal

I dont think the process of populating the pillar is onerous, and it
gives you what you want - a single file to update, and all your states
will update accordingly.

Of course, there are other approaches:
You can access arguments by reading them in from source.
stable:

13

testprefs_read.sls:

{% set debian_stable = salt['file.read']('/srv/salt/test/stable') %}
 

new_template:
  qvm.clone:
    - name: template-test
    - source: debian-{{ debian_stable }}-minimal

Or, set in yaml - versions.yaml:

debian_stable: 13

testprefs_yaml.sls:

{% import_yaml  "test/versions.yaml" as args %}
 
new_template:
  qvm.clone:
    - name: template-test
    - source: debian-{{ args['debian_stable'] }}-minimal

It sounds as if you would prefer the single file approach. That’s fine
for you. Again, a single file to update and pull in to your states.
Choose whatever works best for you. You might like to create a
config_directory to hold these files to make a central point of reference
from which you can access data in your state files.

I never presume to speak for the Qubes team. When I comment in the Forum I speak for myself.
1 Like
9

I missed commenting on some of your other issues.

Do this -

...
clone-<name-of-template>_source:
  qvm.template_installed:
    - source-template

clone-<name-of-template>:
  qvm.clone:
    - name: name-of-template
    - source: source-template

Well that’s how include works
If you include a SLS file everything in it is added to the state run.
You can override this with an exclude statement to avoid some IDs from
being run, but the real answer is to refactor and simplify the file you
are including.
Issues like this almost always point to a need to simplify your SLS
files.

There’s an open issue about this.
The simplest work round is to separate the creation from the
configuration. Not a bad thing to do in any case: it simplifies the
state files and avoids duplications.

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.