Circular reference in salt ? [also: salt is pain]

[appih5587]

I’m currently trying to make my Qubes setup reproducible using salt.

Before I get to my current issue, I should point out that as a new user, salt has been nothing but a very painful experience so far. I’m not saying this to rant against salt - I just think first-hand feedback and experience from new users might be of interest. I’ve spent the last 10 hours wading through forum posts, scattered docs, github repositories, asking AI assistants - all to perform a seemingly very basic task, to absolutely no avail.
Here’s how I would sum it up in a nutshell: if I want to reproduce something that someone has already done (e.g. follow exactly a tutorial, copy a state from someone else), then Salt works fine - and once it works, it’s great. But if I want to create something a bit different, and the first try does not work, then it’s hell understanding why it doesn’t work, what the canonical way to do it is, and what the potential error message is even trying to tell me.

My actual issue:

To reduce tech debt and code duplication (and preserve my sanity), I needed a way to easily create a VM from a template from a base template.

Since afaik states can’t accept arguments, I created a macro.

# Creates[if not existing] a template from a base[will be pulled if necessary]
{% macro ensure_template_from_base(tpl_name, tpl_base, color, ns) %}
{{tpl_name}}-ensure-base-template:
  qvm.template_installed:
    - name: {{tpl_base}}
    {% if ns.require %}
    - require:
      - {{ns.require}}
    {% endif %}

{{tpl_name}}-create-template:
  qvm.clone:
    - name: {{tpl_name}}
    - source: {{tpl_base}}
    - require:
      - qvm: {{tpl_name}}-ensure-base-template

{{tpl_name}}-configure-template:
  qvm.prefs:
    - name: {{tpl_name}}
    - label: {{color}}
    - require:
      - qvm: {{tpl_name}}-create-template

{% set ns.require = 'qvm: ' ~ tpl_name ~ '-configure-template' %}
{% endmacro %}

# Creates[if not existing] a template+vm from a base[will be pulled if necessary]
{% macro ensure_template_and_vm_from_base(tpl_name, vm_name, tpl_base, color, ns) %}
{{ ensure_template_from_base(tpl_name, tpl_base, color, ns) }}

{{vm_name}}:
  qvm.vm:
    - present:
      - template: {{tpl_name}}
      - label: {{color}}
    - prefs:
      - template: {{tpl_name}}
      - label: {{color}}
  require:
    - {{ns.require}}

{% set ns.require = 'qvm: ' ~ vm_name %}
{% endmacro %}

A few side-comments:

• I didn’t find any canonical way to combine macros with require, e.g. to make a state require the “last” state in a macro without either hardcoding the last state’s name (), or passing a namespace ns to every single macro, which greatly adds to the boilerplate.
• The hard-requirement that state IDs be unique is a huge pain. I have every reason to want to create a “local” state e.g. configure-vm-prefs in different macros, which sets a given subset of the vm’s prefs. In any other language this would be a breeze, I feel like I’m missing some piece of the puzzle here. Appending the vm’s name to the state, a painful “solution”, is not even enough - I may want the macro to take a subset of the prefs only, and then name management becomes a nightmare.

This macro is then called e.g. from

{% from 'custom_salt/lib.sls' import ensure_template_and_vm_from_base %}
{% set ns = namespace(require=None) %}

# ---- Config
{% set tpl_base = 'debian-13-xfce' %}
{% set vm_name = tpl_base ~ '-gpu' %}
{% set tpl_name = vm_name ~ '-t' %}
{% set color = 'green' %}
{% set pci_list = ['00_01.0-00_00.0', '00_01.0-00_00.1'] %}
# ----------------

{% if grains['id'] == 'dom0' %}

{{ ensure_template_and_vm_from_base(tpl_name, vm_name, tpl_base, color, ns) }}

configure-gpu-template:
  qvm.prefs:
    - name: {{tpl_name}}
    - vcpus: 6
    - memory: 4000
    - maxmem: 0
    - virt_mode: hvm
    - kernel: ''
    - require:
      - qvm: {{tpl_name}}

{{vm_name}}-prefs:
  qvm.prefs:
    - name: {{vm_name}}
    - memory: 16000
    - maxmem: 0
    - pcidevs: {{ pci_list | json }}
    - require:
      - qvm: {{vm_name}}

{% elif grains['id'] == tpl_name %}

enable-non-free:
  cmd.run:
    - name: sed -i 's/non-free-firmware/& non-free/' /etc/apt/sources.list
    - unless: grep -q "non-free-firmware non-free" /etc/apt/sources.list

nvidia-package-install:
  pkg.installed:
    - pkgs:
      - linux-headers-amd64
      - nvidia-open-kernel-dkms
      - nvidia-driver
    - require:
      - cmd: enable-non-free

On this setup, qubesctl state.highstate will fail with

...
----------
          ID: configure-gpu-template
    Function: qvm.prefs
        Name: debian-13-xfce-gpu-t
      Result: False
     Comment: Recursive requisite found
     Changes:   
----------
          ID: debian-13-xfce-gpu-prefs
    Function: qvm.prefs
        Name: debian-13-xfce-gpu
      Result: False
     Comment: Recursive requisite found
     Changes:   
...

I have no clue where the requisite is recursive: both states simply require that their VM/template exists.

From my understanding, I’m asking for an extremely basic requirement flow: create a template, create a vm, and update their prefs once they’re respectively created. (Clearly, the lack of any detail in the error message doesn’t help either…)

Note: (I think it’s important to insist on that point): I’m not trying to blame anything/anyone on this - it’s quite likely that the problem here lies between the chair and the computer - but I consider myself decently proficient, and have spent (I think) quite some time trying to understand what is going on to no avail - if this happens to me, it likely happens in similar fashion to other users who will just get discouraged and give up on salt. I’m writing what I encountered to 1) (major) share my experience as a new user, which regular users of this forum might not be aware of and 2) (minor) to find a solution to my technical problem.

Thank you

[otter2]

Salt sucks (and ansible does too)

@appih5587

To reduce tech debt and code duplication (and preserve my sanity), I needed a way to easily create a VM from a template from a base template.

I haven’t looked into your issue, but salt generally is not designed to work in a dynamic way - if you require passing arguments, consider shell or another scripting language. Otherwise you should try using pillar instead.

@appih5587

I didn’t find any canonical way to combine macros with require, e.g. to make a state require the “last” state in a macro without either hardcoding the last state’s name (), or passing a namespace ns to every single macro, which greatly adds to the boilerplate.

You could try require_in from the macro instead, with some jinja magic if you want to make it situational. Honestly I don’t see that big of an issue with hard coding macro’s ID - not like you are going to change it. Especially if it is assembled by jinja to keep IDs unique.

@appih5587

Appending the vm’s name to the state, a painful “solution”, is not even enough - I may want the macro to take a subset of the prefs only, and then name management becomes a nightmare.

How you end up with a vm configuring macro that doesn’t have vm name in its namespace? I think I don’t understand the second part well enough.

[otter2]

I can’t really make sense of your salt so far.

In configure-gpu-template you require state qvm: {{tpl_name}} to complete first, as far as I can tell it doesn’t exist.

Another problem I see is the existence of ensure_template_and_vm_from_base in the first place - there is little benefit from hiding so little in a macro. Might as well just add this to {{vm_name}}-prefs.

I’ve ran a test, what you are trying to do in the macro does work. Here is my test:

test.sls

{% set ns = namespace(test = None) %}

{% macro nested(ns) %}
{% if ns.test %}
dingus:
  cmd.run:
    - name: echo 'nested {{ ns.test }}'
    - require: {{ ns.test }}
{% else %}
dingus:
  cmd.run:
    - name: echo "nested ns.test isn't set"
  {% endif %}
{% set ns.test = 'dingus' %}
{% endmacro %}

{% macro test_my_shit(ns) %}
{{ nested(ns) }}

{% if ns.test %}
dorkus:
  cmd.run:
    - name: echo {{ ns.test }}
{% else %}
dorkus:
  cmd.run:
    - name: echo "ns.test isn't set"
{% endif %}
    - require:
      - cmd: {{ ns.test }}
{% set ns.test = 'dorkus' %}
{% endmacro %}

{{ test_my_shit(ns) }}

chongus:
  cmd.run:
    - name: echo {{ ns.test }}
    - require: 
      - cmd: {{ ns.test }}

[unman]

If the formatting here is off can some kind soul fix it for me?

Hi @appih5587

Sorry to hear you are having difficulties with Salt. From my experience
of using Salt and promoting and teaching users how to work with it, it’s
often straight forward and simpler than you think. I think this is
particularly the case in Qubes, where users can be quite productive with
very little knowledge and experience.
For me, the Tao of Python relates directly to Salt:

Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.

Particularly the readability.

Now that’s out of the way, (and I can hear your eyes rolling), lets dig
in to your post.

Arguments:

Can states accept arguments? Of course. But I guess you actually mean,
can state files accept arguments, and the answer here is also, “of
course.”

Pillar data

The canonical way of passing arguments in to a state file is using a
pillar. You define variables in the pillar and access then within a
state.
If you look in my shaker, you’ll see that when installing the cacher
qube we create a pillar item to hold a value for update_proxy. Then we
can use this from other states to see if we need to rewrite repos
definitions by including:

{% if salt['pillar.get']('update_proxy:caching') %}
{% set proxy = 'cacher' %}
{% endif %}

Note that the pillar does not need to be permanent - you can also define
them on the fly and access them at the command line. Here’s an example -
test.testprefs_pillar.sls:

# qubesctl state.apply test.testprefs pillar='{"name": "test"}'
 
test-prefs:
  qvm.prefs:
    - name: {{ pillar['name'] }}
    - memory: 800
    - maxmem: 8000

From files

Naturally, there are other approaches. You can access arguments by
reading them in from source.
args:

test

testprefs_read.sls:

{% set args = salt['file.read']('/srv/salt/test/args') %}
 
test-prefs:
  qvm.prefs:
    - name: {{ args }}
    - memory: 800
    - maxmem: 8000

Importing yaml

Or you can import them args.yaml :

name: test

testprefs_yaml.sls:

{% import_yaml  "test/args.yaml" as args %}
 
test-prefs:
  qvm.prefs:
    - name: {{ args['name'] }}
    - memory: 800
    - maxmem: 8000

Problem with recursion

The output shows that the error is in the qvm.prefs function:

{{vm_name}}-prefs:
  qvm.prefs:
    - name: {{vm_name}}
    - require:
      - qvm: {{vm_name}}

What I think is wrong here is that reference to the qvm module, which
itself contains qvm.prefs. I think that is the circularity.

A much simpler approach is to use create.sls:

test_create:
  qvm.present:
    - name: test
    - template: debian-13
    - label: purple
    - prefs:
      - default_dispvm: deb-dvm

And then require it like this, testprefs_require.sls:

include:
  - test.create
 
test-prefs:
  qvm.prefs:
    - name: test
    - memory: 800
    - maxmem: 8000
    - require:
      - sls: test.create

Or, and so much easier, test for qvm.exists

I would probably set prefs at the time of creation, rather than
create, and then set prefs, but either approach is workable. Sometimes
it’s better to break things down.

The hard-requirement that state IDs be unique is a huge pain.

Yes it is, but you get used to it, and it helps with organisation and
troubleshooting.

I hope this is somewhat helpful. Remember that you can follow the flow
by appending -l debug to the salt call.

I never presume to speak for the Qubes team. When I comment in the Forum I speak for myself.

[whoami]

@appih5587

I should point out that as a new user, salt has been nothing but a very painful experience so far.

Feel you.

Same for me, I would love to have my auto qube creation script based on Salt but Bash was the only option for me to realize my objectives.