Packaging Salt states / formulas for use in Qubes OS

Thanks @deeplow!

I only took a quick look, but that reference sure helps! It still requires to copy from a qube to dom0, but it builds an RPM package on the fly and that is something I’ll definitely look into!

I’ll also take a better look to how the RPM is used once in dom0. I’ve a gut feeling that importing it into dom0 from a package repository (vs copying it via a tarball) may allow for additional confidence that is what it pretends to be -I know there is a two-step process to update packages- but I haven’t looked enough into how updates work yet to be sure and I might well be wrong. This is a great first step, thank you for the pointer!

Also your link lead me to a way of tagging qubes for targetting by qubectl! :bulb::tada: That’s awesome, thanks again!

1 Like

Glad it helped! :slight_smile:

I’m in the process of posting my salt formulae to GitHub.
Building packages is simple, and I have an example of how to do this
for a Kali template.
I’ll add spec files for the other formulae too, and host some packages
on https://qubes.3isec.org - too late to do it now, so maybe tomorrow.

2 Likes

Thanks for the examples @unman. I’ve done some progress and have a proof-of-concept for building a package out of the Salt state (using Tito).

I think I’ve got a good understanding of what it would take to publish the RPM (and it seems relatively straightforward).

I’ll look further into checking integrity of the sources and signing the resulting RPM packages, and once the loop is complete I’ll look into turning the Salt state into a formula.

1 Like

Today I learned: split-GPG and RPM packaging play nice together in general. :bulb:
I took note of the corresponding RPM configuration because it seems generic enough.

From there, signing and verifying packages can be performed “the usual way”:

# sign both source package and regular package:
rpm --resign example-1.0.0-1.f32.src.rpm
rpm --resign example-1.0.0-1.fc32.noarch.rpm

# verify them:
rpm -K example-1.0.0-1.fc32.src.rpm
rpm -K example-1.0.0-1.fc32.noarch.rpm

Cross-referencing @unman’s examples:

1 Like

I’ll call this done for now. If anyone is interested in setting up a personal YUM/DNF repository for RPM packages, here is a starting point:

1 Like

I’ve merged that pull request, and got the verify / build / verify / sign loop complete. Things can certainly be refined, but I think the essential elements are there and I installed qubes-mgmt-salt-user-split-ssh-0.1.0-1.fc32.noarch.rpm in my dom0 :slightly_smiling_face: :tada:

Next step I believe is turning the Salt state into a formula (corresponding issue in GitHub).

If anyone want to join me figuring that out, please chime in, you’d be welcome!

2 Likes

Quick update: this is coming along nicely. As of qubes-mgmt-salt-user-split-ssh-0.2.2-1.fc32.noarch.rpm the Salt formula is behaving the way I expect, and I am satisfied with its structure.

For reference, the configuration file looks like this (and persists between updates):

# /srv/user_pillar/split-ssh/config.yaml
---
vaults:
  - name: ssh-vault
    template: fedora-32
    label: black
    mem: 400
    vcpus: 2
    autostart: True
clients:
  - name: work
    template: fedora-32
    label: blue
    mem: 400
    vcpus: 2
    autostart: False
  - name: another-ssh-client
    template: fedora-32
    label: blue
    mem: 400
    vcpus: 2
    autostsart: True

Beyond a few limitations (e.g. I haven’t looked into supporting other templates than fedora-based ones though it is certainly possible; I don’t personally need more than one vault and left support for multiple vaults incomplete) the formula covers what I believe are the most common uses cases.

There are a couple of things that I’d like to refine in the way the RPM packages / installation behave (e.g. the package currently includes a workaround for this issue that is not very elegant) but I think at this point the next step is writing some documentation.

If you’re able to follow along the code (Salt formula, RPM packaging, or RPM publishing, linked in earlier updates) and have thoughts on what would be most valuable to explain, please let me know here! :slightly_smiling_face:

1 Like

Referencing a similar thread about packaging Salt config as Debian packages from the mailing list. (I can’t perform the cross reference there from the forum.)

Documentation update:

I’ve also set up some context and outlined the missing sections before I take on the next guide.


Of course, feedback (well-intended and actionable) is welcome! I think the qubes-packages’ discussions are a good place for that.

2 Likes

Documentation update:

2 Likes

@deeplow if you’ve got time to skim through the tutorial linked above, I’d love to hear your thoughts! :slightly_smiling_face:

I’m thinking of structuring the generalization as a collection of how-to guides / code snippets based on the tutorial steps: how to create/ensure a qube exists, how to set up qube preferences, how to ensure packages are available in a qube, how to ensure presence of persistent files in a qube, how to ensure presence of non-persistent files in a qube, and so on.

That’s what I think would match more closely your suggestion of boilerplate config with examples:

One thing that could be particularly useful (I think) would be a boilerplate salt config with some examples.

1 Like

Hi!

Thank you very much for this work.
Kind of busy right now, but I was looking and I think I will finally learn the basics of Salt (from the docs you linked). It seems not too difficult and incredibly useful.
Just want to say, I think this is a good path.
Note: I am a intermediate level user, not too technical, not excessively comfortable with the cmd line.

Thanks

2 Likes

Not a lot at the moment. I’ll see what I can but. But I welcome the contribution very much and extend the offer for other people to provide feedback on it.

Sounds nice! I must also say that your “read-the-docs” looks much nicer than the default markdown files of the community documentation. But have you considered making your contributions there?

Or if you prefer to do on your website, maybe after you’re done you could open a pull request to add a link to that under “external documentation” over on the official docs.

I considered submitting this to the community docs, but at the moment I don’t have the bandwidth to write and manage the back and forth that publishing on third-party docs requires. I am very much figuring out the format as I go, and self-publishing gives me the room to iterate quickly.

I’ll take on your suggestion, though, and open a pull request to add a link or two under the “external documentation” section of the official docs when I’m done writing the how-tos. :slightly_smiling_face:

2 Likes

@gonzalo-bulnes just want to point to this. If you already knew it, please ignore this message.

regards

2 Likes

I hadn’t seem that. This is super useful @qubicrm thank you! :heart:

A quick update on this project, in case anyone is interested in taking part.

After packaging my Split-SSH and Split-GPG formulas, I stopped publishing for a long time. For various reasons; but one of them was that I was finding those formulas way too complex to be pleasant to work with (as a developer, no as an end-user).

It turns out that one idea that was brought up in this topic held water: targeting qubes through their tags, instead of their names. While I didn’t doubt it was a good idea, it is only relatively recently that I found a way to do it that I found satisfactory, thanks to walking into some code written by @brian.

I’m in the process of publishing two Salt formulas that enable QVM tags and QVM features to be used as pillar data, and I found out that:

  • The RPM packaging setup I did two years ago wasn’t bad. It’s actually pretty convenient. :nail_care:
  • Using QVM tags to target qubes alows to simplify significantly the formulas, while retaining the flexibility that I was trying to maintain at high cost. (Spoiler for those who’ve seen that code: the custom Salt pillar created out of a pile of YAML files and Jinja templates is entirely gone.)
  • I identify as a desirable pattern / boundary to leave the existence of the qubes (qvm.present) out of the scope of the formulas that configure them. You may want to create states that ensure the presence of qubes, and for example their tags, but I’d recommend keeping those separate from application-style formulas like split-ssh or @unman’s cacher.

Once that done, I’ll publish new versions of the Split-SSH and Split-GPG packages that have those packages as dependencies and take full advantage of the new targeting strategy (codename: version 2.0.0). I’ve been using them for a while now, and I think they might be interesting if you’re into creating your own Salt formulas. The RPM package-level dependency management is also something that I want to explore.

Last but not least, I’m currently working in parallel towards making the RPM packages I create reproducible. I share progress on GitHub if you’re interested. See the Reproducible RPM builds? issue.

As usual, once I’m done I hope to write some docs about those new processes. Keep tuned, and/or let me know what parts would be most interesting to start with.

4 Likes