Ansible in Qubes OS

fepitre · April 6, 2025, 2:55pm

Hi everyone,

I’m excited to announce that qubes-ansible is now available for testing! You can check it out at https://github.com/fepitre/qubes-ansible.

You have two options to get started:

Build it yourself using qubes-builderv2. Append to your builder.yml components list:

components:
  (...)
  - ansible:
      url: https://github.com/fepitre/qubes-ansible
      maintainers:
        # fepitre's @invisiblethingslab.com
        - 77EEEF6D0386962AEA8CF84A9B8273F80AC219E6

Install it via COPR repositories until we put packages into Qubes OS repositories. If you’re not sure how to enable the COPR repository, check out this guide.

I encourage you to use a management VM rather than installing it directly into dom0 (see instructions).

This work follows the one made at https://github.com/kushaldas/qubes_ansible.

For more details, please refer to the README and examples provided in the repository. There’s also another straightforward example available at https://github.com/fepitre/qubes-demo.

I have not management to test completely the whole features yet neither adding specific module and connection tests. Any help is welcome for that. Happy testing, and I’m looking forward to your feedback!

unman · April 7, 2025, 1:16am

Great work. I’m testing this now.

deeplow · April 7, 2025, 4:30pm

It is refreshing to see how much simpler this implementation is compared to its salt counterpart. And how much of a difference it makes to see real-time output. (Also kudos to @kushaldas for the initial work!)

I was able to successfully set this up as a management qubes. The only things that didn’t work were:

setting specific netvms (but this is a limitation with the Admin API and this is probably related)
Any policy stuff, of course. It may need dedicated “directives” that make use of the Policy Admin API.

I see that no intermediate <vm_name>-mgmt qube is created at the moment. While I’m very happy with it performance-wise, I think it should have an answer before production use (I guess this is at already being discussed at Discussing the security aspect of Ansible for Qubes OS · Issue #15 · kushaldas/qubes_ansible · GitHub).

I wonder if there is a way to simply pass all features to qubesadmin and have it do the checking if they are valid. This way it woudn’t need additional effort. But I am not fully sure if this is possible in ansible. I filed an issue (on the original fork) Delegate feature/preference checking to Qubes Admin. · Issue #35 · kushaldas/qubes_ansible · GitHub.

kushaldas · April 7, 2025, 6:26pm

I will start looking into things soon. Finally I have a proper system to do development/usage

brian · April 22, 2025, 10:18pm

I see that no intermediate <vm_name>-mgmt qube is created at the moment. While I’m very happy with it performance-wise, I think it should have an answer before production use (I guess this is at already being discussed at Discussing the security aspect of Ansible for Qubes OS · Issue #15 · kushaldas/qubes_ansible · GitHub).

When this is implemented, would it be possible to pipe each disposable mgmt qube’s output to a multi-file log viewer, like multitail?

deeplow · April 23, 2025, 5:57am

Should be doable to pass the execussion log back to the management qube in real time. What you do with those logs afterwards is up to you

ddevz · April 25, 2025, 8:14pm

Thanks for doing this!

When testing it out I found a issue with using roles.

First, a success case: Running ansible-playbook -i ./inventory create_qube.yaml does seem to work with the following create_qube.yaml contents:

- hosts: local
  connection: local
  tasks:
      - name: Create a test qube
        qubesos:
          guest: fedora-demo
          label: red
          state: present
          template: "fedora-40-xfce"

However, using roles for the same task does not seem to work, returning issues with temporary directories.

The execution and normal errors (without -vvvv) can be seen by unfolding this line

[user@mgmtvm ansible]$ ansible-playbook -i ./inventory ./qubes-system-playbook.yml 

PLAY [appvms] ******************************************************************

TASK [Gathering Facts] *********************************************************
fatal: [vault-demo]: UNREACHABLE! => {"changed": false, "msg": "Failed to create temporary directory. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \"` echo ~/.ansible/tmp `\"&& mkdir \"` echo ~/.ansible/tmp/ansible-tmp-1745610830.8092453-2762-19757050029218 `\" && echo ansible-tmp-1745610830.8092453-2762-19757050029218=\"` echo ~/.ansible/tmp/ansible-tmp-1745610830.8092453-2762-19757050029218 `\" ), exited with result 2", "unreachable": true}
fatal: [work-demo]: UNREACHABLE! => {"changed": false, "msg": "Failed to create temporary directory. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \"` echo ~/.ansible/tmp `\"&& mkdir \"` echo ~/.ansible/tmp/ansible-tmp-1745610831.3592-2763-157209519876765 `\" && echo ansible-tmp-1745610831.3592-2763-157209519876765=\"` echo ~/.ansible/tmp/ansible-tmp-1745610831.3592-2763-157209519876765 `\" ), exited with result 2", "unreachable": true}
fatal: [project-demo]: UNREACHABLE! => {"changed": false, "msg": "Failed to create temporary directory. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \"` echo ~/.ansible/tmp `\"&& mkdir \"` echo ~/.ansible/tmp/ansible-tmp-1745610832.4571009-2765-173191854429628 `\" && echo ansible-tmp-1745610832.4571009-2765-173191854429628=\"` echo ~/.ansible/tmp/ansible-tmp-1745610832.4571009-2765-173191854429628 `\" ), exited with result 2", "unreachable": true}
fatal: [admin-demo]: UNREACHABLE! => {"changed": false, "msg": "Failed to create temporary directory. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \"` echo ~/.ansible/tmp `\"&& mkdir \"` echo ~/.ansible/tmp/ansible-tmp-1745610832.4740686-2764-277546098508573 `\" && echo ansible-tmp-1745610832.4740686-2764-277546098508573=\"` echo ~/.ansible/tmp/ansible-tmp-1745610832.4740686-2764-277546098508573 `\" ), exited with result 2", "unreachable": true}

PLAY RECAP *********************************************************************
admin-demo                 : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   
project-demo               : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   
vault-demo                 : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0   
work-demo                  : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0

The full error message (with -vvvv) of the first qube can be seen by unfolding this line

TASK [Gathering Facts] ********************************************************* task path: /home/user/ansible/qubes-system-playbook.yml:2 CMD IS: /bin/sh -c 'echo ~user && sleep 0' CMD: /bin/sh -c 'echo ~user && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'vault-demo', b'qubes.VMShell'] <vault-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'vault-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c 'echo ~user && sleep 0' CMD: /bin/sh -c 'echo ~user && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'work-demo', b'qubes.VMShell'] <work-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'work-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c 'echo ~user && sleep 0' CMD: /bin/sh -c 'echo ~user && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'admin-demo', b'qubes.VMShell'] <admin-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'admin-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c 'echo ~user && sleep 0' CMD: /bin/sh -c 'echo ~user && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'project-demo', b'qubes.VMShell'] <project-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'project-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c 'echo "pwd" && sleep 0' CMD: /bin/sh -c 'echo "pwd" && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'vault-demo', b'qubes.VMShell'] <vault-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'vault-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c 'echo "pwd" && sleep 0' CMD: /bin/sh -c 'echo "pwd" && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'work-demo', b'qubes.VMShell'] <work-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'work-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c 'echo "pwd" && sleep 0' CMD: /bin/sh -c 'echo "pwd" && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'project-demo', b'qubes.VMShell'] <project-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'project-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c 'echo "pwd" && sleep 0' CMD: /bin/sh -c 'echo "pwd" && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'admin-demo', b'qubes.VMShell'] <admin-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'admin-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c '( umask 77 && mkdir -p " echo ~/.ansible/tmp "&& mkdir " echo ~/.ansible/tmp/ansible-tmp-1745612144.3724449-4766-688422035116 " && echo ansible-tmp-1745612144.3724449-4766-688422035116=" echo ~/.ansible/tmp/ansible-tmp-1745612144.3724449-4766-688422035116 " ) && sleep 0' CMD: /bin/sh -c '( umask 77 && mkdir -p " echo ~/.ansible/tmp "&& mkdir " echo ~/.ansible/tmp/ansible-tmp-1745612144.3724449-4766-688422035116 " && echo ansible-tmp-1745612144.3724449-4766-688422035116=" echo ~/.ansible/tmp/ansible-tmp-1745612144.3724449-4766-688422035116 " ) && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'vault-demo', b'qubes.VMShell'] <vault-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'vault-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c '( umask 77 && mkdir -p " echo ~/.ansible/tmp "&& mkdir " echo ~/.ansible/tmp/ansible-tmp-1745612144.9422057-4768-185072024464326 " && echo ansible-tmp-1745612144.9422057-4768-185072024464326=" echo ~/.ansible/tmp/ansible-tmp-1745612144.9422057-4768-185072024464326 " ) && sleep 0' CMD: /bin/sh -c '( umask 77 && mkdir -p " echo ~/.ansible/tmp "&& mkdir " echo ~/.ansible/tmp/ansible-tmp-1745612144.9422057-4768-185072024464326 " && echo ansible-tmp-1745612144.9422057-4768-185072024464326=" echo ~/.ansible/tmp/ansible-tmp-1745612144.9422057-4768-185072024464326 " ) && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'admin-demo', b'qubes.VMShell'] <admin-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'admin-demo', b'qubes.VMShell'] CMD IS: /bin/sh -c '( umask 77 && mkdir -p " echo ~/.ansible/tmp "&& mkdir " echo ~/.ansible/tmp/ansible-tmp-1745612146.0433779-4767-33120221380384 " && echo ansible-tmp-1745612146.0433779-4767-33120221380384=" echo ~/.ansible/tmp/ansible-tmp-1745612146.0433779-4767-33120221380384 " ) && sleep 0' CMD: /bin/sh -c '( umask 77 && mkdir -p " echo ~/.ansible/tmp "&& mkdir " echo ~/.ansible/tmp/ansible-tmp-1745612146.0433779-4767-33120221380384 " && echo ansible-tmp-1745612146.0433779-4767-33120221380384=" echo ~/.ansible/tmp/ansible-tmp-1745612146.0433779-4767-33120221380384 " ) && sleep 0' Local cmd: [b'qvm-run', b'--pass-io', b'--service', b'work-demo', b'qubes.VMShell'] <work-demo> RUN [b'qvm-run', b'--pass-io', b'--service', b'work-demo', b'qubes.VMShell'] fatal: [vault-demo]: UNREACHABLE! => { "changed": false, "msg": "Failed to create temporary directory. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \" echo ~/.ansible/tmp \"&& mkdir \" echo ~/.ansible/tmp/ansible-tmp-1745612144.3724449-4766-688422035116 \" && echo ansible-tmp-1745612144.3724449-4766-688422035116=\" echo ~/.ansible/tmp/ansible-tmp-1745612144.3724449-4766-688422035116 \" ), exited with result 2, stderr output: usage: qvm-run [--verbose] [--quiet] [--help] [--user USER] [--autostart]\n [--no-autostart] [--pass-io] [--localcmd COMMAND] [--gui]\n [--no-gui] [--colour-output COLOUR] [--colour-stderr COLOUR]\n [--no-colour-output] [--no-colour-stderr]\n [--filter-escape-chars] [--no-filter-escape-chars] [--service]\n [--no-shell] [--dispvm [BASE_APPVM] | --all]\n [--exclude EXCLUDE]\n [VMNAME] COMMAND ...\nqvm-run: error: no such domain: 'vault-demo'\n", "unreachable": true }

The file 'inventory' can be seen by unfolding this line

[local]
localhost

[local:vars]
ansible_connection=local

[appvms]
vault-demo
work-demo
admin-demo
project-demo

[appvms:vars]
ansible_connection=qubes
connection=local

[templatevms]
fedora-demo

[templatevms:vars]
ansible_connection=qubes
connection=local

The file qubes-system-playbook.yml:

- hosts: appvms
  connection: local
  roles:
    - role: qube-exists

The file roles/qube-exists/tasks/main.yml:

    - name: Create Qube
      qubesos:
        guest: { inventory_hostname }
        label: red
        state: present seems to not work
        template: "fedora-40-xfce"

Any ideas what could be causing this? To test the permissions, I tried setting the temp directory to ~/.ansible/tmp then chown -R 777 ~/.ansible/tmp , but no change. maybe it is trying to write the the temp directory in the non-existant device?