Automated Qubes OS Installation using Kickstart and/or PXE Network Boot

Community Guides
8

I have written some salt files to:

  1. Check to make sure that fedora-36-minimal is installed, and install it if it isn’t
  2. Clone it into sys-pxe-template (this is the bit I’m having trouble with)
  3. Change sys-pxe-template into a disposable VM template
  4. Install the following packages into sys-pxe-template:
  • nfs-server
  • tftp-server
  • dhcp-server
  • syslinux
  1. Put the following config files into sys-pxe-template:

  • /etc/dhcp/dhcpd.conf

    • Configure DHCP server with subnet of 192.168.100.0/24, and TFTP boot server

  • /var/lib/tftpboot/*

    • All the PXE files from syslinux
    • The pxelinux.cfg and custom grub2 EFI files from Qubes OS /boot directory (needed to multiboot Xen)

  • /usr/local/bin/download-and-verify-latest-qubes-iso

    • Bash script to download and verify the latest Qubes OS ISO

  • /etc/systemd/system/download-and-verify-latest-qubes-iso.service

    • Systemd service to automatically run download-and-verify-latest-qubes-iso when the Qube starts

  • /etc/systemd/network/20-all-interfaces.network

    • Set static IP address of sys-pxe to 192.168.100.1
    • Disable Qubes virtual NICs because they aren’t needed

All of these config files are actually written in plain text in the *.sls files for transparency and to keep the install size down.

  1. Create a disposable VM called sys-pxe, and base it on sys-pxe-template
  • All the user has to do is open this DispVM, and their ethernet NIC automatically becomes a Qubes OS PXE Network Boot Server!

These are the salt files:

sys-pxe.top 

# -*- coding: utf-8 -*-
# vim: set syntax=yaml ts=2 sw=2 sts=2 et :

# Installs 'sys-pxe' Qubes OS ISO Network Boot Server Qube.
#
# Pillar data will also be merged if available within the ``qvm`` pillar key:
#   ``qvm:sys-pxe``
#
# located in ``/srv/pillar/dom0/qvm/init.sls``
#
# Execute:
#   qubesctl top.enable qvm.sys-pxe
#   qubesctl --all state.highstate

{% if salt['pillar.get']('qvm:sys-pxe:name', 'sys-pxe') != salt['pillar.get']('qvm:sys-gui:name', 'sys-gui') %}
{% set vmname = salt['pillar.get']('qvm:sys-pxe:name', 'sys-pxe') %}
{% else %}
{% set vmname = salt['pillar.get']('qvm:sys-gui:name', 'sys-gui') %}
{% endif %}

base:
  dom0:
    - match: nodegroup
    - qvm.sys-pxe
  {{ salt['pillar.get']('qvm:sys-pxe:template', 'fedora-36-minimal') }}:
    - qvm.sys-pxe-template
  {{ vmname }}:
    - qvm.sys-pxe-vm

sys-pxe-template.sls 

# -*- coding: utf-8 -*-
# vim: set syntax=yaml ts=2 sw=2 sts=2 et :

##
# qvm.sys-pxe-template
# ====================
##

fedora-36-minimal:
  qvm.template_installed: []

#sys-pxe-template:
#  qvm.template_installed: []

#dom0:
#  cmd.run:
#    - qvm-clone fedora-36-minimal sys-pxe-template

sys-pxe-template:
  qvm.vm:
    - present:
      - label: black
    - prefs:
      - label: black
      - dispvm-allowed: True
    - features:
      - enable:
        - appmenus-dispvm

  pkg.installed:
    - pkgs:
      - qubes-core-admin-client
      - syslinux
      - tftp-server
      - nfs-server
      - dhcp-server
      - systemd-networkd

  /var/lib/tftpboot/pxelinux.cfg/default:
    file.managed:
      - user: root
      - mode: 644
      - makedirs: True
      - contents: |
          default vesamenu.c32
          timeout 100
          
          menu background splash.png
          
          label Qubes-Manual
          menu label ^Qubes OS 4.1.1 - Automated Install - Kickstart
          kernel mboot.c32
          append qubes/xen.gz console=none --- qubes/vmlinuz inst.stage2=nfs:192.168.100.1:/var/lib/tftpboot/qubes/iso/Qubes.iso plymouth.ignore-serial-consoles ip=dhcp inst.ks=nfs:192.168.100.1:/var/lib/tftpboot/qubes/iso/anaconda-ks.cfg i915.alpha_support=1 quiet rhgb --- qubes/initrd.img
          
          label Qubes-Manual 
          menu label ^Qubes OS 4.1.1 - Manual Install - Regular ISO Boot
          kernel mboot.c32
          append qubes/xen.gz console=none --- qubes/vmlinuz inst.stage2=nfs:192.168.100.1:/var/lib/tftpboot/qubes/iso/Qubes.iso plymouth.ignore-serial-consoles ip=dhcp i915.alpha_support=1 quiet rhgb --- qubes/initrd.img
          
          label local
          menu label Boot from ^Local Drive
          localboot 0xffff
  
  /var/lib/tftpboot/EFI/qubes/grub.cfg:
    file.managed:
      - user: root
      - mode: 644
      - makedirs: True
      - contents: |
          function load_video {
          	insmod vga
          	insmod vbe
          	insmod efi_gop
          	insmod efi_vga
          	insmod ieee1275_fb
          	insmod video_bochs
          	insmod video_cirrus
          	insmod all_video
          }
          
          load_env
          
          load_video
          set gfxpayload=keep
          insmod gzio
          
          insmod gfxterm
          insmod gfxtext


  /etc/systemd/network/20-all-interfaces.network:
    file.managed:
      - user: root
      - mode: 644
      - makedirs: True
      - contents: |
          # Match all ethernet NICs
          [Match]
          Type=ether
          # For some reason, this network device exists and causes TFTP to fail if it's active...
          Name=!enX0
          
          # Set Ethernet NIC static IP address of 192.168.100.1
          [Network]
          Address=192.168.100.1/24
          # May not be necessary, but including them just in case
          Gateway=192.168.100.1
          DNS=192.168.100.1

   /etc/dhcp/dhcpd.conf:
    file.managed:
      - user: root
      - mode: 644
      - makedirs: True
      - contents: |
          option arch code 93 = unsigned integer 16;
          # If you are planning to PXE install Qubes OS on more than 253 machines at the same time, you are absolutely insane, and I love it!
          # If you are, then you probably know how to edit this config file to expand the subnet...
          subnet 192.168.100.0 netmask 255.255.255.0 {
          authoritative;
          default-lease-time 600;
          max-lease-time 7200;
          ddns-update-style none;
          option domain-name-servers 192.168.100.1;
          option routers 192.168.100.1;
          option broadcast-address 192.168.100.255;
          option subnet-mask 255.255.255.0;
          range 192.168.100.2 192.168.100.254;
          if option arch = 00:07 {
          # amd64 UEFI
          filename "uefi/shimx64.efi";
          next-server 192.168.100.1;
          } else if option arch = 00:0b {
          # aarch64 UEFI
          filename "uefi/shimaa64.efi";
          server-name 192.168.100.1;
          } else {
          filename "pxelinux.0";
          next-server 192.168.100.1;
          }
          
          }


  /var/lib/tftpboot/EFI/qubes/themes/qubes/theme.txt:
    file.managed:
      - user: root
      - mode: 644
      - makedirs: True
      - contents: |
          # Copyright (C) 2016 Harald Sitter <sitter@kde.org>
          #
          # This program is free software; you can redistribute it and/or
          # modify it under the terms of the GNU General Public License as
          # published by the Free Software Foundation; either version 3 of
          # the License or any later version accepted by the membership of 
          # KDE e.V. (or its successor approved by the membership of KDE
          # e.V.), which shall act as a proxy defined in Section 14 of
          # version 3 of the license.
          #
          # This program is distributed in the hope that it will be useful,
          # but WITHOUT ANY WARRANTY; without even the implied warranty of
          # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
          # GNU General Public License for more details.
          #
          # You should have received a copy of the GNU General Public License
          # along with this program.  If not, see <http://www.gnu.org/licenses/>.
          
          # paperwhite - #fcfcfc
          # icongrey - #4d4d4d
          # plasmablue - #3daee9
          # black - #000000
          
          # Global Property
          # General settings
          title-text: ""
          title-font: "Unifont Regular 14"
          message-font: "Unifont Regular 14"
          message-color: "#7f8c8d"
          message-bg-color: "#4d4d4d" # TODO: whatever is this for?
          desktop-image: "qubes.png"
          
          # title
          # NOTE: can't put this in a vbox because GRUB is crap and item highlighting
          #   is broken if you put the boot_menu in a vbox...
          # TODO: file bug report
          + label {
              top = 50%-225 # (150+43+32) menu + height + spacer
              left = 0%
              width = 100%
              text = "Qubes OS Installer - Network Boot"
              align = "center"
              font = "Unifont Regular 32"
              color = "#ffffff"
          }
          
          # Show the boot menu
          + boot_menu {
              left = 50%-200
              width = 450
              # NB: this is scooped upwards from the middle.
              #     effectively 50px are below and the remaining 150 above
              top = 50%-150
              height = 200
              # Icon
              icon_width = 4
              icon_height = 0
              # Item
              item_height = 33
              item_padding = 1
              item_icon_space = 0
              item_spacing = 1
              item_font =  "Unifont Regular 16"
              item_color = "#4d4d4d"
              selected_item_font = "Unifont Bold 16"
              selected_item_color = "#ffffff"
          }
          
          + vbox {
              left = 50%-200 # same as menu
              top = 50%+113 # (50+16+19+28) half menu + spacer + progress + spacer
              width = 400 # same as menu
              + label { width = 400 align = "center" color = "#4d4d4d" font = "Unifont Regular 14" text = "[Enter] Boot the selected OS" }
              + label { width = 400 align = "center" color = "#4d4d4d" font = "Unifont Regular 14" text = "[Up and Down Key] navigation" }
              + label { width = 400 align = "center" color = "#4d4d4d" font = "Unifont Regular 14" text = "[E] Edit Selection" }
              + label { width = 400 align = "center" color = "#4d4d4d" font = "Unifont Regular 14" text = "[C] GRUB Command Line" }
          }
          
          # Show a styled horizontal progress bar
          + progress_bar {
              id = "__timeout__"
              left = 0
              top = 100%-32
              width = 100%
              height = 32
              show_text = false
              bar_style = "progress_bar_*.png"
              highlight_style = "progress_bar_hl_*.png"
          }
          
          # Show text progress bar
          + progress_bar {
              id = "__timeout__"
              left = 50%-200 # same as menu
              top = 50%+66 # (50+16) half menu + spacer
              width = 400 # same as menu
              height = 19 # 14pt
              show_text = true
              font = "Unifont Regular 14"
              text_color = "#4d4d4d"
              align = "center"
              text = "@TIMEOUT_NOTIFICATION_MIDDLE@"
              bar_style = "progress_bar2_*.png"
          }

  /etc/exports:
    file.managed:
      - user: root
      - mode: 644
      - makedirs: True
      - contents: |
          /var/lib/tftpboot/qubes/iso	*(rw,no_root_squash)






 
  cmd.run:
    - name: systemctl enable tftp.service
    - name: systemctl enable tftp.socket
    - name: systemctl enable systemd-networkd
    - name: exportfs -r
    - name: systemctl enable nfs-server
    - name: systemctl enable dhcpd
# TODO:  Add more stuff to put the syslinux files in /var/lib/tftpboot

sys-pxe-vm.sls (needs a bit of work)

# -*- coding: utf-8 -*-
# vim: set syntax=yaml ts=2 sw=2 sts=2 et :

##
# qvm.sys-pxe-vm
# ==============
##

# WIP: currently use default user 'user'
/var/lib/tftpboot/pxelinux.cfg/default:
  file.managed:
    - user: root
    - mode: 644
    - makedirs: True
    - contents: |
        default vesamenu.c32
        timeout 100
        
        menu background splash.png
        
        label Qubes-Manual
        menu label ^Qubes OS 4.1.1 - Automated Install - Kickstart
        kernel mboot.c32
        append qubes/xen.gz console=none --- qubes/vmlinuz inst.stage2=nfs:192.168.100.1:/var/lib/tftpboot/qubes/iso/Qubes.iso plymouth.ignore-serial-consoles ip=dhcp inst.ks=nfs:192.168.100.1:/var/lib/tftpboot/qubes/iso/anaconda-ks.cfg i915.alpha_support=1 quiet rhgb --- qubes/initrd.img
        
        label Qubes-Manual 
        menu label ^Qubes OS 4.1.1 - Manual Install - Regular ISO Boot
        kernel mboot.c32
        append qubes/xen.gz console=none --- qubes/vmlinuz inst.stage2=nfs:192.168.100.1:/var/lib/tftpboot/qubes/iso/Qubes.iso plymouth.ignore-serial-consoles ip=dhcp i915.alpha_support=1 quiet rhgb --- qubes/initrd.img
        
        label local
        menu label Boot from ^Local Drive
        localboot 0xffff

/var/lib/tftpboot/EFI/qubes/grub.cfg:
  file.managed:
    - user: root
    - mode: 644
    - makedirs: True
    - contents: |
        function load_video {
        	insmod vga
        	insmod vbe
        	insmod efi_gop
        	insmod efi_vga
        	insmod ieee1275_fb
        	insmod video_bochs
        	insmod video_cirrus
        	insmod all_video
        }
        
        load_env
        
        load_video
        set gfxpayload=keep
        insmod gzio
        
        insmod gfxterm
        insmod gfxtext
        
        terminal_output gfxterm
        insmod gfxmenu
        loadfont $prefix/themes/qubes/unifont-bold-16.pf2
        loadfont $prefix/themes/qubes/unifont-regular-14.pf2
        loadfont $prefix/themes/qubes/unifont-regular-16.pf2
        loadfont $prefix/themes/qubes/unifont-regular-32.pf2
        insmod png
        set theme=$prefix/themes/qubes/theme.txt
        export theme
        
        set timeout_style=menu
        set timeout=10
        
        menuentry 'Qubes OS 4.1.1 - Automated Install - Kickstart'  --class fedora --class gnu-linux --class gnu --class os {
        #	echo "To make a tasty Qubes OS..."
        #	echo "We start with the Xen Hypervisor..."
        	echo "Loading Xen Hypervisor..."
        	multiboot2 qubes/xen.gz console=none
        #	echo "...add a whole raw Linux Kernel..."
        	echo "Loading Linux Kernel..."
        	module2 qubes/vmlinuz ip=dhcp inst.stage2=nfs:192.168.100.1:/var/lib/tftpboot/qubes/iso/Qubes.iso plymouth.ignore-serial-consoles i915.alpha_support=1 quiet rhgb inst.ks=nfs:192.168.100.1:/var/lib/tftpboot/qubes/iso/anaconda-ks.cfg
        #	echo "...infuse it with your Kickstart Config, for extra flavoury goodness..."
        	echo "Adding Kickstart File...l"
        #	echo "...and finally, sprinkle it with a robust Initramfs..."
        	echo "Loading Initramfs..."
        	module2 qubes/initrd.img
        #	echo "...and we have a delicious Qubes OS.  Enjoy ;-)"
        }
        
         
        menuentry 'Qubes OS 4.1.1 - Manuak Install - Regular ISO Boot'  --class fedora --class gnu-linux --class gnu --class os {
        #	echo "To make a tasty Qubes OS..."
        #	echo "We start with the Xen Hypervisor..."
        	echo "Loading Xen Hypervisor..."
        	multiboot2 qubes/xen.gz console=none
        #	echo "...add a whole raw Linux Kernel..."
        	echo "Loading Linux Kernel..."
        	module2 qubes/vmlinuz ip=dhcp inst.stage2=nfs:192.168.100.1:/var/lib/tftpboot/qubes/iso/Qubes.iso plymouth.ignore-serial-consoles i915.alpha_support=1 quiet rhgb
        #	echo "...and finally, sprinkle it with a robust Initramfs..."
        	echo "Loading Initramfs..."
        	module2 qubes/initrd.img
        #	echo "...and we have a delicious Qubes OS.  Enjoy ;-)"
        }
        
        menuentry 'Exit this GRUB' {
        	exit
        }

/var/lib/tftpboot/uefi/BOOTX64.CSV:
  file.managed:
    - user: root
    - mode: 644
    - makedirs: True
    - contents: |
        shimx64.efi,Fedora,,This is the boot entry for Fedora
        grubx64.efi,Qubes,,This is the boot entry for Qubes OS
 
/usr/local/bin/download-and-verify-latest-qubes-iso:
  file.managed:
    - user: root
    - mode: 644
    - makedirs: True
    - contents: |
        #!/bin/bash
        # Download the Qubes OS 4.1.1 ISO
        gpg2 --import /etc/pki/rpm-gpg/RPM-GPG-KEY-qubes*
        gpg2 --import /usr/share/qubes/qubes-master-ket.asc
        wget https://ftp.qubes-os.org/iso/Qubes-R4.1.1-x86_64.iso.asc -O /var/lib/tftpboot/qubes/iso/Qubes-R4.1.1-x86_64.iso.asc &
        wget https://ftp.qubes-os.org/iso/Qubes-R4.1.1-x86_64.iso -O /var/lib/tftpboot/qubes/iso/Qubes-R4.1.1-x86_64.iso
        wget https://ftp.qubes-os.org/iso/Qubes-R4.1.1-x86_64.iso.DIGESTS -O /var/lib/tftpboot/qubes/iso/Qubes-R4.1.1-x86_64.iso.DIGESTS
        md5sum -c /var/lib/tftpboot/qubes/iso/Qubes-R4.1.1-x86_64.iso.DIGESTS
        sha1sum -c /var/lib/tftpboot/qubes/iso/Qubes-R4.1.1-x86_64.iso.DIGESTS
        sha256sum -c /var/lib/tftpboot/qubes/iso/Qubes-R4.1.1-x86_64.iso.DIGESTS
        sha512sum -c /var/lib/tftpboot/qubes/iso/Qubes-R4.1.1-x86_64.iso.DIGESTS
        gpg2 -v --verify /var/lib/tftpboot/qubes/iso/Qubes-R4.1.1-x86_64.iso.asc /var/lib/tftpboot/qubes/iso/Qubes-R4.1.1-x86_64.iso

/etc/systemd/system/download-and-verify-latest-qubes-iso:
  file.managed:
    - user: root
    - mode: 644
    - makedirs: True
    - contents: |
        [Unit]
        Description=Download & Verify Latest Qubes OS ISO from ftp.qubes-os.org
        [Service]
        Type=oneshot
        ExecStart=/usr/local/bin/download-and-verify-latest-qubes-iso
        RemainAfterExit=yes
        
        [Install]
        WantedBy=multi-user.target

sys-pxe.sls 

# -*- coding: utf-8 -*-
# vim: set syntax=yaml ts=2 sw=2 sts=2 et :

##
# qvm.sys-pxe
# ===========
#

fedora-36-minimal:
  qvm.template_installed: []

#sys-pxe-template:
#  qvm.template_installed: []

{% from "qvm/template.jinja" import load -%}

{% if salt['pillar.get']('qvm:sys-pxe:name', 'sys-pxe') != salt['pillar.get']('qvm:sys-gui:name', 'sys-gui') %}

{% set vmname = salt['pillar.get']('qvm:sys-pxe:name', 'sys-pxe') %}

{% load_yaml as defaults -%}
name:          sys-pxe
qvm.present:
  - name: sys-pxe
  - label:     red
  - mem:       400
prefs:
  # Uncomment if you want additional repos via HTTP to be avaialble
  #  - netvm:     "sys-firewall"
  # Default - No access to the internet
  - netvm:     ""
  - virt_mode: hvm
  - autostart: true
  - pci_strictreset: false
  - pcidevs:   {{ salt['grains.get']('pci_net_devs', [])|yaml }}
  - class:     DispVM
  - template:  sys-pxe-template

{%- endload %}

{{ load(defaults) }}

{% else %}

{% set vmname = salt['pillar.get']('qvm:sys-gui:name', 'sys-gui') %}

{{ vmname }}-pxe:
  qvm.prefs:
    - name: {{ vmname }}
    - virt_mode: hvm
    - pcidevs:   {{ salt['grains.get']('pci_net_devs', [])|yaml }}
    - pci_strictreset: false

{% endif %}

Is there any chance anyone could have a look at them and help me fix them?

Thank you so much in advance :slight_smile:

1 Like