Automated Qubes OS Installation using Kickstart and/or PXE Network Boot

[alzer89]

@qstateless

I see. If PXE is limited to serve one client computer at a time, then I would just try duplicating the PXE software setup many times on the PXE server, running each PXE server instance in a separate qube with a different LAN IP on the network. So PXE-1 serves Client-1, PXE-2 serves Client-2, PXE-3 serves Client-3, etc.

Hahaha. No no no no no. My explanations are becoming more and more abiguous. My bad

I meant the initial loading of the xen.gz, vmlinuz and initrd.img files. The PXE server seems to not be able to serve those files in parallel to multiple machines simultaneously.

Once you serve the ISO over another protocol (I used NFS), there’s no bottleneck whatsoever

At least, that’s what I encountered in my testing of getting 6 legacy boot laptops and 6 UEFI boot laptops to PXE boot the Qubes ISO simultaneously…

Maybe there’s a config I haven’t set up that allows this…maybe…

@qstateless

Benefits:

• Prevents drive firmware attacks / drive firmware persistence on the client computers.

I would respectfully argue that these attacks are just moved to a different place…

@qstateless

Reduces the storage drives footprint down to my PXE server and my file server.

Again, you’d still have the same drive requirements, but they would be in a different place. Instead of being locally connected to the target machine, they’d be inside another machine.

This means you’d have to accommodate for TWO attack vectors:

• Firmware attacks on TWO devices:
  • The drive that is hosting your Qubes OS boot image, as well as the actual machine that this drive is connected to
  • Your network stack on both your client and server, as well as any intermediary network infrastructure

Many would argue that this is more cumbersome than connecting a physical drive to a SATA/NVME/PCI slot on a motherboard, but it appears tht this approach better suits your use case.

Just be sure to factor all of this in when you use it

@qstateless

Enables greater anti-forensics on the client computers (can be important even for common situations like civil lawsuits where everyday lawyers try to subpoena your drives, or hackers plant stuff on them remotely and SWAT you, etc).

Ok, I’ll give you that one

@qstateless

Less drives to manage and worry about as potential liabilities with something nasty sitting on them that I can’t conveniently audit.

Another good point

@qstateless

Reduces the storage drives footprint down to my PXE server and my file server, rather than also having dozens of endpoint computers containing storage drives.

Can be a blessing and a curse simultaneously, but I’ll give you that one too

@qstateless

Adds convenience of quickly wiping to a known clean OS state on client computers, which can then easily be done many times per day, compared to a major pain that doesn’t hardly ever get done otherwise.

Assuming the “known good state” Qubes OS install that you’re booting from stays in “mint condition”, but yes, I’ll give you that one too

@qstateless

Like having DispVM capability for the entire Qubes OS running on your machine. Just reboot to a fresh read-only instance of RAM-based Qubes OS provided from the PXE server. 99.9% Trustworthy as clean. Would never reinstall Qubes OS multiple times per day like this to a known clean state, but network booting could make this easy.

Was about to say this, but you beat me to it

@qstateless

Having 2 or 3 network cards per client computer might be necessary, where one NIC is sacrificed for the PXE network boot of the Qubes OS, which is okay for me.

There is potential for VLANs to alleviate some of these issues, allowing you to use a single NIC, but I’d need to investigate a lot deeper into this than I already have…

I don’t want to give you any wrong information

@qstateless

However, if a RAM-based Qubes is entirely loaded into the client’s RAM first, before Xen/Qubes Dom0 boots up, then maybe the connection to the PXE server can be cutoff once the Qubes OS is loaded into client RAM and the NIC could be used normally by Qubes in a sys-net?

That would definitely be possible, but if you were to do that to a current Qubes OS install without any modifications, do you realise how much RAM you’d actually need…?

The dom0 partition in the current standard Qubes OS install is at least 24GB.
That means:

• You’d lose at least 24GB of the client’s RAM to a dom0 ramdisk
  • This would only really be viable on server motherboards with a billion and one RAM slots (mad respect if you have such a motherboard, I’m super jealous!)
  • Your options for doing this are basically including a complete dom0 in the initrd.img, essentially making the initramfs 25GB, which would likely take 30-90+ minutes to serve via PXE
• Copying dom0 into RAM via NFS would be quicker, but would require a fair amount of “hacky shenanigans” to accomplish
  • Most of the work has been done with what you’ve already tinkered with regarding running Qubes OS entirely in RAM

@qstateless

Yes! I am game with you for achieving this setup! I have several duplicate machines ready to go for pursuing this setup.

Excellent.

@qstateless

Presently, with Qubes in tmpfs, I have made custom scripts that restore persistent AppVM configuration and personal file access, upon startup of Dom0 and AppVMs.

Ok, so then that’s a potential methodology for PXE.

@qstateless

If need be, I would just try duplicating the PXE software setup many times on the PXE server, running each PXE server instance in a separate qube with a different LAN IP on the network. So PXE-1 serves Client-1, PXE-2 serves Client-2, PXE-3 serves Client-3, etc.

Also, I could have the PXE server write-protect or, or if need be, just overwrite the Qubes OS install regularly.

Through a combination of tactics, it seems feasible to create a RAM-based Qubes OS PXE boot setup, for multiple client computers, that does not have to change with each reboot.

You’re starting to get the hang of PXE. Nice

@qstateless

Nooooo way. Wouldn’t let that happen. Their machines can’t touch the file server shares my work VM contents is stored upon. Physically isolated network. Also, their own AppVMs would have separate credentials (per PC and per VM) to access different shares on the file server, depending upon who’s specific machine and what specific AppVM is accessing the file server.

So then I guess you’ve decided that each machine gets their own Qubes OS install on the server

@qstateless

what do you think is the best way to make a RAM-based Qubes OS ready to use in sys-pxe?

Off the top of my head, I would say:

1. PXE serve xen.gz, vmlinuz, and initrd.img

• Have the initrd.img contain a complete read-only dom0

OR

• Have a dom0 served via NFS as an image file once the initrd.img was served via PXE, allowing faster boot times of client machines, particularly if multiple clients are booting simultaneously

2. Have Templates served via NFS

• Allows on-demand loading/unloading of templates and AppVMs
• Templates can be loaded by multiple clients simultaneously, provided they are read-only, avoiding clients “fighting over control of the templates”

3. Serve AppVMs (or at least configuration files for AppVMs) via NFS

• Allows each client to have their own persistent configuration

4. Serve some AppVMs via VNC

• Allows some AppVMs to run on the server
• Can be useful when running Qubes OS on a very underpowered client, and you need those extra system resources

5. Serve a user database via LDAP or PAM

• Would allow all clients to boot from the same Qubes install, log in with their unique username and password, and have all their configs and custom AppVMs load on any client machine
• Optional (and is likely not as straightforward as I am describing it, so don’t get your hopes up just yet )

---

But in any case, you are definitely onto something, and I propose forking this thread to better facilitate this endeavour