Install Qubes OS with boot partition and a detached LUKS header on USB

Boot into Qubes OS installer and on GUI screen switch to shell on another TTY by pressing Ctrl+Alt+F2.

Assuming:
/dev/sda - disk where you want to install Qubes OS
/dev/sdb - USB disk where you want to install Qubes OS boot partition

Find out the right disk names on your machine in Qubes OS installer shell by running this command and checking the output:

fdisk -l

In this shell run these commands:

dd if=/dev/zero of=header.img bs=16M count=1
cryptsetup luksFormat /dev/sda -c aes-xts-plain64 -h sha512 -s 512 -y -i 10000 --use-random --force-password --header header.img
cryptsetup open --header header.img /dev/sda luks
mkfs.btrfs --csum xxhash -L qubes_dom0 -d single /dev/mapper/luks

Return to Qubes OS installer GUI by pressing Ctrl+Alt+F6.
Configure the installation as normal except for Installation Destination:
Installation guide | Qubes OS
At the Installation Destination screen click on “Refresh…” at the bottom right corner and in the opened window press on “Disk rescan” button and then OK.
At the Device Selection choose your Qubes OS installation destination and boot partition destination disks /dev/sda and /dev/sdb.
At the Storage Configuration select “Advanced Custom (Blivet-GUI)”.
Press Done.
In the “BLIVET GUI PARTITIONING” screen select “sdb” disk.
Delete the old partitions on the disk if needed.

Add new partition using “+” button:
Device type: Partition
Size: 512 MiB
Filesystem: EFI System
Mountpoint: /boot/efi
Press OK button.

Add new partition using “+” button:
Device type: Partition
Size: 1 GiB
Filesystem: ext4
Mountpoint: /boot
Press OK button.

In the “BLIVET GUI PARTITIONING” screen select “qubes_dom0” Btrfs Volume.
Add new subvolume using “+” button:
Name: root
Mountpoint: /
Press OK button.

Press on Done button in “BLIVET GUI PARTITIONING” and then “Accept Changes” button in “SUMMARY OF CHANGES” window.
Press “Begin Installation” button.

After installation is completed don’t press “Reboot System” button.
Switch to shell on another TTY by pressing Ctrl+Alt+F2.
Run these commands in the installer shell:

cp header.img /mnt/sysroot/root/
cd /mnt/sysroot
mount -t proc /proc proc/
mount -t sysfs /sys sys/
mount --rbind /dev dev/
chroot /mnt/sysroot
btrfs subvolume create /swap
btrfs filesystem mkswapfile --size=4g --uuid clear /swap/swapfile

Edit /etc/fstab file using nano or any other text editor:

nano /etc/fstab

Add noauto option to the /boot and /boot/efi mounts like this:

UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX /boot                   ext4    defaults,discard,noauto 1 2
UUID=XXXX-XXXX          /boot/efi               vfat    umask=0077,shortname=winnt,discard,noauto 0 2

And add this line at the end:

/swap/swapfile none swap defaults,discard=once 0 0

Save and close /etc/fstab file.
Edit /etc/default/grub file using nano or any other text editor:

nano /etc/default/grub

Remove rd.luks.uuid=XXXXX option and add rd.driver.pre=btrfs option to the GRUB_CMDLINE_LINUX variable. It should look like this:

GRUB_CMDLINE_LINUX="rd.driver.pre=btrfs plymouth.ignore-serial-consoles 6.7.7-1.qubes.fc37.x86_64 x86_64 rhgb quiet"

Save and close /etc/default/grub file.
Create new /etc/dracut.conf.d/crypt.conf file using nano or any other text editor:

nano /etc/dracut.conf.d/crypt.conf

Add this text inside:

add_dracutmodules+=" crypt “
install_items+=" /root/header.img “

Save and close /etc/dracut.conf.d/crypt.conf file.
Find out the ID of your Qubes OS destination disk /dev/sda:

ls -la /dev/disk/by-id/ | grep /sda

For example it’ll look like this:

lrwxrwxrwx 1 root root   9 May 10 20:15 ata-YOUR_DISK_ID -> ../../sda

Here ata-YOUR_DISK_ID is your disk ID.

Edit /etc/crypttab file using nano or any other text editor:

nano /etc/crypttab

Add this text inside:

luks /dev/disk/by-id/ata-YOUR_DISK_ID none header=/root/header.img,force,discard

Save and close /etc/crypttab file.
Run these commands:

grub2-mkconfig -o /boot/grub2/grub.cfg
dracut -f --regenerate-all
exit

Return to Qubes OS installer GUI by pressing Ctrl+Alt+F6.
Press on “Reboot System” button.
After this proceed with normal Qubes OS post-installation process:
Installation guide | Qubes OS

Since /boot and /boot/efi partitions are stored on USB disk then you’ll need to attach this disk to your dom0 when doing dom0 updates so the files there will be updated.
It’s better to use disposable sys-usb for this setup.
When you want to update your dom0 you can follow these steps:
Disconnect all your USB devices.
Restart sys-usb to clear it’s state.
Connect your USB disk with Qubes OS /boot partition.
Run this command in dom0 to mount the /boot and /boot/efi partitions in dom0 assuming that /dev/sda is the name of your USB disk with Qubes OS /boot partition in sys-usb:

qvm-block attach dom0 sys-usb:sda1 && qvm-block attach dom0 sys-usb:sda2 && sudo mount /boot && sudo mount /boot/efi

Run dom0 update.
After dom0 update is finished run this command in dom0 to unmount and remove /boot and /boot/efi partitions from dom0:

sudo umount /boot/efi /boot && qvm-block d dom0 sys-usb:sda1 && qvm-block d dom0 sys-usb:sda2

At this point you can disconnect the USB disk with Qubes OS /boot partition from your machine and continue to use sys-usb with other USB devices as normal.

3 Likes

Assuming the installer and the boot process itself have no USB protection (like sys-usb or USBGuard), how does one trust a USB device with complex proprietary firmware (such as USB storage) at install or boot time? I.e. how and why does one trust such a device at such early and vulnerable phase? I don’t dare to connect even the USB port to the UPS :slight_smile:

You can use USB device if it’s OK for your threat model.
USB device that you use to boot the Qubes OS from should be assumed as trusted.
The same way that USB device containing the Qubes OS installer that is used to install the Qubes OS should be assumed as trusted.
Installation security | Qubes OS
Instead of using USB device to boot Qubes OS from it could be considered to use the SD card reader (if available in your machine) and SD card to boot from. There is no firmware in SD cards (AFAIK).

I was wrong, there is a firmware there as well:
https://www.bunniestudios.com/blog/2013/on-hacking-microsd-cards/

So I guess if you use a separate PCI USB controller or PCI SD card reader only to boot from your boot device then it’ll be the same as booting from SATA/NVMe drive.
You’ll have to trust that your boot device firmware is not malicious the same way that you’ll need to trust that your SATA/NVMe drive firmware is not malicious. And that both your boot device and SATA/NVMe drive couldn’t be physically accessed by attacker.

Isn’t the point that with one’s USB goes access to the physical device? (Assuming one carries the USB once shutting down the device and walking away.)

There is always a chance that attacker can access it in certain situations (like during border control search or when you sleep). The same goes for attacker accessing the machine.
Also USB device firmware could be malicious even before it’ll end up in your hands. But the same goes for any other hardware.

I think the main issue with USB storage specifically is that it can behave as other devices like USB keyboard.
On the other hand even if e.g. SD card has malicious firmware then it still can’t be used like a keyboard. It only can be used to further exploit the vulnerability in the SD card reader to gain further access of your system.

@apparatus

I know the things you mention but they still don’t answer the actual question. However, since it requires a more in-depth discussion, it would better fit a separate thread :slight_smile:

I’ve created separate topic:

Very nice topic. I had a similar config long ago. With the difference that it was on a Desktop which booted via PXE by default. The tftp and early boot stage server was via an SBC with wireless connection to router connected to the desktop via Ethernet. The SBC was hidden inside air duct. And the LUKS volume started right after a dummy Windows partition. So without the PXE server (e.g. if the PC was stolen and moved elsewhere), PC would have booted to Windows and anyone would have thought it to be an ordinary Windows PC.

There was an independent replica of bootloader and /boot which I keep on a USB stick in a place no one would guess.