Install Qubes OS with boot partition and a detached LUKS header on USB

Community Guides
1

Boot into Qubes OS installer and on GUI screen switch to shell on another TTY by pressing Ctrl+Alt+F2.

Assuming:
/dev/sda - disk where you want to install Qubes OS
/dev/sdb - USB disk where you want to install Qubes OS boot partition

Find out the right disk names on your machine in Qubes OS installer shell by running this command and checking the output:

fdisk -l

In this shell run these commands:

dd if=/dev/zero of=header.img bs=16M count=1
cryptsetup luksFormat /dev/sda -c aes-xts-plain64 -h sha512 -s 512 -y -i 10000 --use-random --force-password --header header.img
cryptsetup open --header header.img /dev/sda luks
mkfs.btrfs --csum xxhash -L qubes_dom0 -d single /dev/mapper/luks

Return to Qubes OS installer GUI by pressing Ctrl+Alt+F6.
Configure the installation as normal except for Installation Destination:
Installation guide | Qubes OS
At the Installation Destination screen click on “Refresh
” at the bottom right corner and in the opened window press on “Disk rescan” button and then OK.
At the Device Selection choose your Qubes OS installation destination and boot partition destination disks /dev/sda and /dev/sdb.
At the Storage Configuration select “Advanced Custom (Blivet-GUI)”.
Press Done.
In the “BLIVET GUI PARTITIONING” screen select “sdb” disk.
Delete the old partitions on the disk if needed.

Add new partition using “+” button:
Device type: Partition
Size: 512 MiB
Filesystem: EFI System
Mountpoint: /boot/efi
Press OK button.

Add new partition using “+” button:
Device type: Partition
Size: 1 GiB
Filesystem: ext4
Mountpoint: /boot
Press OK button.

In the “BLIVET GUI PARTITIONING” screen select “qubes_dom0” Btrfs Volume.
Add new subvolume using “+” button:
Name: root
Mountpoint: /
Press OK button.

Press on Done button in “BLIVET GUI PARTITIONING” and then “Accept Changes” button in “SUMMARY OF CHANGES” window.
Press “Begin Installation” button.

After installation is completed don’t press “Reboot System” button.
Switch to shell on another TTY by pressing Ctrl+Alt+F2.
Run these commands in the installer shell:

cp header.img /mnt/sysroot/root/
cd /mnt/sysroot
mount -t proc /proc proc/
mount -t sysfs /sys sys/
mount --rbind /dev dev/
chroot /mnt/sysroot
btrfs subvolume create /swap
btrfs filesystem mkswapfile --size=4g --uuid clear /swap/swapfile

Edit /etc/fstab file using nano or any other text editor:

nano /etc/fstab

Add noauto option to the /boot and /boot/efi mounts like this:

UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX /boot                   ext4    defaults,discard,noauto 1 2
UUID=XXXX-XXXX          /boot/efi               vfat    umask=0077,shortname=winnt,discard,noauto 0 2

And add this line at the end:

/swap/swapfile none swap defaults,discard=once 0 0

Save and close /etc/fstab file.
Edit /etc/default/grub file using nano or any other text editor:

nano /etc/default/grub

Remove rd.luks.uuid=XXXXX option and add rd.driver.pre=btrfs option to the GRUB_CMDLINE_LINUX variable. It should look like this:

GRUB_CMDLINE_LINUX="rd.driver.pre=btrfs plymouth.ignore-serial-consoles 6.7.7-1.qubes.fc37.x86_64 x86_64 rhgb quiet"

Save and close /etc/default/grub file.
Create new /etc/dracut.conf.d/crypt.conf file using nano or any other text editor:

nano /etc/dracut.conf.d/crypt.conf

Add this text inside:

add_dracutmodules+=" crypt "
install_items+=" /root/header.img "

Save and close /etc/dracut.conf.d/crypt.conf file.
Find out the ID of your Qubes OS destination disk /dev/sda:

ls -la /dev/disk/by-id/ | grep /sda

For example it’ll look like this:

lrwxrwxrwx 1 root root   9 May 10 20:15 ata-YOUR_DISK_ID -> ../../sda

Here ata-YOUR_DISK_ID is your disk ID.

Edit /etc/crypttab file using nano or any other text editor:

nano /etc/crypttab

Add this text inside:

luks /dev/disk/by-id/ata-YOUR_DISK_ID none header=/root/header.img,force,discard

Save and close /etc/crypttab file.
Run these commands:

grub2-mkconfig -o /boot/grub2/grub.cfg
dracut -f --regenerate-all
exit

Return to Qubes OS installer GUI by pressing Ctrl+Alt+F6.
Press on “Reboot System” button.
After this proceed with normal Qubes OS post-installation process:
Installation guide | Qubes OS

Since /boot and /boot/efi partitions are stored on USB disk then you’ll need to attach this disk to your dom0 when doing dom0 updates so the files there will be updated.
It’s better to use disposable sys-usb for this setup.
When you want to update your dom0 you can follow these steps:
Disconnect all your USB devices.
Restart sys-usb to clear it’s state.
Connect your USB disk with Qubes OS /boot partition.
Run this command in dom0 to mount the /boot and /boot/efi partitions in dom0 assuming that /dev/sda is the name of your USB disk with Qubes OS /boot partition in sys-usb:

qvm-block attach dom0 sys-usb:sda1 && qvm-block attach dom0 sys-usb:sda2 && sudo mount /boot && sudo mount /boot/efi

Run dom0 update.
After dom0 update is finished run this command in dom0 to unmount and remove /boot and /boot/efi partitions from dom0:

sudo umount /boot/efi /boot && qvm-block d dom0 sys-usb:sda1 && qvm-block d dom0 sys-usb:sda2

At this point you can disconnect the USB disk with Qubes OS /boot partition from your machine and continue to use sys-usb with other USB devices as normal.

3 Likes
2

Assuming the installer and the boot process itself have no USB protection (like sys-usb or USBGuard), how does one trust a USB device with complex proprietary firmware (such as USB storage) at install or boot time? I.e. how and why does one trust such a device at such early and vulnerable phase? I don’t dare to connect even the USB port to the UPS :slight_smile:

3

You can use USB device if it’s OK for your threat model.
USB device that you use to boot the Qubes OS from should be assumed as trusted.
The same way that USB device containing the Qubes OS installer that is used to install the Qubes OS should be assumed as trusted.
Installation security | Qubes OS
Instead of using USB device to boot Qubes OS from it could be considered to use the SD card reader (if available in your machine) and SD card to boot from. There is no firmware in SD cards (AFAIK).

4

I was wrong, there is a firmware there as well:
https://www.bunniestudios.com/blog/2013/on-hacking-microsd-cards/

So I guess if you use a separate PCI USB controller or PCI SD card reader only to boot from your boot device then it’ll be the same as booting from SATA/NVMe drive.
You’ll have to trust that your boot device firmware is not malicious the same way that you’ll need to trust that your SATA/NVMe drive firmware is not malicious. And that both your boot device and SATA/NVMe drive couldn’t be physically accessed by attacker.

5

Isn’t the point that with one’s USB goes access to the physical device? (Assuming one carries the USB once shutting down the device and walking away.)

6

There is always a chance that attacker can access it in certain situations (like during border control search or when you sleep). The same goes for attacker accessing the machine.
Also USB device firmware could be malicious even before it’ll end up in your hands. But the same goes for any other hardware.

7

I think the main issue with USB storage specifically is that it can behave as other devices like USB keyboard.
On the other hand even if e.g. SD card has malicious firmware then it still can’t be used like a keyboard. It only can be used to further exploit the vulnerability in the SD card reader to gain further access of your system.

8

@apparatus

I know the things you mention but they still don’t answer the actual question. However, since it requires a more in-depth discussion, it would better fit a separate thread :slight_smile:

9

I’ve created separate topic:

10

Very nice topic. I had a similar config long ago. With the difference that it was on a Desktop which booted via PXE by default. The tftp and early boot stage server was via an SBC with wireless connection to router connected to the desktop via Ethernet. The SBC was hidden inside air duct. And the LUKS volume started right after a dummy Windows partition. So without the PXE server (e.g. if the PC was stolen and moved elsewhere), PC would have booted to Windows and anyone would have thought it to be an ordinary Windows PC.

There was an independent replica of bootloader and /boot which I keep on a USB stick in a place no one would guess.

11

@apparatus: Thank you for sharing. Could you please explain in a little bit more depth what you are doing or recommending exactly?

To my understanding
a) you are using a btrfs volume instead or additionally to an lvm volume? why?
b) you are placing the boot partition and the luks header of the root device on an usb stick?
c) you are setting up a swapfile? why?
d) you are passing this changes to grub and dracut

Usually it is not possible to umount /boot during runtime of the OS. But it looks like your setups unmounts /boot after startup and the user can detach the usb stick - which could be considered a security feature. At least you are writing that the user has to reattach the usb stick and /boot if one wants to update kernel, modules and initramfs.

12

The reasons for using btrfs instead of lvm can be found here:

Also btrfs is now default filesystem in Fedora instead of lvm:
https://fedoraproject.org/wiki/Changes/BtrfsByDefault

But you can use this guide and change btrfs to lvm as well.

It’s to achieve deniable encryption:

Without USB stick the disk will look like unused/empty unpartitioned disk.

You mean why use swap at all or why swap file instead of separate partition?
Swap is needed for dom0 not to fail in case of some rare memory exhausting situation.
I’m using swap file instead of separate swap partition because separate swap partition requires the disk to be partitioned and it’ll interfere with deniable encryption.
I’ve thought of a way to use separate swap partition without partitioning the disk, but it’s not supported by systemd:

Can you describe your question in more details? I didn’t get it.

You can safely unmount the /boot during runtime, the USB stick is only needed to boot the initramfs, you can remove the USB stick after you enter the disk password and dom0 will start to boot.
And if you have sys-usb then USB stick won’t be present in dom0 by default because all USB controllers will be attached to sys-usb.
You only need to mount the /boot and /boot/efi when you update dom0 system:

2 Likes
13

Thank you for adding these explanations. If I understood you correctly your setup creates a block device /dev/sda with encrypted data which provides more or less plausibly deniability. The famous wrench taken out of the equation. Does /dev/sda hold a partition table?

In my eyes your setup also adds another layer of security as the boot partition is a weak spot in all linux distros. Most distros don’t encrypt the boot partition and don’t cryptographically sign the files on the boot partition. It’s just the kernel, the modules, the initramfs and the EFI boot blobs, so what could possible go wrong? :wink:

That was just my thought process on

14

Looks like I can answer this question myself:

It does not expose a partition table.

Awesome!

15

Yes, it’ll be a /dev/sda device without partition table.
But since TRIM is enabled there may be some security implications:

Disk Trimming
So you may want to disable TRIM.

Yes, it can protect the /boot and /boot/efi from malicious modifications in some cases.
But if you’re using USB disk with the same USB controller attached to sys-usb that you use to connect some untrusted USB devices then it’s possible that some malicious USB device will compromise sys-usb, write malicious firmware in USB controller and restarting disposable sys-usb won’t get rid of it and this malware in the USB controller firmware may then modify the files on your boot USB disk that you connect to it.
Also unlike Anti evil maid or TrenchBoot, it won’t protect against malicious EFI firmware.