Qubes OS Installation - Detached encrypted boot and header

Will do in the next 24h.

Progress

Part 2 of Playing with qubes

Is a rewrite of my old notes, Qubes OS Installation - Detached encrypted boot and header

This will cover how you use it with xfs and btrfs on dom0

This is UEFI based only.

  • Prerequisite :

    • QubesOS Installation Medium.
    • 2 Drives for separated Root and Boot/EFI Partition, and we would call them with :
      • Root = /dev/nvme0n1
      • Boot = /dev/sda
  • After booting into installation in language section, press ctrl + alt + f2 to enter tty2

  • Format drive we would use for the installation.

    • dd if=/dev/urandom of=/dev/nvme0n1 bs=1M status=progress

    • dd if=/dev/urandom of=/dev/sda bs=1M status=progress

  • Create Boot, EFI, Header Partition.

    • sgdisk -n 0:0:+200MiB -t 0:ef02 /dev/sda

    • sgdisk -n 0:0:+200MiB -t 0:ef00 /dev/sda

    • sgdisk -n 0:0:+16MiB -t 0:8309 /dev/sda

XFS

  • Create custom Luks configuration.

    • cryptsetup -c aes-xts-plain64 -h sha512 -s 512 -y -i 10000 luksFormat /dev/nvme0n1

    You may want to use -i 1 (iterations 1ms) to speed up decrypting process for practice.

    • cryptsetup luksOpen /dev/nvme0n1 luks-root

    • pvcreate /dev/mapper/luks-root

    • vgcreate qubes_dom0 /dev/mapper/luks-root

    • lvcreate -L 8G -n swap qubes_dom0

    You might want to read 8.2. Recommended system swap space if you’re not sure about swap space

    • lvcreate -T -L 40G qubes_dom0/root-pool

    • lvcreate -T -l +90%FREE qubes_dom0/vm-pool

    • lvcreate -V30G -T qubes_dom0/root-pool -n root-pool

    • lvs

    To see how much space you have in the vm-pool, and use it to create vm lv.

    • lvcreate -V800G -T qubes_dom0/vm-pool -n vm

    • mkfs.xfs /dev/qubes_dom0/vm

    • mkswap /dev/mapper/swap

BTRFS

  • Create Root and Swap Partition.
    • sgdisk -n 0:0:+8GiB -t 0:8200 /dev/nvme0n1

    • sgdisk -n 0:0:0 -t 0:8304 /dev/nvme0n1

  • Create custom Luks configuration.
    • cryptsetup -c aes-xts-plain64 -h sha512 -s 512 -y -i 10000 luksFormat /dev/nvme0n1p1

    • cryptsetup -c serpent-xts-plain64 -h sha512 -s 512 -y -i 10000 luksFormat /dev/nvme0n1p2

    • cryptsetup luksOpen /dev/nvme0n1p1 luks-root

    • cryptsetup luksOpen /dev/nvme0n1p2 luks-swap

    • mkfs.btrfs --csum blake2b -L qubes_dom0 -d single /dev/mapper/luks-root

    • mkswap /dev/mapper/swap

Proceed GUI Installation

Video / Pict ?

Begin installation.

  • After installation go back to tty2

    • cp -r /usr/lib/grub/x86_64-efi /mnt/sysroot/boot/efi/EFI/qubes/

    • chroot /mnt/sysroot/

    • mount -oremount,ro /boot

    • install -m0600 /dev/null /tmp/boot.tar

    • tar -C /boot --acls --xattrs --one-file-system -cf /tmp/boot.tar .

    • umount /boot/efi

    • umount /boot

  • Reformat Boot Partition

    • cryptsetup -c twofish-xts-plain64 -h sha512 -s 512 -y -i 1 --use-random --type luks1 luksFormat /dev/sdb1

    • uuidR=“$(blkid -o value -s UUID /dev/nvme0n1)”

    • uuidB=“$(blkid -o value -s UUID /dev/sda1)”

    • cryptsetup luksOpen /dev/sda1 luks-$uuidB

    • mkfs.xfs /dev/mapper/luks-$uuidB

    • xfs_admin -U $uuidB /dev/mapper/luks-$uuidB

  • Configure fstab

    • sed -i ‘s/dev/mapper/root/dev/mapper/luks-’$uuidR’\g’ /etc/fstab

    • sed -i ‘s\UUID=F4N*/dev/mapper/luks-’$uuidR’\g’ /etc/fstab

    Replace F4N with First 4 UUID Number of /dev/sda1

    • mount -v /boot

    • tar -C /boot --acls --xattrs -xf /tmp/boot.tar

    • mount /dev/sda1 /boot/efi

  • Configure keys

    • mkdir -m0700 /etc/keys

    • ( umask 0077 && dd if=/dev/urandom bs=1 count=64 of=/etc/keys/root.key conv=excl,fsync )

    • ( umask 0077 && dd if=/dev/urandom bs=1 count=64 of=/etc/keys/boot.key conv=excl,fsync )

    • cryptsetup luksAddKey /dev/nvme0n1 /etc/keys/root.key

    • cryptsetup luksAddKey /dev/sda1 /etc/keys/boot.key

    • cryptsetup luksHeaderBackup /dev/nvme0n1 --header-backup-file header

    • dd if=/header of=/dev/sda3 bs=16M count=1 status=progress

    • shred -uvz /header

    • shred -uvz /tmp/boot.tar

  • Configure Crypttab

    • echo -e “luks-$uuidR /dev/nvme0n1 /etc/keys/root.key luks,discard,key-slot=1,header=/dev/sda3\nluks-$uuidB UUID=$uuidB /etc/keys/boot.key luks,key-slot=1” > /etc/crypttab

  • Configure GRUB

    • echo “GRUB_ENABLE_CRYPTODISK=y” >> /etc/default/grub

    • sed btrfs driver # TODO

    • grub2-mkconfig -o /boot/efi/EFI/qubes.cfg

  • Configure Dracut

    • echo -e ‘add_dracutmodules+=" crypt “\ninstall_items+=” /etc/keys/root.key /etc/keys/boot.key ”’ > /etc/dracut.conf.d/qubes.conf

    • sed -i ‘s?block_uuid.map"?block_uuid.map"\necho “/dev/nvme0n1 ‘$uuidR’\n/dev/disk/by-uuid/’$uuidB’ ‘$uuidB’" > “${initdir}/etc/block_uuid.map”?g’ /usr/lib/dracut/modules.d/90crypt/module-setup.sh

    • sed -i ‘s?$initdir/etc/crypttab?$initdir/etc/crypttab\necho “luks-’$uuidR’ /dev/nvme0n1 /etc/keys/root.key luks,discard,key-slot=1,header=/dev/sda3\nluks-‘$uuidB’ UUID=‘$uuidB’ /etc/keys/boot.key luks,key-slot=1” > $initdir/etc/crypttab?g’ /usr/lib/dracut/modules.d/90crypt/module-setup.sh

    • dracut -vf /boot/initramfs-*

    • exit

    • umount /mnt/sysroot/boot/efi

    • umount /mnt/sysroot/boot

    • umount -l /mnt/sysroot

    • umount -l /mnt/sysimage

    • swapoff /dev/qubes_dom0/swap

    • vgchange -a n qubes_dom0

    • cryptsetup luksClose /dev/mapper/luks-root

    • cryptsetup luksClose /dev/mapper/luks-*

    • wipefs -a /dev/nvme0n1

    • reboot

  • Will tested tomorrow on my machine
  • Check for typo
  • Record video how to do this (will use vm).

I will try later, I remember in my first research, it failed.

don’t bother with the error, just proceed you’ll be fine, and you may want to follow the progress above instead of #1, and look for #1 or in btrfs thread for the image.

Maybe it’ll work if you use PARTUUID /dev/disk/by-partuuid/ instead of UUID?

UUID is a filesystem-level UUID, which is retrieved from the filesystem metadata inside the partition. It can only be read if the filesystem type is known and readable.

PARTUUID is a partition-table-level UUID for the partition, a standard feature for all partitions on GPT-partitioned disks. Since it is retrieved from the partition table, it is accessible without making any assumptions at all about the actual contents of the partition. If the partition is encrypted using some unknown encryption method, this might be the only accessible unique identifier for that particular partition.

fstab - What is UUID, PARTUUID and PTUUID? - Unix & Linux Stack Exchange

I’m not sure, if you look at my approach I wrote the header on the partition, so the partition will have the same metadata as root.

I’m surprised by this,

it actually work.

I think, it’s not necessary, I did shred on both boot.rar and header

I assume that because you’re writing just header on raw partition then there is no filesystem there so there is no UUID for this partition. But this partition still has PARTUUID and could be addressed with /dev/disk/by-partuuid/.

Shred works fine for HDD, but it doesn’t work for SSD:
dm-crypt/Specialties - ArchWiki

I’ve checked it myself and my guess was wrong. When you write header to the raw partition then it’ll have UUID of the luks partition.

Ah i see, that make sense with ssd but still, I think one need to bypass crypt first to examine this. But your approach to creating detach header before installation is acceptable, I’ve add it to my note, but not with these guide. It has been 1y+ and everything still fine.

As expected.

@51lieal The image seems to be missing from your post. Provided below.

[anaconda /] vi /etc/crypttab 
---# Change root device value so it look like this 

![Qubes OS-2021-09-09-05-41-16|690x100](upload://wq7nqayfAzpGbeT5p8gfLqB1XMc.png)

---

Take a look at this for anyone who want a better guide.

1 Like

After completion of the installation in full - meaning upon arrival to the fully installed desktop system? Or after completion of the first part of the installation, prior to booting and putting the LUKS password for the first time and prior to configuration and template installation? Thanks!

Take a look at this for better guide.

yes, after first part of installation.

I’ve included video there, you should check.

1 Like

@51lieal I followed this is exact guide and it worked well but now I would like to switch back to using only the ssd without booting from SD card. Is it possible without reinstalling? I guess I would need to shrink the partition but I don’t know how to do it since the whole disk is encrypted.

@51lieal Please clarify for us when we should use the $uuidR or $uuidB as a variable and when we should use it alphanumerically, thank you.

1 Like

So is this possible to do with a flashdrive for boot and one for header?

Hello.

So far, I’ve gotten to the point where I entered

tar -C /boot --acls --xattrs --one-file-system -cf /tmp/boot.tar

But every time I tried it, I got this output

tar: Cowardly refusing to create an empty archive
Try 'tar --help' or 'tar --usage' for more information.

I’m sure I followed the instructions, so far, to the letter so where could I have gone wrong?

Not sure how this is going to effect your guides yet (need to dig into it more deeply), but there are big changes coming in grub, though unfortunately qubes 4.1.2 only has grub 2.4.

LUKS2 has been added to grub, though they are still waiting on Argon2i/d, it is in the AUR version.

(Continue in next post, since new users can only include two links…)

Also: “Grub gained detached header support on 8th June 2022 with commit 1deb5214.”

(Continue in next post…)

“As of this notice the current Grub version 2.06 predates this change, however the Arch Linux Grub package is based of the master branch and therefore includes this support. Other distributions may have to wait for the next Grub release.”

the problem is not in the grub, main problem is in anaconda.
you can manually pick argon2i/d in the 4.1.

Hey. Has anyone been able to run this kind of installation recently?
Qubes installation GUI has changed a lot since the original posting of this thread and it seems 51lieal has gone radio silent.

I contacted him personally last year to ask for help for a particular setup involving a detached encrypted boot and header along with a single OS drive and a two-disk RAID to store the vm-pool. Sadly, I’ve been too busy with a lot of stuff to actually go through with the thing.

I recently tried taking a stab at this.

Here’s the first of the scripts.
Script a

#!/bin/bash

# Check lsblk first then change the variables according to the listed devices' directories
RT="/dev/sdb" # M.2 SSD
VM1="/dev/sda" # SATA SSD 1
VM2="/dev/sdc" # SATA SSD 2
FD="/dev/sde" # USB Flash Drive
EF="/dev/sde1"
BT="/dev/sde2"

# Rescan Drive
partprobe $RT
partprobe $FD
partprobe $VM1
partprobe $VM2

# Create boot, EFI, and LUKS partitions for USB Flash Drive (FD)
sgdisk -n 0:0:+600MiB -t 0:ef02 $FD
sgdisk -n 0:0:+2GiB -t 0:ef00 $FD
sgdisk -n 0:0:+16MiB -t 0:8309 $FD
sgdisk -n 0:0:+16MiB -t 0:8309 $FD
sgdisk -n 0:0:+16MiB -t 0:8309 $FD
partprobe $FD
mkfs.vfat $EF
mkfs.xfs -f $BT

# Encrypt root partition
cryptsetup -c serpent-xts-plain64 -h ripemd160 -s 512 -y -i 60000 --use-random --pbkdf argon2id luksFormat $RT #M.2 SSD
cryptsetup -c twofish-xts-plain64 -h whirlpool -s 512 -y -i 60000 --use-random --pbkdf argon2id luksFormat $VM1 # SATA SSD 1
cryptsetup -c twofish-xts-plain64 -h whirlpool -s 512 -y -i 60000 --use-random --pbkdf argon2id luksFormat $VM2 # SATA SSD 2


# Open root device
cryptsetup luksOpen $RT luks-root #M.2 SSD
cryptsetup luksOpen $VM1 luks-vm1 # SATA SSD 1
cryptsetup luksOpen $VM2 luks-vm2 # SATA SSD 2

# Create PV
pvcreate /dev/mapper/luks-root #M.2 SSD
pvcreate /dev/mapper/luks-vm1 # SATA SSD 1
pvcreate /dev/mapper/luks-vm2 # SATA SSD 2

# Create VG
vgcreate qubes_dom0 /dev/mapper/luks-root #M.2 SSD
vgcreate qubes_domU /dev/mapper/luks-vm1 /dev/mapper/luks-vm2 # SATA SSD 1 & 2, prepare to make RAID array

# Create LV
lvcreate -n swap -L 16G qubes_dom0 #Create (encrypted) SWAP partition in root
lvcreate -T -L 64G qubes_dom0/root-pool
lvcreate -V32G -T qubes_dom0/root-pool -n root
# According to 51lieal, if the root exceeds 32GB, it will extend the partition to 64GB (as defined above)
lvcreate -m 1 --type raid1 -L 100G --nosync -n vm qubes_domU # Create RAID1 array

# Format Partition
mkfs.xfs /dev/qubes_domU/vm
mkswap /dev/qubes_dom0/swap

After running script a, I’m supposed to go back to the GUI to finish the installation. The install, as 51lieal and I discussed, would involve the logical volume manager however I was unable to set any mount points for the partitions created in /sdb (my main OS drive) so I tried to contact him. No reply so far.

So I wonder if anybody else here can help me.
Thanks.