Qubes OS Installation - Detached encrypted boot and header

actually there’s some command that you should’nt do, but even if you do it have no harm, so I think you must fail somewhere.

I’ve decided to rewrite a complete full guide about this step in Advanced Qubes Installation (light installer / 4Kn Debian Template / Detached header / Encrypted Boot / Dom0 & DispVM in tmpfs) part 2

actually wipefs -a is enough, it would erase crypt metadata too.
you may want to try using iter 1 first for practicing while wait for me to rewrite better guide.

$500 crypto to anyone who can clear this guide and publish a idiot proof spoon fed, step by step guide on how to do detached boot and header thing

3 Likes

I am 100% down to match that offer so bounty is $1000

2 Likes

correct me if i am wrong but there is still the problem with our current new disks ssd / nvme and iirc even newer hdds having the key remaining in their little extra cache after use.
Forgive me that i do not have the correct technical terms for all this but i think as long as we do not know how this caches work and how to proper clear them with each shutdown, no one should feel 100% secure, of course it all is depending on the individual thread vector.

Instead of creating luks header on disk with:
[anaconda /] cryptsetup luksOpen /dev/nvme0n1 luks
And later removing it from disk as described in this guide you can just create luks with specified key file:

cryptsetup --key-file=/dev/mapper/lukskey --keyfile-offset=offset --keyfile-size=size luksFormat /dev/sda --offset 32768 --header /mnt/header.img
cryptsetup open --header /mnt/header.img --key-file=/dev/mapper/lukskey --keyfile-offset=offset --keyfile-size=size /dev/sda enc

Then there won’t be any traces left on the disk.
https://wiki.archlinux.org/title/Dm-crypt/Specialties#Encrypted_/boot_and_a_detached_LUKS_header_on_USB

Also it’s better to specify drives by their ID instead of arbitrary name like /dev/nvme0n1 or /dev/sda3:

cryptsetup open /dev/disk/by-id/usbdrive-part2 cryptboot
cryptsetup --header /mnt/header.img --key-file=/dev/mapper/lukskey --keyfile-offset=‘‘offset’’ --keyfile-size=‘‘size’’ open /dev/disk/by-id/harddrive enc
usbdrive is your USB drive by-id, and harddrive is your main hard drive by-id.

Also correction to the guide:
You need to leave 5-10% free space in VG in case it’ll be needed to extend pools:
lvcreate -T -l +90%FREE qubes_dom0/vm-pool
Why doesn't Qubes 4.1 use full disk space (LVM)? - #3 by marmarek
Best Way To Resize Dom0 Pool? - #9 by brendanhoar
LVM metadata problem - Free space in thin pool reached threshold - #3 by user-a9dc9a0d

you are correct, that’s why i decided to write full guide and using tmpfs for dom0 and dispvm. that would be part 4 of Running dom0 and DispVM in tmpfs.

the problem is if we did detach header before installation, anaconda would’nt understand that. try it, it would fail.

that’s correct, in my new quick install here, I already leave 10% free space

Just donate to qubes and post it here, I’m sure I will prioritize this, and complete the Playing with Qubes.

1 Like

i don’t see how that would protect from the unknowns of the manufacturer on disk device caching

Deleted, got it wrong.

Actually I only answer to this statement.

Running dom0 and dispvm from ram is decreasing thread vector.

For how this cache works I also don’t know, someday I’ll learn about this.

I just practiced again and found an error I forgot to post last time. It’s the very first error I encounter in the process. Here’s the error along with the few commands just before it:

[anaconda root@localhost /]# pvcreate /dev/mapper/luks
Physical volume "/dev/mapper/luks" successfully created.

[anaconda root@localhost /]# vgcreate qubes_dom0 /dev/mapper/luks
Volume group "qubes_dom0" successfully created

[anaconda root@localhost /]# lvcreate -n swap -L 4G qubes_dom0
Logical volume "swap" created.

[anaconda root@localhost /]# lvcreate -T -L 20G qubes_dom0/root-pool
Thin pool volume with chunk size 64.00 KiB can address at most 15.81 TiB of data.
/usr/bin/dmeventd: stat failed: No such file or directory
WARNING: Failed to monitor qubes_dom0/root-pool.
Logical volume "root-pool" created.

The other error I posted before also pertains to dmeventd. Even if nobody here has a solution, it helps to document and share errors.

1 Like

I’ve tried it out by configuring LUKS like this:

dd if=/dev/urandom of=key.img bs=32M count=1
cryptsetup luksFormat key.img
cryptsetup open key.img lukskey
dd if=/dev/zero of=header.img bs=16M count=1
cryptsetup --key-file=/dev/mapper/lukskey --keyfile-offset=0 --keyfile-size=8388608 luksFormat /dev/sda --offset 32768 --header header.img
cryptsetup open --header header.img --key-file=/dev/mapper/lukskey --keyfile-offset=0 --keyfile-size=8388608 /dev/sda qubes_dom0

pvcreate /dev/mapper/luks
vgcreate qubes_dom0 /dev/mapper/luks
lvcreate -n swap -L 512M qubes_dom0 
lvcreate -T -L 5G qubes_dom0/root-pool 
lvcreate -T -l +100%FREE qubes_dom0/vm-pool
lvcreate -V5G -T qubes_dom0/root-pool -n root
lvcreate -V4G -T qubes_dom0/vm-pool -n vm
mkfs.xfs /dev/qubes_dom0/vm

And installer can see the disk configuration created this way just fine when I press on “Reload storage configuration from disk”:

Or will it fail at a later stage?

For a key, I recommend /dev/random instead of /dev/urandom. The former ensures high entropy. (though I don’t know the answer to the rest of your question)

If you’re talking about this:
dd if=/dev/urandom of=key.img bs=32M count=1
Then I think it shouldn’t really matter because cryptsetup will use /dev/random when it’ll initialize a LUKS partition here:
cryptsetup luksFormat key.img
And the key used to encrypt /dev/sda later won’t be raw key.img file created from /dev/urandom but the decrypted key.img file that was encrypted with key from /dev/random.

I’ve looked into it more and luksFormat is using /dev/urandom by default:

#cryptsetup --help
Default compiled-in device cipher parameters:
	loop-AES: aes, Key 256 bits
	plain: aes-cbc-essiv:sha256, Key: 256 bits, Password hashing: ripemd160
	LUKS: aes-xts-plain64, Key: 256 bits, LUKS header hashing: sha256, RNG: /dev/urandom
	LUKS: Default keysize with XTS mode (two internal keys) will be doubled.

So maybe it’ll make sense to add --use-random option to cryptsetup luksFormat command.
But there’s a note in arch wiki about it:

Note that /dev/random blocking pool has been removed. Therefore, --use-random flag is now equivalent to --use-urandom .

So maybe using /dev/random is pointless.

1 Like

I have paid 50% of my donation as promised. $250 via xmr to Qubes. Will pay the balance when you publish. I Send you PM with txid for proof.

4 Likes

Will do in the next 24h.

Progress

Part 2 of Playing with qubes

Is a rewrite of my old notes, Qubes OS Installation - Detached encrypted boot and header

This will cover how you use it with xfs and btrfs on dom0

This is UEFI based only.

  • Prerequisite :

    • QubesOS Installation Medium.
    • 2 Drives for separated Root and Boot/EFI Partition, and we would call them with :
      • Root = /dev/nvme0n1
      • Boot = /dev/sda
  • After booting into installation in language section, press ctrl + alt + f2 to enter tty2

  • Format drive we would use for the installation.

    • dd if=/dev/urandom of=/dev/nvme0n1 bs=1M status=progress

    • dd if=/dev/urandom of=/dev/sda bs=1M status=progress

  • Create Boot, EFI, Header Partition.

    • sgdisk -n 0:0:+200MiB -t 0:ef02 /dev/sda

    • sgdisk -n 0:0:+200MiB -t 0:ef00 /dev/sda

    • sgdisk -n 0:0:+16MiB -t 0:8309 /dev/sda

XFS

  • Create custom Luks configuration.

    • cryptsetup -c aes-xts-plain64 -h sha512 -s 512 -y -i 10000 luksFormat /dev/nvme0n1

    You may want to use -i 1 (iterations 1ms) to speed up decrypting process for practice.

    • cryptsetup luksOpen /dev/nvme0n1 luks-root

    • pvcreate /dev/mapper/luks-root

    • vgcreate qubes_dom0 /dev/mapper/luks-root

    • lvcreate -L 8G -n swap qubes_dom0

    You might want to read 8.2. Recommended system swap space if you’re not sure about swap space

    • lvcreate -T -L 40G qubes_dom0/root-pool

    • lvcreate -T -l +90%FREE qubes_dom0/vm-pool

    • lvcreate -V30G -T qubes_dom0/root-pool -n root-pool

    • lvs

    To see how much space you have in the vm-pool, and use it to create vm lv.

    • lvcreate -V800G -T qubes_dom0/vm-pool -n vm

    • mkfs.xfs /dev/qubes_dom0/vm

    • mkswap /dev/mapper/swap

BTRFS

  • Create Root and Swap Partition.
    • sgdisk -n 0:0:+8GiB -t 0:8200 /dev/nvme0n1

    • sgdisk -n 0:0:0 -t 0:8304 /dev/nvme0n1

  • Create custom Luks configuration.
    • cryptsetup -c aes-xts-plain64 -h sha512 -s 512 -y -i 10000 luksFormat /dev/nvme0n1p1

    • cryptsetup -c serpent-xts-plain64 -h sha512 -s 512 -y -i 10000 luksFormat /dev/nvme0n1p2

    • cryptsetup luksOpen /dev/nvme0n1p1 luks-root

    • cryptsetup luksOpen /dev/nvme0n1p2 luks-swap

    • mkfs.btrfs --csum blake2b -L qubes_dom0 -d single /dev/mapper/luks-root

    • mkswap /dev/mapper/swap

Proceed GUI Installation

Video / Pict ?

Begin installation.

  • After installation go back to tty2

    • cp -r /usr/lib/grub/x86_64-efi /mnt/sysroot/boot/efi/EFI/qubes/

    • chroot /mnt/sysroot/

    • mount -oremount,ro /boot

    • install -m0600 /dev/null /tmp/boot.tar

    • tar -C /boot --acls --xattrs --one-file-system -cf /tmp/boot.tar .

    • umount /boot/efi

    • umount /boot

  • Reformat Boot Partition

    • cryptsetup -c twofish-xts-plain64 -h sha512 -s 512 -y -i 1 --use-random --type luks1 luksFormat /dev/sdb1

    • uuidR=“$(blkid -o value -s UUID /dev/nvme0n1)”

    • uuidB=“$(blkid -o value -s UUID /dev/sda1)”

    • cryptsetup luksOpen /dev/sda1 luks-$uuidB

    • mkfs.xfs /dev/mapper/luks-$uuidB

    • xfs_admin -U $uuidB /dev/mapper/luks-$uuidB

  • Configure fstab

    • sed -i ‘s/dev/mapper/root/dev/mapper/luks-’$uuidR’\g’ /etc/fstab

    • sed -i ‘s\UUID=F4N*/dev/mapper/luks-’$uuidR’\g’ /etc/fstab

    Replace F4N with First 4 UUID Number of /dev/sda1

    • mount -v /boot

    • tar -C /boot --acls --xattrs -xf /tmp/boot.tar

    • mount /dev/sda1 /boot/efi

  • Configure keys

    • mkdir -m0700 /etc/keys

    • ( umask 0077 && dd if=/dev/urandom bs=1 count=64 of=/etc/keys/root.key conv=excl,fsync )

    • ( umask 0077 && dd if=/dev/urandom bs=1 count=64 of=/etc/keys/boot.key conv=excl,fsync )

    • cryptsetup luksAddKey /dev/nvme0n1 /etc/keys/root.key

    • cryptsetup luksAddKey /dev/sda1 /etc/keys/boot.key

    • cryptsetup luksHeaderBackup /dev/nvme0n1 --header-backup-file header

    • dd if=/header of=/dev/sda3 bs=16M count=1 status=progress

    • shred -uvz /header

    • shred -uvz /tmp/boot.tar

  • Configure Crypttab

    • echo -e “luks-$uuidR /dev/nvme0n1 /etc/keys/root.key luks,discard,key-slot=1,header=/dev/sda3\nluks-$uuidB UUID=$uuidB /etc/keys/boot.key luks,key-slot=1” > /etc/crypttab

  • Configure GRUB

    • echo “GRUB_ENABLE_CRYPTODISK=y” >> /etc/default/grub

    • sed btrfs driver # TODO

    • grub2-mkconfig -o /boot/efi/EFI/qubes.cfg

  • Configure Dracut

    • echo -e ‘add_dracutmodules+=" crypt “\ninstall_items+=” /etc/keys/root.key /etc/keys/boot.key ”’ > /etc/dracut.conf.d/qubes.conf

    • sed -i ‘s?block_uuid.map"?block_uuid.map"\necho “/dev/nvme0n1 ‘$uuidR’\n/dev/disk/by-uuid/’$uuidB’ ‘$uuidB’" > “${initdir}/etc/block_uuid.map”?g’ /usr/lib/dracut/modules.d/90crypt/module-setup.sh

    • sed -i ‘s?$initdir/etc/crypttab?$initdir/etc/crypttab\necho “luks-’$uuidR’ /dev/nvme0n1 /etc/keys/root.key luks,discard,key-slot=1,header=/dev/sda3\nluks-‘$uuidB’ UUID=‘$uuidB’ /etc/keys/boot.key luks,key-slot=1” > $initdir/etc/crypttab?g’ /usr/lib/dracut/modules.d/90crypt/module-setup.sh

    • dracut -vf /boot/initramfs-*

    • exit

    • umount /mnt/sysroot/boot/efi

    • umount /mnt/sysroot/boot

    • umount -l /mnt/sysroot

    • umount -l /mnt/sysimage

    • swapoff /dev/qubes_dom0/swap

    • vgchange -a n qubes_dom0

    • cryptsetup luksClose /dev/mapper/luks-root

    • cryptsetup luksClose /dev/mapper/luks-*

    • wipefs -a /dev/nvme0n1

    • reboot

  • Will tested tomorrow on my machine
  • Check for typo
  • Record video how to do this (will use vm).

I will try later, I remember in my first research, it failed.

don’t bother with the error, just proceed you’ll be fine, and you may want to follow the progress above instead of #1, and look for #1 or in btrfs thread for the image.

Maybe it’ll work if you use PARTUUID /dev/disk/by-partuuid/ instead of UUID?

UUID is a filesystem-level UUID, which is retrieved from the filesystem metadata inside the partition. It can only be read if the filesystem type is known and readable.

PARTUUID is a partition-table-level UUID for the partition, a standard feature for all partitions on GPT-partitioned disks. Since it is retrieved from the partition table, it is accessible without making any assumptions at all about the actual contents of the partition. If the partition is encrypted using some unknown encryption method, this might be the only accessible unique identifier for that particular partition.

fstab - What is UUID, PARTUUID and PTUUID? - Unix & Linux Stack Exchange