Take a look in my new thread, and review what changes in there.
Hmm for your thought i just want to learn and try this, I’ve been asked by someone to do this in qubes os. Even they are trying to give a donation, I will not answer. There’s a better place to give a donation like Qubes OS or Free Software Foundation
Some people, including some core developers, are interacting with this forum via email. They do not receive the edits. For them, it would be useful to include the contents of the updated post into the spoiler:
Used device in testing and confirm is worked :
windows vmware (host using old pc) and my laptop.
both are using uefi.
Keep in mind that below are disk i used in the tutorial, you can use 2 flashdrive (1 boot, 1 header) + 1 hdd or whatever you want.
/dev/nvme0n1 = system
/dev/sda = flashdrive
Please watch out any space, slash, periode in command issue / files IT REALLY MATTER
—# WARNING CONFIRM YOUR DISK FIRST BEFORE EXECUTING COMMAND
[anaconda /] dd if=/dev/urandom of=/dev/nvme0n1 bs=1M status=progress
[anaconda /] dd if=/dev/urandom of=/dev/sda bs=1M status=progress
Using /dev/urandom will take longer than /dev/zero but more secure.
[anaconda /] gdisk /dev/sda
—# efi partition
n
1
2048
1230845
EF00
—# boot partition
n
2
1230848
3278842
8300
—# header partition
n
3
(enter)
(enter)
(enter)
w
Y
—# I use iter time 1 for speeding up decrypt process you should increase it in real installation, see 5.13 for details.
[anaconda /] cryptsetup -c aes-xts-plain64 -h sha512 -s 512 -y -i 1 --use-random luksFormat /dev/nvme0n1
[luks prompt /] YES
[luks prompt /] (enter password)
[luks prompt /] (verify password)
[anaconda /] cryptsetup luksOpen /dev/nvme0n1 luks
[luks prompt /] (enter password)
[anaconda /] pvcreate /dev/mapper/luks
[anaconda /] vgcreate qubes_dom0 /dev/mapper/luks
[anaconda /] lvcreate -n swap -L 4G qubes_dom0
[anaconda /] lvcreate -T -L 20G qubes_dom0/root-pool
[anaconda /] lvcreate -T -l +100%FREE qubes_dom0/vm-pool
[anaconda /] lvs
—#
root-pool 20.00g
swap 4.00g
vm-pool 19.90g
—#
[anaconda /] lvcreate -V20G -T qubes_dom0/root-pool -n root
[anaconda /] lvcreate -V19.9G -T qubes_dom0/vm-pool -n vm
[anaconda /] mkfs.xfs /dev/qubes_dom0/vm
—# I use xfs because it much faster than ext4 when boot up, btw i dont do benchmark it’s just My Life Experience based on using qubes.
Otherwise :
[anaconda /] mkfs.ext4 /dev/qubes_dom0/vm
—# reformat disk
qubes_dom0-root, reformat, ext4, /, update settings.
qubes-dom0-swap, reformat, swap, update settings.
sda1, reformat, Efi System Partition, /boot/efi, update settings.
sda2, reformat, ext2, /boot, update settings.
—# leave qubes_dom0-vm and sda3 untouched.
[anaconda /] cp -r /usr/lib/grub/x86_64-efi /mnt/sysroot/boot/efi/EFI/qubes/
[anaconda /] chroot /mnt/sysroot/
[anaconda /] mount -oremount,ro /boot
[anaconda /] install -m0600 /dev/null /tmp/boot.tar
[anaconda /] tar -C /boot --acls --xattrs --one-file-system -cf /tmp/boot.tar .
[anaconda /] umount /boot/efi
[anaconda /] umount /boot
[anaconda /] exit
—# WARNING CONFIRM YOUR DISK FIRST BEFORE EXECUTING COMMAND
[anaconda /] dd if=/dev/urandom of=/dev/sda2 bs=1M status=progress
[anaconda /] cryptsetup -c twofish-xts-plain64 -h sha512 -s 512 -y -i 1 --use-random --type luks1 luksFormat /dev/sda2
[luks prompt /] YES
[luks prompt /] (enter password)
[luks prompt /] (verify password)
—#
[anaconda /] uuidR=“$(blkid -o value -s UUID /dev/nvme0n1)”
[anaconda /] uuidB=“$(blkid -o value -s UUID /dev/sda2)”
[anaconda /] cryptsetup luksOpen /dev/sda2 luks-$uuidB
[anaconda /] mkfs.ext2 -m0 -U $uuidB /dev/mapper/luks-$uuidB
—#
[anaconda /] vi /mnt/sysroot/etc/fstab
—# Change UUID=…on boot line to /dev/mapper/luks-(your $uuidB) and leave the rest to default value
[anaconda /] chroot /mnt/sysroot
[anaconda /] mount -v /boot
[anaconda /] tar -C /boot --acls --xattrs -xf /tmp/boot.tar
[anaconda /] mount /dev/sda1 /boot/efi
[anaconda /] echo “GRUB_ENABLE_CRYPTODISK=y” >> /etc/default/grub
—# create luks keys so we dont have to enter any password after grub
[anaconda /] mkdir /etc/keys
[anaconda /] dd if=/dev/urandom bs=1 count=64 of=/etc/keys/root.key conv=excl,fsync
[anaconda /] dd if=/dev/urandom bs=1 count=64 of=/etc/keys/boot.key conv=excl,fsync
[anaconda /] cryptsetup luksAddKey /dev/nvme0n1 /etc/keys/root.key
[luks prompt /] (system password)
[anaconda /] cryptsetup luksAddKey /dev/sda2 /etc/keys/boot.key
[luks prompt /] (boot password)
[anaconda /] cryptsetup luksHeaderBackup /dev/nvme0n1 --header-backup-file header
—# WARNING CONFIRM YOUR DISK FIRST BEFORE EXECUTING COMMAND
[anaconda /] dd if=/header of=/dev/sda3 bs=16M count=1 status=progress
[anaconda /] exit
[anaconda /] echo “luks-$uuidB UUID=$uuidB /etc/keys/boot.key luks,key-slot=1” | tee -a /mnt/sysroot/etc/crypttab
[anaconda /] mount --bind /dev /mnt/sysroot/dev
[anaconda /] mount --bind /dev/pts /mnt/sysroot/dev/pts
[anaconda /] mount --bind /sys /mnt/sysroot/sys
[anaconda /] mount --bind /proc /mnt/sysroot/proc
[anaconda /] chroot /mnt/sysroot
[anaconda /] vi /etc/crypttab
—# Change root device value so it look like this
[anaconda /] vi /etc/dracut.conf.d/misc.conf
add_dracutmodules+=" crypt "
install_items+=" /etc/keys/root.key /etc/keys/boot.key /sbin/cryptsetup "
[anaconda /] vi /usr/lib/dracut/modules.d/90crypt/module-setup.sh
—# write a persistence device at /etc/block_uuid.map in generated initramfs
echo “/dev/nvme0n1 $uuidR
/dev/disk/by-uuid/$uuidB $uuidB” > “{initdir}/etc/block_uuid.map”
—# write a persistence device at /etc/crypttab in generated initramfs (we can’t inject /etc/crypttab files in dracut.conf)
echo “luks-$uuidR /dev/nvme0n1 /etc/keys/root.key luks,discard,key-slot=1,header=/dev/sda3
luks-$uuidB UUID=$uuidB /etc/keys/boot.key luks,key-slot=1” > $initdir/etc/crypttab
[anaconda /] grub2-mkconfig -o /boot/efi/EFI/qubes/grub.cfg
[anaconda /] dracut -v -f /boot/initramfs-*
[anaconda /] exit
[anaconda /] umount /mnt/sysroot/boot/efi
[anaconda /] umount /mnt/sysroot/boot
[anaconda /] umount -l /mnt/sysroot
[anaconda /] umount -l /mnt/sysimage
[anaconda /] swapoff /dev/qubes_dom0/swap
[anaconda /] vgchange -a n qubes_dom0
[anaconda /] cryptsetup luksClose /dev/mapper/luks
[anaconda /] cryptsetup luksClose /dev/mapper/luks-$uuidB
[anaconda /] cryptsetup luksErase /dev/nvme0n1
[luks prompt /] YES
[anaconda /] wipefs -a /dev/nvme0n1
[anaconda /] reboot
—# Screenshoot
suspend not working, my current /etc/default/grub looks like this
GRUB_CMDLINE_LINUX=“rd.luksid…”
Where should i add mem_sleep_default=deep??
GRUB_CMDLINE_LINUX=“rd.luks.uuid=… rhgb quiet mem_sleep_default=deep” like this??
Yes put in there for kernel parameter.
Updated the command line parameters and grub.
The system tries to sleeps and wakes up suddenly with code 30 for the first time. If i try to suspend second time the system goes to sleep and need hard reset.
@51lieal Thanks a lot for this writeup. I’ve added some markdown formatting to make commands more readable. Hope that’s ok.
How to update to the latest kernel? I have installed the latest kernel but still it’s using old kernel. I have changed in qube manager global settings to use by default 5.16 but still it’s using 5.10.
I read this doc
When you set kernel in global settings, it only apply for domU, for dom0 you need to manually enter the kernel version you want in grub menu, or use ‘0’ value in GRUB_DEFAULT for the first entry in grub which defaulty the newest kernel installed.
Do you mean, instead of
GRUB_DEFAULT=saved
in /etc/default/grub, to put
GRUB_DEFAULT=0
Impressive! I think I did it wrong. After rebooting, it prompts for passphrase like it should. But when I enter the passphrase, it does nothing. I let it sit for longer than two minutes. The iter-time is 10000 which should be 10 seconds.
One point of confusion was whether to leave $uuidB or $uuidR verbatim as they are, or substitute them for the alphanumeric ID. In other words, I didn’t always know whether to leave the variable as a variable, or substitute the variable for the exact ID it references. I substituted them for exact IDs in /mnt/sysroot/etc/fstab and in the generated initramfs. Everywhere else, I left them verbatim as variables.
Another point of confusion was which column to change in crypttab. Is “root device value” the first column, or the second column? I read the man7.org page about crypttab(5) and a few other articles explaining it, but none of them make clear enough which column is the “device value.” This may seem obvious and a stupid question but I’m stumped.
Finally, I had an error at vgchange -a n qubes_dom0 . The error said:
/usr/sbin/dmeventd: stat failed: No such file or directory
WARNING: Failed to unmonitor qubes_dom0/root-pool.
0 logical volume(s) in volume group "qubes_dom0" now active
When I run lsblk, it shows partitions on the flash drive but nothing on the internal drive. The internal drive had partitions at some point during the process, but they apparently all got obliterated somehow. I wonder whether it happened at cryptsetup luksErase /dev/nvme0n1 since it warned me the drive would become unusable 
Where can I go from here?
actually there’s some command that you should’nt do, but even if you do it have no harm, so I think you must fail somewhere.
I’ve decided to rewrite a complete full guide about this step in Advanced Qubes Installation (light installer / 4Kn Debian Template / Detached header / Encrypted Boot / Dom0 & DispVM in tmpfs) part 2
actually wipefs -a is enough, it would erase crypt metadata too.
you may want to try using iter 1 first for practicing while wait for me to rewrite better guide.
$500 crypto to anyone who can clear this guide and publish a idiot proof spoon fed, step by step guide on how to do detached boot and header thing
I am 100% down to match that offer so bounty is $1000
correct me if i am wrong but there is still the problem with our current new disks ssd / nvme and iirc even newer hdds having the key remaining in their little extra cache after use.
Forgive me that i do not have the correct technical terms for all this but i think as long as we do not know how this caches work and how to proper clear them with each shutdown, no one should feel 100% secure, of course it all is depending on the individual thread vector.
Instead of creating luks header on disk with:
[anaconda /] cryptsetup luksOpen /dev/nvme0n1 luks
And later removing it from disk as described in this guide you can just create luks with specified key file:
cryptsetup --key-file=/dev/mapper/lukskey --keyfile-offset=offset --keyfile-size=size luksFormat /dev/sda --offset 32768 --header /mnt/header.img
cryptsetup open --header /mnt/header.img --key-file=/dev/mapper/lukskey --keyfile-offset=offset --keyfile-size=size /dev/sda enc
Then there won’t be any traces left on the disk.
https://wiki.archlinux.org/title/Dm-crypt/Specialties#Encrypted_/boot_and_a_detached_LUKS_header_on_USB
Also it’s better to specify drives by their ID instead of arbitrary name like /dev/nvme0n1 or /dev/sda3:
cryptsetup open /dev/disk/by-id/usbdrive-part2 cryptboot
cryptsetup --header /mnt/header.img --key-file=/dev/mapper/lukskey --keyfile-offset=‘‘offset’’ --keyfile-size=‘‘size’’ open /dev/disk/by-id/harddrive enc
usbdrive is your USB drive by-id, and harddrive is your main hard drive by-id.
Also correction to the guide:
You need to leave 5-10% free space in VG in case it’ll be needed to extend pools:
lvcreate -T -l +90%FREE qubes_dom0/vm-pool
Why doesn't Qubes 4.1 use full disk space (LVM)? - #3 by marmarek
Best Way To Resize Dom0 Pool? - #9 by brendanhoar
LVM metadata problem - Free space in thin pool reached threshold - #3 by user-a9dc9a0d
you are correct, that’s why i decided to write full guide and using tmpfs for dom0 and dispvm. that would be part 4 of Running dom0 and DispVM in tmpfs.
the problem is if we did detach header before installation, anaconda would’nt understand that. try it, it would fail.
that’s correct, in my new quick install here, I already leave 10% free space
Just donate to qubes and post it here, I’m sure I will prioritize this, and complete the Playing with Qubes.
i don’t see how that would protect from the unknowns of the manufacturer on disk device caching
Deleted, got it wrong.