After doing this setup multiple times now, I have written a far more detailed guide on how to manually setup your entire disk from scratch, including manual selection of encryption cyphers & iterations.
This is useful if you are building your system in a way that is different to standard Qubes Install, for example in a Dual Boot setup or just leave some space free at the end for future use, since by default Qubes will consume the entire disk you install to.
# Manual Disk Partitioning for Custom Installation of Qubes OS 4.2
# The following commands can either be run from the Qubes OS GUI Installer, the rescue mode, or a 3rd party boot tool/os that runs from memory.
# Step 0 assumes it's being run from the Qubes installer.
# Step 0: Switch to the terminal from the installer
# Press CTRL + ALT + F2 to access the terminal
# Step 1: View current partitions
lsblk # List all block devices
# Step 2: Reinitialize the drive as GPT to wipe all partitions
gdisk /dev/nvme0n1 # Open gdisk on the target drive
# Press 'o' to create a new empty GPT partition table
# Press 'n' to create a new partition
# 1. For Partition 1:
# - Enter: '1' (Partition Number)
# - First sector: Press 'Enter' (default)
# - Last sector: '+922M' (900MB size) # making this a little larger than the default 600M since it holds 3 kernels
# - Hex code: 'EF00 (EFI System)' - use 0700 for "basic data" (blank partition, Qubes will format it later)
# Press 'n' to create another partition
# 2. For Partition 2:
# - Enter: '2' (Partition Number)
# - First sector: Press 'Enter' (default)
# - Last sector: '+1024M' (1GB size)
# - Hex code: '8300 (Linux Filesystem for /boot)'
# Press 'n' to create another partition
# 3. For Partition 3:
# - Enter: '3' (Partition Number)
# - First sector: Press 'Enter' (default)
# - Last sector: Press 'Enter' (use remaining space)
# - Hex code: '8E00 (Linux LVM)'
# Press 'w' to write changes and exit
# Step 3 PREP:
cryptsetup benchmark # If performance is important, run this to determine the fastest
# encryption protocol specific to your system and change
# the below -c command to match it
# Step 3: Set up LUKS encryption with Argon2id and 10,000 iterations
cryptsetup luksFormat -c aes-xts-plain64 -s 512 --pbkdf argon2id -i 5000 -y --use-random /dev/nvme0n1p3
# This uses AES-XTS-512, Argon2id KDF with 5,000 iterations, and entropy (from --use-random)
# The # of iterations (defined by -i 5000) can also be set in "time" instead using: '--iter-time 5000' (5 seconds)
# Step 4: Open encrypted partition
cryptsetup open /dev/nvme0n1p3 luks # Open encrypted partition as 'luks'
# Step 5: Set up LVM on encrypted partition
pvcreate /dev/mapper/luks # Create physical volume
vgcreate qubes_dom0 /dev/mapper/luks # Create volume group
# Step 6: Create logical volumes
lvcreate -n swap -L 4096M qubes_dom0 # Create 4096MB (4GB) swap volume
# Step 7: Create thin pools
lvcreate -T -L 40960M qubes_dom0/root-pool # Create 40960MB (40GB) thin pool for root
lvcreate -T -l 99%FREE qubes_dom0/vm-pool # Create the 'vm-pool' thin pool with 99% of the available free space
# ^ 99% to leave a useful amount of free space for possible repair operations in future, feel free to use 100% instead
# Step 8: Check the size of the new vm-pool
lvs -o +lv_size --units m # Get the exact size of vm-pool (needed for optional step 9b)
# Step 9: Create logical volumes from thin pools
lvcreate -V40960M -T qubes_dom0/root-pool -n root # Create 40960MB (40GB) root logical volume
# Step 9b: Create vm-pool and give it alias 'vm'
#lvcreate -V<exact_size>M -T qubes_dom0/vm-pool -n vm # Replace <exact_size> with the size obtained from 'lvs'
# ^ this line may not be necessary, have left it here for reference in case it is
# Step 10: Format the 'root' and 'vm' volumes
mkfs.ext4 /dev/qubes_dom0/root # Format 'root' logical volume as EXT4
#mkfs.ext4 /dev/qubes_dom0/vm # Format 'vm' logical volume as EXT4
# ^ above line from old instructions - this is a 'thin_pool' managed by Qubes as such is not intended to be formatted at all
# Step 11: Prepare the swap volume
mkswap /dev/qubes_dom0/swap # Mark the swap volume as swap space
# Step 12: Deactivate all logical volumes
vgchange -an qubes_dom0 # Deactivate all logical volumes in the volume group
# Step 13: Close the LUKS device
cryptsetup close luks # Close the LUKS device
# Step 14: Return to installer
# Press CTRL + ALT + F6 to return to the Qubes OS installer GUI
# Recommended to reboot system before resuming installation
Great !
I’m still struggling with “the perfect” partitionning, pondering between BTRFS, EXT4, LVM2, …
Question: You mention 922M = 900MB, but I can’t find what that first “M” refers to as none (that I found) equals to 900 MB
For critics (WIP, to be continued), please:
NVMe0n1 256GB for boot + dom1 (sysrun: swap, tmp, log)
NVMe1n1+NVMe2n1 2TB RAID1 for dom0 and domU
# Manual Disk Partitioning for Custom Installation of Qubes OS 4.2.x
# The following commands can either be run from the Qubes OS GUI Installer,
# the rescue mode, or a 3rd party boot tool/os that runs from memory.
# Step 0 assumes it's being run from the Qubes installer.
# Step 0: Switch to the terminal from the installer
# Press CTRL + ALT + F2 to access the terminal
# Step 1: View current partitions
=> lsblk # List all block devices
# Step 2: Reinitialize the drive as GPT to wipe all partitions
# A: NVME0n1
=> gdisk /dev/nvme0n1 # Open gdisk on the target drive
=> o # to create a new empty GPT partition table
=> n # to create a new partition (NVMe0n1p1)
=> 1 # For selecting Partition 1:
# - First sector: Press 'Enter' (default)
# - Last sector: '+1024M' (1GB size) # A little larger than default 600M since 3+ kernels
# - Hex code: 'EF00 (EFI System)' - use 0700 for "basic data" (blank, Qubes will format it later)
=> n # to create another partition (NVMe0n1p2)
=> 2 # For Partition 2:
# - First sector: Press 'Enter' (default)
# - Last sector: '+2048M' (2GB size)
# - Hex code: '8300 (Linux Filesystem for /boot)'
=> n # to create another partition (NVMe0n1p3)
=> 3 # For selecting Partition 3:
# - First sector: Press 'Enter' (default)
# - Last sector: '+168G' (56+24+24+64=168GB) Press 'Enter'
# - Hex code: '8E00 (Linux LVM)'
=> n # to create another partition (NVMe0n1p4)
=> 4 # For selecting Partition 4:
# - First sector: Press 'Enter' (default)
# - Last sector: Press 'Enter' (use all remaining)
# - Hex code: 'a504 (FreeBSD)' #(reserved for ZFS one day)
=> w # to write changes and exit
# B: NVME1n1+NVMe2n1 !! Need further intel to make it RAID1
=> gdisk /dev/nvme1n1 # Open gdisk on the target drive
=> o #to create a new empty GPT partition table
=> n # to create a new partition (NVMe1n1p1)
=> 1 # For selecting Partition 1:
# - First sector: Press 'Enter' (default)
# - Last sector: '+720G' # 80+640=720G
# - Hex code: 'fd00' (Linux RAID)
=> w # To write changes and exit
=> lskblk # To get the new partition names
# for RAID, changing from gdisk to mdadm
=> mdadm --create /dev/md10 --level-raid1 --raid-devices=2 /dev/XXX /dev/YYY
# mdadm: defaulting to version 1.2 metadata
# mdadm: array /dev/md10 started
=> lsblk /dev/md10 # To check/confim details
# Create partitions on RAID
=> sudo parted -s -a optimal -- /dev/md10 mklabel gpt
# First partition (for root)
=> sudo parted -s -a optimal -- /dev/mda0 mkpart primary 0% 11%
=> sudo parted -s -a optimal -- /dev/mda0 mkpart primary 11% 100%
sudo parted -s -- /dev/md10 align-check optimal 1
# to print all partition in the RAID:
=> ls /dev/md10
=> lsblk /dev/md10*
# LVM on RAID
# A, create a PV physical volume
# first partition (for root)
sudo pvcreate /dev/md10p1
# second partition (for VM)
sudo pvcreate /dev/md10p2
# B, create a VG volum group
=> sudo vgcreate dom0 /dev/md10p1
=> sudo vgcreate domU /dev/md10p2
# OR, sudo vgcreate dom0 /devmd10p1 /dev/md10p2
# C, create a LV logical volume
=> lvcreate -n <LVname1> -l 100%FREE dom0
=> lvcreate -n <LVname2> -l 100%FREE domU
# OR, => lvcreate -n <LVname1> -L 80G dom0
# AND=> lvcreate -n <LVname2> -L 640G dom0
# D, create a FS on these partitions
=> sudo mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0 /dev/dom0/<LVname1>
=> sudo mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0 /dev/domU/<LVname2>
#OR, => sudo mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0 /dev/dom0/<LVname1>
#AND=> sudo mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0 /dev/dom0/<LVname2>
#E, finaly, mounting these partitions
=> sudo mount -t ext4 /dev/dom0/<LVname1> /mnt
=> sudo mount -t ext4 /dev/domU/<LVname2> /mnt
#OR, => sudo mount -t ext4 /dev/dom0/<LVname1> /mnt
#AND=> sudo mount -t ext4 /dev/dom0/<LVname2> /mnt
**
> !! To be continued/corrected
**
# Step 3 Crypto PREP:
=> cryptsetup benchmark # If performance is important, run this to determine the fastest
# encryption protocol specific to your system and change
# the below -c command to match it
# Step 3b Crypto: Set up LUKS
=> cryptsetup luksFormat -c aes-xts-plain64 -s 512 --pbkdf argon2id -i 5000 -y --use-random /dev/md10
# This uses AES-XTS-512, Argon2id KDF with 5,000 iterations, and entropy (from --use-random)
# The N# of iterations (defined by -i 5000) can also be set in "time" instead using: '--iter-time 5000' (5 seconds)
# Step 4: Open encrypted partition
=> cryptsetup open /dev/md10 luks # Open encrypted partition as 'luks'
# Step 4.1: Set up LVM on encrypted partition
=> pvcreate /dev/mapper/luks # Create physical volume
=> vgcreate qubes_sysvol /dev/mapper/luks # Create volume group for sys-run
# Step 4.2: Create thin pool
=> lvcreate -n swap -L 57344M qubes_dom1 # Create 56GB swap volume
=> lvcreate -n /tmp -L 24576M qubes_dom1 # Create 24GB temp1 volume
=> lvcreate -n /var/tmp -L 24576M qubes_dom1 # Create 24GB temp2 volume
=> lvcreate -n /var/log -L 65536M qubes_dom1 # Create 64GB log volume
# Step 5: Open encrypted partition !! need more intel for RAID1
=> cryptsetup open /dev/md10 luks # Open encrypted partition as 'luks'
# Step 5.1: Set up LVM on encrypted partition
=> pvcreate /dev/mapper/luks # Create physical volume
=> vgcreate qubes_root /dev/mapper/luks # Create volume group for root
# Step 5.2: Create thin pool
=> lvcreate --type raid1 -m 1 -T -L 81920M -n qubes_dom0/root-pool # Create 80GB 'root' thin pool for root
=> lvcreate --type rvgchange -an qubes_dom0 # Deactivate all logical volumes in the volume groupaid1 -m 1 -T -l 100%FREE qubes_dom0/vm-pool # Create 404GB 'vm-pool' thin pool (could be 99% to spare some for potential repair operation)
# Step 6: Check the size of the new vm-pool
=> lvs -o +lv_size --units m # Get the exact size of vm-pool (needed for optional step 9b)
# Step 7: Create logical volumes from thin pools
=> lvcreate -V81920M -T qubes_dom0/root-pool -n root # Create 80GB root logical volume
# Step 9b: Create vm-pool and give it alias 'vm'
=> lvcreate -V413696M -T qubes_dom0/vm-pool -n vm # exact size obtained from step6
# Step 8: Format the 'root' and 'vm' volumes
mkfs.ext4 /dev/qubes_dom0/root # Format 'root' logical volume as EXT4
#mkfs.ext4 /dev/qubes_dom0/vm # Format 'vm' logical volume as EXT4
# ^ above line from old instructions - this is a 'thin_pool' managed by Qubes as such is not intended to be formatted at all
# Step 9: Prepare the swap volume
mkswap /dev/qubes_sysvol/swap # Mark the swap volume as swap space
# Step 10: Deactivate all logical volumes
vgchange -an qubes_dom0 # Deactivate all logical volumes in the volume group
vgchange -an qubes_sysvol # Deactivate all logical volumes in the volume group
# Step 12: Close the LUKS device
cryptsetup close luks # Close the LUKS device
# Step 13: Return to installer
# Press CTRL + ALT + F6 to return to the Qubes OS installer GUI
# Recommended to reboot system before resuming installation
I wanted to give HALF of my SSD to Qubes once, and leave the other half empty. The customizable setup wanted me to individually specify all of the internal partitions, and I didn’t want to change any of them. But unfortunately I had no idea what the normal settings were.
I ended up having to partition the SSD in two halves by a different method (gparted on another system); then it was easy enough to just let Qubes have the first of the two partitions.
I ended up doing it all using the GUI.
Select all three NVMe
Do Not use the “advanced custom (Blivet-GUI)” but the “custom”
On NVMe0n1: Created /boot/efi + /boot + Thin LVM dom2 64GB (size=automatic) swap + Thin LVM dom2 24GB /tmp + Thin LVM dom2 24GB /var/tmp + Thin LVM dom2 80GB /var/log
Then I can’t create any other part on any of the other disks …
So I started over:
On NVMe0n1: Created /boot/efi + /boot
On NVMe1n1: Created Thin LVM dom0 80GB /; selected RAID1; size=auto (dom0-root)
On NVMe1n1: Created Thin LVM dom1 424GB /var/lib/qubes; selected RAID1; size=auto (dom1-var_lib_qubes)
On NVMe0n1: Created Thin LVM dom2 64GB swap + Thin LVM dom2 24GB /tmp + Thin LVM dom2 24GB /var/tmp + Thin LVM dom2 80GB /var/log size=auto (dom2-var_lib_log)
Worked to the second screen !
Selected options as usual (default Debian, sys-net disposable, whonix updates) Started process then an error pops-up:
Dom0 error (on dom0)
['usr/sbin/lvcreate','-l','90%FREE','--thinpool','pool00','qubes_dom0'] failed:
stdout;" "
stderr:" Logical volume pool00 already exist in Volume group qubes_root. "
TY, yeah, I figure it out later … I first started with a completely outstreched search, with GiB vs GB, MB vs MiB, … way too far, the answer was right in front
While at it, if you have time, please review and critique my post above ?
The problem is that the manual partitioning works in a counter-intuitive way and there aren’t any hints in that area to explain what’s happening. So if you use the automatic tools for the boot section, and have another partition you aren’t using, and then add the main Qubes partition, Qubes will list it as only taking up part of the space, like it’s not going to take up that whole partition. But that section becomes varilibqubes and the remaining space that isn’t listed becomes vm-pool during the second part of the installation during the reboot. The ambiguity has probably caused a lot of frustration and a few simple words in that area could make it easier. That’s my view on it.
If someone wants to add to that issue with suggested wording for “help” that can be put into the manual partitioning GUI, it would be cool because my suggestion is sort of vague. I think the manual partition GUI is possibly much easier if there were better instructions.
