Greetings Qubes community, I am configuring a setup based on the one described here (Qubes OS Installation - Detached encrypted boot and header) with some minor changes. I am trying to implement what is described in this comment (My personal experience of attempt to harden qubes VM - #7 by zphaskqjrm), using a minimal Linux distribution installed onto a usb drive. The main drive that Qubes is installed uses LUKS full disk encryption and has a detached header so that the encrypted contents of the drive are indistinguishable from random data. The header and the keyfile needed to unlock it are embedded into the initramfs, which is stored on an encrypted partition on the usb drive along with vmlinuz and the Xen gzip archive. The computer boots the minimal Linux installation on the usb drive, which asks the user to enter in the password to unlock the encrypted partition with the initramfs. Then the initramfs, vmlinuz, and xen.gz are loaded into ram, the encrypted partition is closed, and kexec -e is ran to boot into Qubes. The purpose of this setup is to allow greater flexibility in encryption schemes and password input. I have already tested this setup using virtual machines but booting into Debian instead of Qubes. I’ve encountered smooth operation during testing with Debian.
The problem I am having with Qubes is that I am not able to kexec from a Linux installation into Qubes.
These are what I have ruled out as the likely issue:
The kexec-tools installation on the usb drive: I am able to kexec into Debian when testing with virtual machines.
The files loaded into ram or an error in the process of decrypting the main disk with the Qubes installation: I am able to load the files into ram and kexec successfully from the Heads recovery shell. The initramfs is able to decrypt the main disk and Qubes boots successfully. The trouble I am having is with doing this from the other Linux installation on the usb drive.
The Qubes installation itself: Qubes runs like normal with no problems when using kexec from the Heads recovery shell.
Video drivers or other programs on the usb Linux installation: I am still unable to kexec into Qubes even if no video drivers, Xorg, or other programs are installed on the usb Linux installation. Kexec is able to load the files into ram without issue, booting into Qubes fails.
When trying to boot into Qubes the screen goes completely black. After a short pause of a couple seconds the computer returns to the Heads BIOS menu. There are no error messages or other output shown.
These are the commands that are successful from the Heads recovery shell and fail when running from Linux.
kexec -l xen.gz --command-line=“placeholder console=none dom0_mem=min:1024M dom0_mem=max:4096M ucode=scan smt=off gnttab_max_frames=2048 gnttab_max_maptrack_frames=4096 no-real-mode edd=off” --module=“vmlinuz placeholder root=/dev/mapper/qubes_dom0-root ro rd.lvm.lv=qubes_dom0/root rd.lvm.lv=qubes_dom0/swap rd.lvm.lv=qubes_dom0/boot plymouth.ignore-serial-consoles i915.alpha_support=1 intel_iommu=igfx_off rd.driver.pre=btrfs rhgb quiet” --module=“initramfs.img”
kexec -l -t multiboot-x86 --append=“placeholder console=none dom0_mem=min:1024M dom0_mem=max:4096M ucode=scan smt=off gnttab_max_frames=2048 gnttab_max_maptrack_frames=4096 no-real-mode edd=off” --module=“vmlinuz placeholder root=/dev/mapper/qubes_dom0-root ro rd.lvm.lv=qubes_dom0/root rd.lvm.lv=qubes_dom0/swap rd.lvm.lv=qubes_dom0/boot plymouth.ignore-serial-consoles i915.alpha_support=1 intel_iommu=igfx_off rd.driver.pre=btrfs rhgb quiet” --module=“initramfs.img” xen.gz
Booting without no-real-mode fails even from the Heads recovery shell
Interestingly, the following command succeeds in booting into Qubes from Linux, but Qubes is not functional due to missing the xen.gz file. This makes me think the problem is with Xen, and not the Heads BIOS.
kexec -l vmlinuz --append=‘root=/dev/mapper/qubes_dom0-root ro rd.lvm.lv=qubes_dom0/boot rd.lvm.lv=qubes_dom0/root rd.lvm.lv=qubes_dom0/swap plymouth.ignore-serial-consoles i915.alpha_support=1 intel_iommu=igfx_off rd.driver.pre=btrfs rhgb quiet’ --initrd=initramfs.img
kexec -e is used to boot for all commands
Can anyone help me solve this issue? Hopefully the solution is something simple like a command line parameter. It would be nice if I could even get some error messages to help me debug this instead of just a blank screen. Can the xen.gz file be loaded once in Qubes? It would also work if Heads can boot into the files loaded by kexec after a full shutdown and reboot.