Qubes boot fails after second restart

qubesuser2 · January 3, 2024, 1:09pm

Hey, thanks for any help in advance!!

I’m installing qubes os 4.2.0 on my HP probook 450 G10 on an external drive. On my main drive I have manjaro installed. The install works perfectly fine; The first phase works fine, then I reboot it and unplug the installation medium and the setup of all the template VMs works fine, and I get into a properly working qubes setup. If I then restart the system, however, then from that point onwards booting from the external drive quits immediatly and it starts booting from my main drive. I am sure the qubes os drive has higher boot priority. Even if I specifically instruct the bios to boot from my qubes drive, the computer immediatly returns to the bios.

I have tried to fix it by editing /etc/default/grub by adding efi=attr=uc to GRUB_CMDLINE_XEN_DEFAULT in dom0

I’m wondering how I can make qubeOS boot when booting of the qubeOS drive.

PS: UEFI troubleshooting | Qubes OS
seems to be outdated as xen.cfg no longer exists since 4.2.0

apparatus · January 3, 2024, 1:20pm

You need to use UEFI fallback path:

github.com/QubesOS/qubes-issues

Fall back to UEFI shell if Qubes boot entry is missing

opened 09:26AM - 02 Jun 22 UTC

mehrexe

T: enhancement P: default pr submitted C: boot

[How to file a helpful issue](https://www.qubes-os.org/doc/issue-tracking/) #…## Qubes OS release Latest as of posting, 4.1.0 ### Brief summary I manually unplugged every drive on my computer except for the target drive for the installation, which was all free space unpartitioned. Installer ran typically, chose automatic for the installation type. Booted into the OS' configuration and then the full OS. GRUB is verified to have worked because I managed to see it when initially booting for configuration and because of a later restart due to a display artificating and then system lockup problem I had that I attribute to Nvidia. After using another drive and then unplugging it, and plugging the Qubes drive back, there was no detectable boot source. I booted a live OS and found that the partition structure was intact but didn't investigate further and reinstalled Qubes. I then got the idea to see if this issue happened if I BIOS disabled drives instead of unplugging them. I disabled my Qubes drive and enabled another, and booted into it fine. I then disabled my other drive and reenabled the Qubes drive and the same issue happened again where my system does not see any boot entries. The system does see the drive, and switching on CSM and legacy boot I can boot from the drive but get an error screen saying to boot from proper media. This second time, I _kind of_ fixed it but I must do these steps every time I remove the drive or disable it in BIOS. I lose the option to boot without the Xen hypervisor and the boot hiccups. I essentially have to boot my install media, run Anaconda rescue and: 1. Mount the bootloader partition 2. Copy the contents of /mnt/EFI/qubes/ to /mnt/EFI/BOOT/ 3. Rename grubx64.efi to bootx64.efi 4. Rename grub.cfg to bootx64.cfg 5. efibootmgr -v -c -u -L Qubes2 -l /EFI/BOOT/bootx64.efi -d /dev/sda -p 1 ### Steps to reproduce I believe the best way to reproduce this issue is to do a fresh install of Qubes on hardware similar to mine: Asus Prime Z-390-A Motherboard Intel i7-9700K CPU 16GB DDR4 RAM 1TB HDD Most notably, this motherboard. I will be filing a compatibility report soon, but I believe this motherboard may have something to do with how it reads the GRUB bootloader as it's configured for Qubes. Further, these are the only non-stock BIOS settings: - Disabling the Intel LAN Controller (it causes a PCI reset error and ends the last step of the configuration, doesn't matter since I don't use it, and I believe the fact that it's unplugged is the reason this issue happens) - Enabling Virtualization - Enabling Vt-d - Disabling all other drives ### Expected behavior The GRUB bootloader is supposed to appear after the BIOS initializes regardless of whether the drive was previously unplugged from the system or not (safely of course). ### Actual behavior Please read above. After unplugging the drive or disabling it in BIOS, the entry for GRUB is missing.

github.com/QubesOS/qubes-issues

Raw disk backup of Qubes EFI is unbootable

opened 02:53PM - 15 Jul 23 UTC

adrelanos

T: bug P: default needs diagnosis affects-4.1 C: boot

### Qubes OS release R4.1 and R4.2 ### Brief summary When making a raw …disk backup from a Qubes installed to an internal hard drive to an external hard drive, the external hard drive is unbootable. raw disk backup means a backup using `dd` or 1 to 1 exact copy. ### Steps to reproduce 1. install Qubes normally on a computer that only support EFI booting on the internal harddrive 2. reboot 3. brief test that Qubes is working normally (yes) 4. shutdown Qubes 5. boot from an external drive, boot a live DVD or live USB such as Debian Live 6. perform a raw disk backup from the internal disk to the external disk 7. unplug that disk and try to boot from it in a different computer (or the same Qubes computer) ([example instructions for raw disk backups](https://www.kicksecure.com/wiki/Raw_Disk_Backup)) ### Expected behavior The raw disk backup of Qubes is bootable. ### Actual behavior The raw disk backup of Qubes is unbootable. ### Additional information According to my research that might be because of missing entries in the EFI firmware's NVRAM which is stored on the motherboard. Unfortunately, the EFI boot process doesn't seem by default to be self-contained on 1 harddrive but require extra settings stored outside harddrives on the motherboard (EFI NVRAM). Using `grub2-install` with options `--removable` / `--force-extra-removable` during Qubes installation might help? > [--removable](https://manpages.debian.org/bookworm/grub2-common/grub-install.8.en.html#removable) > the installation device is removable. This option is only available on EFI. > [--force-extra-removable](https://manpages.debian.org/bookworm/grub2-common/grub-install.8.en.html#force~2) > force installation to the removable media path also. This option is only available on EFI. Qubes does not have good support for multiboot support anyhow: * https://github.com/QubesOS/qubes-issues/issues/8351 * https://www.qubes-os.org/faq/#can-i-install-qubes-os-together-with-other-operating-system-dual-bootmulti-boot * It's "patches welcome". (And that's okay.) This ticket is not a feature request to improve multiboot support. Why do I mention this? Because otherwise, when considering options `--removable` / `--force-extra-removable` one *might* argue "but that breaks mutliboot support". I would argue that being able to boot a raw disk backup of Qubes is more important than mutliboot support. [Why do I like full raw disk backups? See this link.](https://forum.qubes-os.org/t/how-do-you-organize-your-backups/3986/16?u=adrelanos) Manually updating the NVRAM can be challenging: * It's vendor dependent because of many different EFI BIOS versions. * Some BIOS don't even have such an option. * Might require booting into an operating system installed on USB and running console commands to fix it. * Difficult, impossible for far most users to do or even to find instructions for it. Qubes with legacy BIOS booting as far as I remember didn't have this issue. However, never notebooks sometimes (or often, dunno) don't even support legacy BIOS booting anymore. Therefore "use legacy BIOS booting" is a non-solution. Also not a good long term solution in either case because of Qubes planned Secure Boot support. This bug might also make Qubes "non-portable". Meaning, Qubes installed to an external drive such as a USB SSD might be bootable on the computer where it was installed but unbootable when attempting to boot the same Qubes USB SSD on another computer. I didn't test this very part mentioned in this very chapter. Anaconda by Fedora might be using `grub2-install` with options `--removable` / `--force-extra-removable` already when installing to external devices (USB) instead of internal harddrive. Where is the Qubes code which defines bootloader / grub2 / EFI installation? Or is this currently all done by upstream's Anaconda?

qubesuser2 · January 3, 2024, 2:54pm

thank you so much! This worked for me. Is the cause of this the hardware that I’m using? Do you recommend mentioning that you have to do this on the entry I’m gonna make in the hardware compatibility list?

apparatus · January 3, 2024, 3:19pm

No, it’s not hardware-specific, it’s just how UEFI works. On boot the UEFI checks the entries and if the entry points to non-existing device (be it USB or some internal drive that was removed from the system) then it removes this entry.

Dogg · March 2, 2024, 8:53am

hi, i have the same issue except for some reason my /mnt/efi/BOOT does not exist in my boot loader partition anymore??? I don’t know much about coding but i’ve spent many hours trying to figure this out with this being where i’ve become stuck.

not just that, a lot of the file locations where i am told to go come back as not existing. this partition has to be the boot loader partition it has /mnt/efi/qubes but nothing else i really don’t know what to do.

Is there a way to wipe and reinstall the boot loader partition altogether?

I can elaborate more tomorrow it’s just really late i’ve been up all night trying to figure this out and my brain is completely fried

qubesuser2 · March 14, 2024, 9:46am

Hey, sorry for the late reply!! I hope you have either fixed it already or have not spent too much time on it. What apparatus said fixed it for me, though I don’t quite remember the exact steps (I agree, the thread he linked is a bit confusing)