Hello all
I know this is a problem that many people have come across as I’ve been reading through answers on here and the internet at large and also trying to make sense of the official Qubes documentation. The truth is I’m not an expert at linux and command line configuration and many of the instructions in solutions I’ve read have just left me with an “ok, so how do I do that?” dead end.
So the problem is I have installed qubes from an image on a usb stick. The installations appears to go without a problem and I’m given the all clear to power down my machine and I should boot into qubes. If I leave the usb stick in when I do this I’m taken straight back to booking from the stick and back to the installation menus. If I remove it the system won’t boot.
Hardware: MSI Z170 A Pro
I belive I have set all the UEFI up as directed on the official website
I think it has something to do with boot partitions or something but it’s all a bit beyond me. Oh I also tried to get into my UEIF boot menu on start up and it doesnt seem that a bootable hard disk partition is there. I used the automatic partitioning settings on the installer as I wasnt sure how to set it up manually.
Adivice very greatfully received
Try to fix your EFI boot entry like this first:
In any situation, if somewhen your bootable device is not found on grub.
Try booting into qubes os installation media and boot into rescue qubes os entry.
Just skip to shell :
Then run efibootmgr -v -c -u -L Qubes OS -l /EFI/qubes/grubx64.efi -d /dev/nvme0n1 -p 1
-d = where your bootloader device remain.
could be : /dev/sda /dev/sdb /dev/nvme0n1 /dev/nvme1n1
-p = which partition.
you can try run lsblk first before running above command.
If it won’t help then try to use EFI fallback path:
opened 09:26AM - 02 Jun 22 UTC
T: enhancement
P: default
pr submitted
C: boot
[How to file a helpful issue](https://www.qubes-os.org/doc/issue-tracking/)
#… ## Qubes OS release
Latest as of posting, 4.1.0
### Brief summary
I manually unplugged every drive on my computer except for the target drive for the installation, which was all free space unpartitioned. Installer ran typically, chose automatic for the installation type. Booted into the OS' configuration and then the full OS.
GRUB is verified to have worked because I managed to see it when initially booting for configuration and because of a later restart due to a display artificating and then system lockup problem I had that I attribute to Nvidia.
After using another drive and then unplugging it, and plugging the Qubes drive back, there was no detectable boot source.
I booted a live OS and found that the partition structure was intact but didn't investigate further and reinstalled Qubes.
I then got the idea to see if this issue happened if I BIOS disabled drives instead of unplugging them. I disabled my Qubes drive and enabled another, and booted into it fine. I then disabled my other drive and reenabled the Qubes drive and the same issue happened again where my system does not see any boot entries.
The system does see the drive, and switching on CSM and legacy boot I can boot from the drive but get an error screen saying to boot from proper media.
This second time, I _kind of_ fixed it but I must do these steps every time I remove the drive or disable it in BIOS. I lose the option to boot without the Xen hypervisor and the boot hiccups. I essentially have to boot my install media, run Anaconda rescue and:
1. Mount the bootloader partition
2. Copy the contents of /mnt/EFI/qubes/ to /mnt/EFI/BOOT/
3. Rename grubx64.efi to bootx64.efi
4. Rename grub.cfg to bootx64.cfg
5. efibootmgr -v -c -u -L Qubes2 -l /EFI/BOOT/bootx64.efi -d /dev/sda -p 1
### Steps to reproduce
I believe the best way to reproduce this issue is to do a fresh install of Qubes on hardware similar to mine:
Asus Prime Z-390-A Motherboard
Intel i7-9700K CPU
16GB DDR4 RAM
1TB HDD
Most notably, this motherboard. I will be filing a compatibility report soon, but I believe this motherboard may have something to do with how it reads the GRUB bootloader as it's configured for Qubes.
Further, these are the only non-stock BIOS settings:
- Disabling the Intel LAN Controller (it causes a PCI reset error and ends the last step of the configuration, doesn't matter since I don't use it, and I believe the fact that it's unplugged is the reason this issue happens)
- Enabling Virtualization
- Enabling Vt-d
- Disabling all other drives
### Expected behavior
The GRUB bootloader is supposed to appear after the BIOS initializes regardless of whether the drive was previously unplugged from the system or not (safely of course).
### Actual behavior
Please read above.
After unplugging the drive or disabling it in BIOS, the entry for GRUB is missing.
opened 02:53PM - 15 Jul 23 UTC
T: bug
P: default
needs diagnosis
affects-4.1
C: boot
### Qubes OS release
R4.1 and R4.2
### Brief summary
When making a raw … disk backup from a Qubes installed to an internal hard drive to an external hard drive, the external hard drive is unbootable.
raw disk backup means a backup using `dd` or 1 to 1 exact copy.
### Steps to reproduce
1. install Qubes normally on a computer that only support EFI booting on the internal harddrive
2. reboot
3. brief test that Qubes is working normally (yes)
4. shutdown Qubes
5. boot from an external drive, boot a live DVD or live USB such as Debian Live
6. perform a raw disk backup from the internal disk to the external disk
7. unplug that disk and try to boot from it in a different computer (or the same Qubes computer)
([example instructions for raw disk backups](https://www.kicksecure.com/wiki/Raw_Disk_Backup))
### Expected behavior
The raw disk backup of Qubes is bootable.
### Actual behavior
The raw disk backup of Qubes is unbootable.
### Additional information
According to my research that might be because of missing entries in the EFI firmware's NVRAM which is stored on the motherboard. Unfortunately, the EFI boot process doesn't seem by default to be self-contained on 1 harddrive but require extra settings stored outside harddrives on the motherboard (EFI NVRAM).
Using `grub2-install` with options `--removable` / `--force-extra-removable` during Qubes installation might help?
> [--removable](https://manpages.debian.org/bookworm/grub2-common/grub-install.8.en.html#removable)
> the installation device is removable. This option is only available on EFI.
> [--force-extra-removable](https://manpages.debian.org/bookworm/grub2-common/grub-install.8.en.html#force~2)
> force installation to the removable media path also. This option is only available on EFI.
Qubes does not have good support for multiboot support anyhow:
* https://github.com/QubesOS/qubes-issues/issues/8351
* https://www.qubes-os.org/faq/#can-i-install-qubes-os-together-with-other-operating-system-dual-bootmulti-boot
* It's "patches welcome". (And that's okay.)
This ticket is not a feature request to improve multiboot support. Why do I mention this? Because otherwise, when considering options `--removable` / `--force-extra-removable` one *might* argue "but that breaks mutliboot support". I would argue that being able to boot a raw disk backup of Qubes is more important than mutliboot support.
[Why do I like full raw disk backups? See this link.](https://forum.qubes-os.org/t/how-do-you-organize-your-backups/3986/16?u=adrelanos)
Manually updating the NVRAM can be challenging:
* It's vendor dependent because of many different EFI BIOS versions.
* Some BIOS don't even have such an option.
* Might require booting into an operating system installed on USB and running console commands to fix it.
* Difficult, impossible for far most users to do or even to find instructions for it.
Qubes with legacy BIOS booting as far as I remember didn't have this issue. However, never notebooks sometimes (or often, dunno) don't even support legacy BIOS booting anymore. Therefore "use legacy BIOS booting" is a non-solution. Also not a good long term solution in either case because of Qubes planned Secure Boot support.
This bug might also make Qubes "non-portable". Meaning, Qubes installed to an external drive such as a USB SSD might be bootable on the computer where it was installed but unbootable when attempting to boot the same Qubes USB SSD on another computer. I didn't test this very part mentioned in this very chapter. Anaconda by Fedora might be using `grub2-install` with options `--removable` / `--force-extra-removable` already when installing to external devices (USB) instead of internal harddrive.
Where is the Qubes code which defines bootloader / grub2 / EFI installation? Or is this currently all done by upstream's Anaconda?
1 Like
Sorry mistake, i replied to the wrong thread. I did the lsblk command and it spat that out. Im just not sure what the instructions meant and what i should put into the command
Run this command:
efibootmgr -v -c -u -L 'Qubes OS' -l /EFI/qubes/grubx64.efi -d /dev/sda -p 1
Thanks Sorry some of this shell commands I find a bit difficult.
I ran the command, exited the shell and re booted. Sadly nothing changed and I was returned straigt into UEFI screen again
I dont undersand this EFI fallback thing you suggested. IS there anything else I could try?
You need to boot into Qubes OS installer and run these commands in shell:
mount /dev/sda1 /mnt
cp -r /mnt/EFI/qubes/* /mnt/EFI/BOOT/
mv /mnt/EFI/BOOT/grubx64.efi /mnt/EFI/BOOT/BOOTX64.efi
mv /mnt/EFI/BOOT/grub.cfg /mnt/EFI/BOOT/BOOTX64.cfg
umount /mnt
No other ideas for now.
I’ll do all that immediately. Thanks for your patience
balko
November 1, 2023, 1:59pm
9
Shouldn’t it have single quotes around Qubes OS
?
That appears to have done it and I’m not booted into Qubes.
I’m most greatful for your time. Many thanks
Yes, it should be in quotes. But I don’t know if it’ll work or not without them, need to be tested.
I’ll edit the command in my post.
1 Like
Sorry guys the “not” was a typo. I AM booted into qubes and have been playing with it so I guess the quotes werent needed.
Thanks again for all the help
1 Like
I marked @apparatus ’s post as the solution, please correct if I got it wrong @draylath !