Hello all.
I find myself in a fairly unfortunate situation:
Some time ago, I reinstalled Qubes with a detached encrypted boot and luks header following this guide:
All was good up until yesterday when I performed updates of dom0. I believe some package was updated that tried to mess with initramfs but since the boot disk was detached it wasn’t able to do it properly.
At startup the boot is decrypted, then the classic Qubes decryption screen appears (obviously without the password prompt, as that’s usually handled with a saved key) and then black screen with the following:
Warning: /dev/mapper/qubes_dom0-root does not exist
Warning: /dev/qubes_dom0/root does not exist
Warning: /dev/qubes_dom0/swap does not exist
Generating "/run/initramfs/rdsosreport.txt"
Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
Give root password for maintenance
(or press Control-D to continue):
As I am not familiar with the grub rescue, I would normally boot from a live system, mount everything and chroot into the host but as the luks header is detached from my Qubes installation, I’m only able to mount /boot.
I am lost, please advise. ANY help will be greatly appreciated.
First, don’t panic. I am almost sure everything is there and it’s just about regenerating grub and I am sure @51lieal could do his best to help you (sorry for tagging you).
Just consider changing the topic to something more attractive, like
Cannot boot Qubes with detached boot and header
to its better visibility, and also try some search. These are my limits regarding the issue.
About that… During the installation of Qubes, when chrooted I set the following params in order to speed up and already excruciating slow boot:
GRUB_TIMEOUT=0
GRUB_TIMEOUT_STYLE=hidden
So now pressing e does nothing.
However I decrypted the detached boot partition and checked /boot/grub2/ and to my surprise I did not find grub.cfg.
Should I decrypt everything on live, mount /boot into /dev/qubes_dom0/root and generate a new grub.cfg while chrooted? If so, could you indicate me the best way?
Your boot config is in EFI partition (/boot/efi/EFI/qubes/grub.cfg), not in /boot/grub2,
So after you decrypt your /boot and mount it to /mnt, you should mount your efi to /mnt/efi.
I’ve did this once and resolving this wouldn’t be that hard.
Can you confirm /boot folder in /qubes_dom0/root already removed ?
Try boot again and if you still entering dracut emergency, please provide /run/initramfs/rdsosreport.txt it actually tell us how it boot so it could give a hint of what happen.
No, remember we are doing detached boot, so It’s should be nothing in /boot folder.
Everything is done in your flashdrive.
What I meant was mounting /dev/qubes_dom0/root to /mnt then mounting /dev/mapper/luksboot (from the decrypted boot partition of the flashdrive) to /mnt/boot: mount -o rw /dev/mapper/luksboot /mnt/boot.
But as I now found grub.cfg in the efi partition this should no longer be necessary.
I think there’s a problem in your sd card based on the image you share, I’ve confirm that upgrading kernel with or without detached drive will not affect anything, except that you already installed kernel.
Can you share the full log on the dracut emergency ?
Wait what exactly made you think that, because this was the right call. I restored a backup I had made of the sd card and just like that the system booted up correctly.
Seriously, thanks a lot for the help and the time you dedicated me.
But now I wonder, what’s the right way to update grub and initramfs with a detached boot?
Because everything is boot as normal, first boot it called your efi configuration, second it asking you to decrypt the password (in that case, its absolutely your configuration), then in the log i’ve seen that it was looking for /dev/mmcblk.
Try using good quality of sd card, as far i know, 4 - 8 gb using low grade chip.
and for your backup, it’s okay to have a backup in the /dev/qubes_dom0/root, no need to add additional drive.
There’s no easy way, just make sure sys-usb is not automatically started when you want to upgrade kernel.
What I meant is, since a /boot exists when the system is on, would I need to delete it, mount the sd card on /boot and then update grub/initramfs? Or is there another procedure?
if your detached boot drive never been unmounted, then just do as normal, and you don’t need to update grub / initramfs, it would be done automatically.
I’ve re read your situation in the #1 perhaps you using sys-usb ? remember that when you using sys-usb, it’s passing the usb controller to sys-usb and that would umount your detached boot drive. That’s why i said you need to turn off auto start sys-usb when you want to upgrade kernel.
And for your situation now if you want to upgrade kernel,
boot as normal.
umount your detached boot.
make sure it already removed.
uninstall recently installed kernel, then remove everything on the /boot rm -rf /boot/*
turn off auto start sys-usb then reboot.
boot as normal, install new kernel then turn on auto start sys-usb and reboot (or you can reboot first to use newer kernel, remove old kernel, then turn on auto start sys-usb and reboot).
I really keep it clean, so i always switch to new kernel then remove old kernel.
I do use sys-usb but the sdcard controller is not attached to it, it’s attached to dom0. I’ve been highly reluctant to attach it to sys-usb as I’m afraid it’ll make my system unbootable, since I have to decrypt the root partition with the sd card.
I have a correctly working Qubes installation based on the detached headers tutorial (UEFI). Everything is working fine.
However, I have had problems in two situations:
When using a sys-usb, the system won’t boot correctly.
In addition to sys-usb, using rd.qubes.hide_all_usb in my grub configuration will have the same effect. Does not boot.
Sorry for not putting up more details on the failures ( can’t remember in which case I’m landing on dracut shell, and in which one I just get straight to boot). Quite reluctant to retry without some advices first.
So a couple questions, hopefully the good ones:
Is it possible at all to use rd.qubes.hide_all_usb in this setup ? Would be great since there is quite no point to use sys-usb otherwise.
About the need to turn-off sys-usb autostart before upgrading, has someone found a better/automated solution ? One again, being quite busy/lazy I have not tried to write any such script.
Wondering if anyone has had the same problems.
Also, my efi/boot/headers are on a USB drive, in case it makes any difference, but I can’t see which one…
Grateful for any support, and thanks a lot for these great tutorials !