Cannot boot Qubes with detached boot and header after update - need rescue

I’m able to decrypt it with the --header parameter but when I try to mount it I get the following error:

# mount /dev/mapper/luks /mnt
mount: /mnt: unknown filesystem type 'LVM2_member'.

I was instead able to mount the LV:

mount /dev/qubes_dom0/root /mnt

There was indeed a /boot folder with the following structure:

#ls -a /mnt/boot/
. .. efi  xen-4.14.4.config  xen-4.14.4.gz

I removed it, rebooted but the issue persisted.
Qubes still does NOT boot

Ah yes my bad, this right.

After decrypting /boot, you should be showed a grub menu, press ‘e’ then check how it would boot, or you can take a image and post it here.

About that… During the installation of Qubes, when chrooted I set the following params in order to speed up and already excruciating slow boot:

GRUB_TIMEOUT=0
GRUB_TIMEOUT_STYLE=hidden

So now pressing e does nothing.
However I decrypted the detached boot partition and checked /boot/grub2/ and to my surprise I did not find grub.cfg.

Should I decrypt everything on live, mount /boot into /dev/qubes_dom0/root and generate a new grub.cfg while chrooted? If so, could you indicate me the best way?

Your boot config is in EFI partition (/boot/efi/EFI/qubes/grub.cfg), not in /boot/grub2,
So after you decrypt your /boot and mount it to /mnt, you should mount your efi to /mnt/efi.

I’ve did this once and resolving this wouldn’t be that hard.
Can you confirm /boot folder in /qubes_dom0/root already removed ?

Try boot again and if you still entering dracut emergency, please provide /run/initramfs/rdsosreport.txt it actually tell us how it boot so it could give a hint of what happen.

No, remember we are doing detached boot, so It’s should be nothing in /boot folder.
Everything is done in your flashdrive.

1 Like

Yes, I removed it already

What I meant was mounting /dev/qubes_dom0/root to /mnt then mounting /dev/mapper/luksboot (from the decrypted boot partition of the flashdrive) to /mnt/boot:
mount -o rw /dev/mapper/luksboot /mnt/boot.
But as I now found grub.cfg in the efi partition this should no longer be necessary.

I’ll do it now and report back

Here’s the final portion when it started showing errors:
https://psty.io/r?q=69d98

The original is very long, let me know if it’s needed (if that’s the case, I’d rather send it privately as it contains more identifiers).

I didn’t bring my laptop today, so i’m not sure how it would resolve for your case.
I’ll publish my notes tomorrow.

1 Like

I think there’s a problem in your sd card based on the image you share, I’ve confirm that upgrading kernel with or without detached drive will not affect anything, except that you already installed kernel.

Can you share the full log on the dracut emergency ?

2 Likes

Wait what exactly made you think that, because this was the right call. I restored a backup I had made of the sd card and just like that the system booted up correctly.
Seriously, thanks a lot for the help and the time you dedicated me.

But now I wonder, what’s the right way to update grub and initramfs with a detached boot?

2 Likes

Because everything is boot as normal, first boot it called your efi configuration, second it asking you to decrypt the password (in that case, its absolutely your configuration), then in the log i’ve seen that it was looking for /dev/mmcblk.

Try using good quality of sd card, as far i know, 4 - 8 gb using low grade chip.
and for your backup, it’s okay to have a backup in the /dev/qubes_dom0/root, no need to add additional drive.

There’s no easy way, just make sure sys-usb is not automatically started when you want to upgrade kernel.

1 Like

What I meant is, since a /boot exists when the system is on, would I need to delete it, mount the sd card on /boot and then update grub/initramfs? Or is there another procedure?

if your detached boot drive never been unmounted, then just do as normal, and you don’t need to update grub / initramfs, it would be done automatically.

2 Likes

Got it, thanks a lot

I’ve re read your situation in the #1 perhaps you using sys-usb ? remember that when you using sys-usb, it’s passing the usb controller to sys-usb and that would umount your detached boot drive. That’s why i said you need to turn off auto start sys-usb when you want to upgrade kernel.

And for your situation now if you want to upgrade kernel,

  1. boot as normal.
  2. umount your detached boot.
  3. make sure it already removed.
  4. uninstall recently installed kernel, then remove everything on the /boot rm -rf /boot/*
  5. turn off auto start sys-usb then reboot.
  6. boot as normal, install new kernel then turn on auto start sys-usb and reboot (or you can reboot first to use newer kernel, remove old kernel, then turn on auto start sys-usb and reboot).

I really keep it clean, so i always switch to new kernel then remove old kernel.

2 Likes

I do use sys-usb but the sdcard controller is not attached to it, it’s attached to dom0. I’ve been highly reluctant to attach it to sys-usb as I’m afraid it’ll make my system unbootable, since I have to decrypt the root partition with the sd card.

And thank you lots for the step-by-step

Hello,

I have a correctly working Qubes installation based on the detached headers tutorial (UEFI). Everything is working fine.

However, I have had problems in two situations:

  • When using a sys-usb, the system won’t boot correctly.
  • In addition to sys-usb, using rd.qubes.hide_all_usb in my grub configuration will have the same effect. Does not boot.

Sorry for not putting up more details on the failures ( can’t remember in which case I’m landing on dracut shell, and in which one I just get straight to boot). Quite reluctant to retry without some advices first.

So a couple questions, hopefully the good ones:

  • Is it possible at all to use rd.qubes.hide_all_usb in this setup ? Would be great since there is quite no point to use sys-usb otherwise.
  • About the need to turn-off sys-usb autostart before upgrading, has someone found a better/automated solution ? One again, being quite busy/lazy I have not tried to write any such script.

Wondering if anyone has had the same problems.
Also, my efi/boot/headers are on a USB drive, in case it makes any difference, but I can’t see which one…

Grateful for any support, and thanks a lot for these great tutorials !

It would be great for @51lieal to support this thread.

well its already solved, you can try create new thread and explain your problem.

I apologise. I posted on the wrong thread.

Here is the correct post.

A post was split to a new topic: After entering luks password im not able to boot