Don’t look now…but suddenly the USB Passport drive won’t boot either.
That’s about a day’s worth of work down the drain.
Qubes rescue says there are no linux partitions on the drive.
It will no no longer boot off the fedora thumb drive, either, so I can’t implement your suggestion at present.
It’s probably the same problem with disappearing EFI menu entry that needs to be fixed with this:
No, because in this case, when I try to boot off the thing, it gives me the standard “insert bootable media and press any key” message.
The SSDs that I copied installations onto just cause the system to hang with a flashing cursor.
I notice your example still uselessly includes your name for the name of the bootable device rather than something like the name of the OS.
I wonder what else in here is obfuscative. If I can ever get the Fedora installer to boot maybe I’ll find out.
I’ve already explained this here:
Yes, but that’s exactly the problem. You KNOW this is confusing yet you persist in using it.
Right now, I’ve managed to get into Fedora Live and (to my complete lack of surprise) there’s no such directory as /boot/efi/EFI/qubes. Clearly there’s something in there you didn’t explain.
I’m going to guess its because you have to perform those steps AFTER you manage to boot into the partition using the first command.
Which did work, entirely too well. I couldn’t boot into anything else no matter what. No F12 menu to choose from; I had to shut the system down.
And now I find myself in the situation where if I unplug everything else but ubuntu it still won’t boot into ubuntu without my going into my BIOS and disabling everything other than the one item named “ubuntu.”
So now my machine is bricked.
From what OS did you do it? From ubuntu?
Boot from Fedora Live and paste here the output of this command:
sudo fdisk -l
Here is what you had to do:
Boot from Fedora Live.
Find out your USB Passport drive device and EFI partition number on it with this command:
sudo fdisk -l
For example, it’s /dev/sda and partition number 1.
Device Star End Sectors Size Type
/dev/sda1 xxx xxx xxx 100M EFI System
/dev/sda2 xxx xxx xxx 1G Linux filesystem
/dev/sda3 xxx xxx xxx 100G Linux filesystem
Then run this command for /dev/sda and partition 1:
sudo efibootmgr -v -c -u -L Qubes -l /EFI/qubes/grubx64.efi -d /dev/sda -p 1
Try to boot in Qubes installed on USB Passport drive.
If it’s successful then run these commands in Qubes dom0 terminal:
sudo cp -r /boot/efi/EFI/qubes/* /boot/efi/EFI/BOOT
sudo mv /boot/efi/EFI/BOOT/grubx64.efi /boot/efi/EFI/BOOT/bootx64.efi
sudo mv /boot/efi/EFI/BOOT/grub.cfg /boot/efi/EFI/BOOT/boox64.cfg
sudo efibootmgr -v -c -u -L Qubes -l /EFI/BOOT/bootx.64efi -d /dev/sda -p 1
If you run these commands from ubuntu:
sudo cp /boot/efi/EFI/qubes/* /boot/efi/EFI/BOOT
sudo mv /boot/efi/EFI/BOOT/grubx64.efi /boot/efi/EFI/BOOT/bootx64.efi
sudo mv /boot/efi/EFI/BOOT/grub.cfg /boot/efi/EFI/BOOT/boox64.cfg
And you had /boot/efi/EFI/qubes/ directory there, then maybe you’ll be able to restore your boot with this:
Boot in ubuntu.
Run this command to backup files just in case:
sudo mkdir /boot/efi/EFI/BOOT-bak
sudo mv /boot/efi/EFI/BOOT/* /boot/efi/EFI/BOOT-bak
Run these commands to restore ubuntu grub as default EFI entry:
sudo cp -r /boot/efi/EFI/ubuntu/* /boot/efi/EFI/BOOT
sudo cp /boot/efi/EFI/BOOT/grubx64.efi /boot/efi/EFI/BOOT/bootx64.efi
sudo cp /boot/efi/EFI/BOOT/grub.cfg /boot/efi/EFI/BOOT/boox64.cfg
OK now that makes more sense.
I was able to boot into Ubuntu, finally.
I did another install of Qubes onto a second passport, but this time I first made sure that there was a partition covering 3.5 TB (out of the 4) and nothing at all in the first half terabyte.
That free space is slightly smaller than the SSD I would like to use. Unfortunately the actual Qubes Installer just takes all of the available space, there’s no way to tell it please only take 500GB. (Lesson learned from the first Passport install.)
So right now I’m dding the passport with the new install, onto the SSD that formerly held a dd of the first thumbdrive.
After that, I’ll remove the passport, and try this procedure from inside Ubuntu then inside Qubes, assuming it boots off the SDD (which it should, until I power down). I’ll then see if the second part of the procedure makes the fix permanent.
Presuming for the moment that I have a functional SSD with Ubuntu and another functional one with Qubes, I’m going to want (for now) to boot into Ubuntu preferentially and Qubes with the F12 menu.
Right now there are according to efibootmgr, five entries in the bootloader menu. At least I assume that’s what typing efibootmgr by itself gives you.
Boot0008* UEFI OS
Boot000B GENERIC STORAGE DEVICE 0828
Boot000C* ubuntu
Boot000D* UEFI OS
Boot000E* Samsung SSD 860 PRO 1TB
As near as I can guess, UEFI OS might be telling it to display a menu. Ubuntu will display a menu, just the ubuntu menu. The other two refer to the same SSD ubuntu is on (certainly the Samsung one does, I’m not so sure about GENERIC STORAGE DEVICE), but may or may not work at all. But all five of these show up just from one dang disk, which is galling since I’m still working on getting the other disk to display once!
What do the asterisks mean?
You need to create filesystem on this 3.5 TB partition so installer won’t try to use it.
For example you created two partitions /dev/sda1 500 GB and /dev/sda2 3.5 TB with fdisk.
Then you need to create filesystem for /dev/sda2 for example ext4:
sudo mkfs.ext4 /dev/sda2
Then installer won’t see /dev/sda2 as free space.
I’m not sure if it’ll work if you just dd from larger LVM partition to a small LVM partition.
If it’s only efibootmgr command then you can run it from ubuntu as well. But for a permanent fix you can’t just make changes in /boot/efi/EFI from your ubuntu system because it’s the ubuntu os EFI partition that is mounted in /boot/efi in ubuntu. You need to either move files in /boot/efi when running Qubes or you can mount the /boot/efi partiton from Qubes disk to some directory like /mnt in ubuntu and move files there.
Run this command to see more info about EFI entries:
efibootmgr -v
This should be your USB drive.
Five boot entries (0000 - 0004), along with the active/inactive flag (* means active) and the name displayed on the screen.
https://manpages.ubuntu.com/manpages/focal/man8/efibootmgr.8.html
Before I get to your last post, I spotted this in one of your lists of directions. I strongly suspect that that last file name should be bootx64.cfg not boox64.cfg.
As for the partitions, the passports come with a full-size partition (which appears to be ntfs, i.e., windows) and you have to tell the installer to wipe them.
Doing what I did (resulting in 500GB of blank space and a 3500GB ext4 partition) resulted in the installer saying “there’s 500GB of free space.” It ignored the 3500GB partition entirely, contrary to your expectation. I suppose it would have been able to delete it if I wanted.
Now it did call the 3500GB partition sdx1, and the three installed are sdx2, etc. rather than 1, 2, 3, but looking at the starting blocks I know that those three are first on the disk. (It’s not x, but what it is seems to partially depend on how many other things are plugged in at that particular boot, so consider x a variable.)
So what I am DDing is in fact smaller than the destination drive. My bigger concern is whether the boot record will still have them numbered funky or not. I’ll know in about an hour when the dd is done.
Playing with blkid, lsblk, efibootmgr -v (I tried it with -v a few minutes after my last comment) has been very educational.
I was even able to mount the EFI directory from the passport and go looking around…which means for once I anticipated a suggestion of yours.
Is there a particular reason to be renaming the grub files to boot files? Could we just tell the boot manager to use the grub files?
i tried relabeling some of the entries in the efi boot manager, but relabeling wouldn’t take for some reason (unless you do something else other than "efibootmgr -v -b 000C -L Ubuntu [just to get ubuntu capitalized].
OK to recap: Once the DD is done, I’ll try putting the SSD into the efibootmgr list, making it second in the order. I’ll increase the delay, too (I already have). If it works, great…I boot into it and move those files. If I understand correctly that should be a permanent fix. If it doesn’t work I’ll experiment with recovering the old installation though it’s too doggone big to move permanently (so maybe it’s just as well it went non-bootable).
efibootmgr -v shows me:
0008 active “UEFI OS” points to a master boot record…which in the last couple of days I realized is very old stuff; this is probably there for backward compatibility.
000B is inactive and it’s the generic storage device (USB, from what you tell me)
000C is active “ubuntu”, it points to Ubuntu’s grub stuff, giving a full id and file path.
000D is active “UEFI OS” and points to Ubuntu’s boot stuff, giving a full id and file path.
000E is active, the samsung drive by name, apparently if it works (and I am not sure it does) it follows some sort of default.
Part of the reason I have been so frustrated with this menu is two totally different options are both labeled UEFI OS and only the latter of the two apparently would actually DO something. Ideally the one named Ubuntu would be called Ubuntu Grub (meaning it presents a menu) and the second one named UEFI OS would be labeled Ubuntu. But I can’t figure out how to change the labels, -b 000C -L doesn’t change it.
You’re right, it should be bootx64.cfg, I’ve copy-pasted and didn’t recheck.
Check the link to stackexchange from this post:
I think you can only set label when you create entry and can’t change it after this. You can delete old entry and create same entry with new label.
Yes, these should be the right steps.
You can try to first create new entry the same as “ubuntu” but with changed label to “Ubuntu Grub” for example and check if it works. If it works then delete old entry.
The new menu entries did work but are reversed for some reason…the one I expected to go into Grub went straight in, and vice versa…I just rechecked; Ubuntu Menu points to the grub64 file, Ubuntu Direct points to the Boot64 file. Maybe I just understood these backwards.
The bad news is my dd from the Passport does NOT work; there are no partitions on the SDD at all. I’m not sure how to account for this, I should at least have got the partition table at the front with the DD command!
Oh, well, back to the USB drive.
How did you run the dd?
I think the right way would be to just dd the devices:
sudo dd if=/dev/sda of=/dev/sdb bs=1M status=progress
Where sda - your USB Passport drive and sdb - your SATA SSD.
That’s what I did.
Working on something a bit more involved now.
Then you can try to clone the partition table and then dd each partitions separately:
sudo sfdisk -d /dev/sda | sudo sfdisk --force /dev/sdb
sudo dd if=/dev/sda1 of=/dev/sdb1 bs=1M status=progress
sudo dd if=/dev/sda2 of=/dev/sdb2 bs=1M status=progress
sudo dd if=/dev/sda3 of=/dev/sdb3 bs=1M status=progress
Just fix commands for right partition numbers.
Actually that’s exactly what I did do overnight (I said I’d be doing something more complicated, that was it). Well, I manually created identical partitions to sdc2, 3 and 4 as sdb 1, 2, and 3 in gparted, at least as close as I could get.
[Edit: After some experimenting, doing sfdisk -d and piping would have been bad, because it would still have carried over the /sda1 partition (which is at the END of the disk even though its #1) which seems to have caused confusion. I’d have wanted to edit its output to omit this partition and renumber the others, then feed THAT into sfdisk. In fact if I have to repeat this process (and I might, using another SSD, just to be safe) it beats the daylights out of trying to manually duplicate the partitions. Fdisk -l doesn’t show them looking exactly the same though it was evidently close enough to boot.]
While waiting on that I made changes to efibootmgr to show the SSD and the USB drive.
This time the SSD apprently did boot…once. The second time it never offered me the prompt to decrypt the drive and eventually presented me with the error message:
/dev/mapper/qubes-dom0-root does not exist
/dev/qubes-dom0/root does not exist
/dev/qubes-dom0/swap does not exist
And so on.
I eventually discovered it was booting off the USB (despite my selecting the SSD in the boot menu) when it was successful, and failing when the USB wasn’t plugged in. Still investigating why. But the boot menu lies to me, it’s not booting or trying to boot from where I’m telling it to go.
I just had a really disturbing thought.
We know the qubes installer simply cannot see ANY Sata drive on my box. We thought we could get around this by installing to USB and copying it to a SATA SSD, then we had to donkey around with the efi stuff to get it to find it (and for that matter the USB drive).
What if the software responsible for booting Qubes ALSO can’t see SATA drives (including the drive its actually on)? At some point it’s handed a bunch of file paths and so forth, and so long as the USB drive is plugged in, it will find them (and think it’s on /dev/sda). But that means it essentially boots the USB drive.
If so there’s simply no way to boot Qubes off of ANY SATA drive on my motherboard. Period. I’m thinking that SATA-aware software is loading things from the SSD to a ramdisk, and handing control over to the ramdisk. But the software ON the ramdisk can’t see the SATA drive! It CAN see the USB drive, and proceeds to boot that, blissfully unaware that that’s not the drive it came from.
If the USB drive isn’t there, of course, it can’t find the files it needs to continue, and hangs.
If this is what’s going on, I can’t run off anything but the USB drive, and basically, until I buy new hardware, all I can really do is just play with Qubes…which I did quite a lot of on the first USB drive (before it “disappeared.”) Now I know how to fix the disappearing and I can do a lot more playing with Qubes [very educational], but I can’t make it my default system, yet.
I am pretty confident based on what I have done, that I can get Qubes configured for my particular use case, albeit still on a USB. I think I might even be able to migrate from the USB to a real system, when I have one, and not have to recreate everything.
It’s possible that Qubes cloned from USB drive to SATA drive won’t be able to see the SATA drive too.
But in your case the problem seems that:
Also you can try latest Qubes ISO build with kernel-latest from here, maybe it’ll fix the problem with disk detection:
https://qubes.notset.fr/iso/
They do have the same IDs in blkid. Which explains in part why the USB drive boots.
The problem is, if the USB drive isn’'t plugged in, the SSD won’t boot properly, at all. It’s not simply a matter of the two getting confused; it’s that the USB is needed under current circumstances.
This afternoon I created an SSD from the newer USB drive, this time using SFDISK to do a dump, correcting for the sector numbers, and then feeding that into the other. Thus the partituion table is effectively identical (other than the missing 3.5TB empty partition.
Same issue. I conclude (having eliminated everything else I can) that not only can the installer not see the SATA drives on my system; the boot software cannot either.
I am actually going to go buy new hardware tomorrow; I’ll attempt to persuade the store personnel to at least let me see if the installer thumb drive can see the SATA hardware.
Meanwhile I can continue experimenting with Qubes on the two USB installs (one onto the full disk, one onto only 1/4 TB of it).
(I may or may not try that new ISO you linked to, but it will be an academic exercise. The fact of the matter is, I want beefier hardware to run Qubes anyway, this just means I buy it earlier.)