Installing QubesOS - Gave up after 3 attempts

User Support
1

I attempted to install QubesOS today. This is more of a postmortem of what didn’t work than an overt request for help, as I wound up spending most of the day dealing with just trying to get it installed and it ultimately crashed and got stuck in a reboot loop. However, if this is a known issue, please do let me know. Maybe I’ll give it another shot.

The context is that I was finally setting up a second NVMe drive to act as a btrfs RAID1 mirror. I’ve been using Arch, but I’ve wanted to give QubesOS a try so I can easily run untrusted things or sandbox things with the highest assurance, possibly switch my primary OS by running Arch in a VM as I migrate away from it.

My original hard drive configuration is an EFI system partition and LUKS over a btrfs filesystem with root and home subvolumes. For the mirror drive, I initially figured I would do an EFI system partition, a QubesOS root partition, and an LVM partition. I could not find a good description of how to manually partition the drive in the official install guide, so I came up with this based on a forum post where someone did a vague explanation of how their QubesOS drive partition table looked.

Initial attempt:

  • biosboot (after the install nagged me that my system didn’t support EFI, I think I may have booted in the wrong mode)
  • fat32 partition for boot (but no mountpoint)
  • btrfs root partition (on /)
  • LVM (empty)

GRUB couldn’t find the (I assume root) partition via UUID. I then attempted a second install.

Next:

  • biosboot (didn’t bother trying without)
  • boot partition (/boot/efi)
  • LVM
  • btrfs root partition in LVM (on /)

GRUB booted just fine, but it had no menu options, and trying to cryptomount what I thought was the LVM partition failed

Third try I just did automatic. This, finally, booted.

Initial setup:

  • Not sure what default template to use, pros/cons aren’t specified
  • Not all options are included in the install guide. I assume it’s out of date.
  • Automatically accept USB mice isn’t explained. Why is this pad?
  • Create vm-pool isn’t explained. Why is this an option?
  • Implications of make sys-net disposable aren’t explained
  • Template install takes a long time with no meaningful progress indication

I mostly left things alone. I did specify to make sys-net disposable, since I figured that was likely better.

After this, QubeOS rebooted. I attempted to boot it twice. Both times it prompted me for the disk password, then it blanked and rebooted.

At this point I gave up. I had spent hours trying to install it, and had simply run out of time and patience. If the distro is supposed to be secure, configuring a few user settings on startup shouldn’t be sufficient to cause it to crash or panic. That screams a memory issue to me, which is a red flag for something intended to be secure. The documentation seems to be out of date. The install was slow (though part of that may have been due to the flash drive I used) and the initial set up was even slower without any clear reason.

I can tell a lot of work has been put in to the OS, but I just don’t have time to figure things out via trial and error because of cut corners in the documentation.

1 Like
2

I too gave up after three attempts, one of which did give a mostly-working system:

Qubes installation on Lenovo Thinkpad X1 2-in-1 Gen 9:

First install:

  • Chose “Debian” as default
  • Boots up to TTY login, no display

Second install:

  • Chose only defaults
  • Boots up to full system (yay), but after being unsupervised for an hour, blanks the screen and cannot recover. Also, I though I’d rather have btrfs than LVM …

Third install:

  • Chose “btrfs” instead of “LVM”
  • Boots up to TTY login, no display

A bit of internet searching did not find a way to start or diagnose the graphical display manually, or how to proceed to diagnose the problem. I logged in, and it seems like a perfectly working system, just not having a graphical interface. I don’t have it any more though: gotta do stuff.

I love the concept. I’ll probably be back in a while for another try. It would be useful to have some terminal help for the hopelessly lost: being root doesn’t seem to give the answers I hoped for - maybe the secrets are hidden in a different VM that I didn’t know how to look at.

3

which Qubes OS version did you use, on which hardware?

Of course, all I can say for now is that this is not a normal behavior. This may be a hardware issue with Xen which drives Qubes OS.

1 Like
4

For future readers considering it:

For its security to be functional QubesOS relies on hardware a lot. It is not a normal linux distribution that just works: check device compatibility first.

It is also likely to be a device issue. Or, more specifically, memory issue due to device issue. Check troubleshooting after installation and ask for help. Do not just search the internet.

Sad but true. Good stuff is on the forum (and chats).

You are installing type-1 hypervisor with fedora 37, fedora 40, debian 12, and whonix by default, some of it over tor. It cannot not be slow.

Qubes is a huge storage, memory and effort hog. It doesn’t have nested virtualization, gpu domain, and seamless gpu passthrough by default among many other things. This is a challenge, not a distro hop. Do not use it without fitting intent, being crazy, or having someone to help you. User discretion advised.

As long as we don’t have gpu domain, display shouldn’t be that hard to troubleshoot since it is managed by dom0: conventional display troubleshooting steps can work.

Once sys-gpu released, display secrets are hidden in sys-gpu. Checking forum is a good first step. Most help on the forum comes as commands. If you mean literal --help, it is not enough to troubleshoot a display without background knowledge. If you want to have docs baked into qubes, please bump this issue, I strongly agree that it must be at least an installable package in dom0, so users can read it even if something doesn’t work.

Justified.

2 Likes
5

I identify with your frustration. Folks will try to help you. Might be the first suggestion only leads to more error messages. But progress.

No doubt solene is much more technically qualified than myself. Also, no doubt, I have been blessed with seeing more error messages while trying to install.

Most who come to the forum with problems installing are trying to accomplish something a bit different than the basic documentation was written to accomplish.

I vaguely recall a rolling error caused by trying to install to a dual boot system, and the boot sector did not set up correctly. That time I also had used dd to create Qubes install USB. I have found more direct success using my recipe.

So, uh, please describe the drives in your computer. What is on the other drive. and whether you want to keep that as it is, For now.

Uh, sometimes it is helpful to behave like a pilot doing a check list. Pilot knows the check list by heart. but goes through it anyway.

Guessing solene will offer a faster solution.

I have several laptops which have two separate drives installed.
One laptop refuses to boot-- if I try to have an operating system on both drives. One drive must be Formatted and left alone. Empty. Not used. I am pretty sure that if I had the knowledge of some here, there is a way to get around that, It is reasonable to be able to use both drives.

I have another laptop that does not care if there are different OS’s on two drives. Likely makes the laptop less secure.

If the documentation included all the possibilities, no one would read it because documentation would be too long. Yeah. documentation could be improved.

I think part of the issue is relates is that Linux may try to respect the boot sector and not modify it. Which way I created the Install Qubes - versus the state of the drive I am trying to install to makes a difference.

If I was to suggest the most direct way to install Qubes. It would be;
Qubes installs easiest to a one drive system where secure boot is turned off, BIOS/EFI is set to Legacy (Yeah, verify Virtualization is turned on. Turn off fast boot)

The drive Qubes is to install to should be cleared and completely re-formatted.
I want to blow away any existing boot sector. Format drive to FAT32. I would prefer to format it in a long form, in that I worry about a quick format only changes the top of the disc partition, and I am afraid later that the Linux file system will trip over something that gets miss-identified. But in practice, that has never happened to me. I am also guessing that if I leave the drive formatted to FAT32 , without a boot sector. Then the Qubes installer wil figure it out. and in practice the installer has performed well. I had to learn how to choose options in the installer.

choose the drive to install to. Tell it recover space.

I would try not to alter the basic install. Guessing when you said you chose Debian as your basic system. You might have said. I did not choose to let Fedora Templates install? You did not intend to say you had modified dom0 (which connects directly between hardware and qubes). Am I being overly specific.

First install, try not to be too clever. Likely what you described did not create the problem.

My Recipe includes writing the Qubes Install USB using Mint Linux. Using the Mint creation too. Or the installer provided by Fedora. using dd to create the qube caused my first boot to fall out into Grub -fixer. but then, I had formatted the drive, and the drive did not have a boo sector. Using a qubes install USB created with Mint apparently was smart enough to fix a boot sector, Edit: I also did a long re-format of USB install key with FAT32. (Did it matter? I dunno.)

I think apparatus has a post somewhere describing the exact commands to modify boot sector for someone else’s computer to get rid of a similar error message to the one you got.

I have never had memory errors, so I do not know what kind of error messages might get created.

I have had issues with brand new drives, in that the computer did not see them, until - Gparted could not see the new drive. So I started Windows 10 on the computer. Anyone who makes hardware will always make sure the Windows install ISO can get it established in the computer. Windows ISO gets the BIOS/EFI to see drive, maybe installs some basic firmware somewhere. If someone knows of a Linux program to accomplish that. I would like to know.

Please let us know. What you say could change how documentation has been written, or perhaps some part of the installer itself.

Proceed as you think best.

6

please point out the outdated content so it can be fixed

2 Likes
7

Maybe I had just luck but I installed qubes with default options on two different “older” laptops and it just worked great out of the box… Also as a Windows Poweruser with a bit Linux Experience (mostly through Proxmox) I made a good progress in the few months I’m using qubes now and the Forum is just great (especially a hand full members with huge knowledge)… So I would say it is not impossible and definitely worth the effort and also worth buying a separate device for it…

3 Likes
8

Here’s an example:

“All” qubes-specific states
vs
Actual list of qubes-specific states

The documentation has a link to the github list, but doesn’t contain the same information. Most likely was linked long ago, set of states grew but documentation never reflected that.

Here’s another one:

Compare command-line tools documentation with command line tools you have

Most of the guides link to dead github pages that link to forums, instead of just linking to forums

There are a lot of old screenshots everywhere - albeit still functional, this is not how qubes look. But I dig it tho old menus do look better. Check out this gorgeous updater: https://www.qubes-os.org/attachment/doc/r4.0-software-update.png. So concise! Why craft this new idiot that doesn’t even fit the default xfce theme if they could’ve just added date, state information and settings to the old one? I bet you it doesn’t fricking teleport to the center of the screen when you attempt to resize or tile it either ahaha /rant

I think there are no information about sys-audio, at all. I can understand GPU passthrough, it is somewhat advanced topic with a lot of fiddly bits that can go wrong, but sound? Come on, everyone needs this basic functionality. It is just briefly mentioned in Audio virtualization, so mysterious. Luckily that is not a big issue, I feel like most users quickly realize that forums is the place to be.

1 Like
9

Qubes-R4.2.3-x86_64 on an AMD Ryzen 9 7950X with an X670E Creator motherboard and an AMD Radeon RX 5700 XT 50th Anniversary GPU.

10

Hmm, I guess Catacombs’ comment doesn’t let me reply directly to it.

The install guide (Installation guide | Qubes OS) should specify somewhere what partitions are needed. It would also be good to have a checklist adjacent to the partitioning tool that runs the check in realtime as you go and marks each requirement as completed, but I realize that’s probably not possible with blivet-gui without changing it.

Otherwise, the documentation should list all the install options, if it’s going to describe them. But better yet, there should be a separate pane adjacent to the install options that describes what they’re going to do as you hover over them. That has the added benefit of putting the documentation with the code going into the release, in addition to simply being more convenient when you’re in the middle of an OS installer and can’t simply open a browser to read the documentation without a separate device.

What you’re saying about separate hard drives just sounds like code smell. If I install on one drive and all my QubeOS partitions are there, it should not touch nor care about other drives beyond some incidental informational display. Until I tell it to do something with the other drives.

If there are hard hardware requirements, then the installer should query and check and warn, especially if they will cause the entire OS to crash on startup without warning.

I imagine the reason most people will be attracted to QubeOS is the security / privacy aspect, so if it can’t live up to it, I think it’s appropriate to do whitelisting of capabilities, preferably, and flag the user if it can’t independently verify that the hardware is compatible and require them to click past a warning that specifies what they need to check.

But if it really is so violent of a requirement that the OS will crash without warning, I wouldn’t complain about a whitelisting of specific hardware as well, with a similar warning about what the user needs to check; though that could be mistaken for putting certain vendors ahead of others.

But for a security product designed to isolate and protect things, I do very much expect it to have restrictive guardrails and give an error message rather than just crashing or silently failing, at least on the path of “normal” use. If I go messing with startup config files with a non-UI tool, then I’m a lot more understanding of the core guarantees of the OS being accidentally violated…but that’s very different from the FTUE flow that everybody needs to go through.

1 Like
11

Well, I assumed it was mostly just virtualization extensions. If I search around I do find this with an X670E chipset,

It should tell you that it needs to download the entire distros for the templates that you select over tor, as well as the download size. This is a privacy / security distro focused on giving you more control over your computer…the OS should be very transparent about what it needs to do online.

The last time I played around with QubeOS was in 2016 when I was maintaining a custom vanilla / Ubuntu -based distro for a robot. My Arch install has lasted about 12 years, and I’ve taken it through a number of breaking changes, including the migration to systemd and upgrading in-place from MBR to GPT, and from ext4 to btrfs to LUKS over btrfs (again, in-place).

My frustration here isn’t that it requires technical knowledge to set up. It’s that the way the system actually works is obscured by a UI, but the UI is an incomplete abstraction over the way the system really works, and it fails without warning. Leaving me stuck wondering whether it was a bad setting (which I’m predisposed to assume because I saw the UI fail to set up GRUB properly due to my custom settings that passed validation). Or if maybe it was a hardware error, as is pointed out, and I could throw away my entire week trying various combinations of settings and learn nothing and get nowhere.

But the additional concern I have is that if something is so unstable that it causes a kernel panic or a crash at such a low level it immediately triggers a reboot, that immediately makes me question whether it’s stable enough to provide any guarantee of separation of privileges or if things are built on quicksand.

My usage is admittedly more in the category of “I want to be extra sure that this AI tool / makefile isn’t up to no good” or “I want hard separation between work and personal stuff” rather than playing with viruses for pentesting, so I imagine it’s probably no worse than running things in Docker or podman, but if it was the latter, I’d be very uncomfortable.

2 Likes
12

I’d want to know what went wrong before buying hardware blindly, but I am more in the category of user for whom QubesOS would make me rest a bit easier, but isn’t absolutely needed.

13

I think we saw a lot of issues with Ryzen 7xxx :frowning: certainly related to Xen

1 Like
14

Sorry… I meant no offense. I just started by clicking on the last post that was shown when I started typing. That was Solene post.

I understand your sense of frustration. Much of what I commented on is the result of my many hours trying to create work-arounds to get Qubes working.

I can recall being frustrated by when I discovered Virtualization not working on one of my laptops, despite having the correct box clicked in BIOS/EFI for Virtualization being turned on. It was not, and it took several turning the Virtualization off. Reboot. On reboot. Then reboot twice more before the Qubes would take off and install. I also discovered I could not (easily) find a program I could run in a live Distro that could test to see if Virtualization was working. Once again. My lack of technical experience and knowledge.

I also thought of having a program, that could say, be run from a live version of Linux that could test to see if the minimal needs of the Qubes is going to be met.

Guessing Qubes following does not have such a huge following of really good developers to develop that testing. Plus the minimal qualifications is likely to keep changing with new versions of Qubes, and trying to accommodate all kinds of different hardware, new knowledge of what should or should not work.

I sense that, since you are aware you want full control over what your computer is doing. That indicates you have a much higher level of technical experience and knowledge than myself. I hope you not only get your particular computer working and you stick around. Likely you could contribute a lot to help out here.

I would say, like you would probably tell me. Greater computer security comes with more complicated computer problems trying to implement.

You seem to be very aware that how drives being sorted out is important. Basic installers should only do what they are directed to accomplish. As you know. Boot Sectors come in a lot of different versions, UEFI, GRUB, GRUB2. I feel it is correct if the Qubes installer does not overwrite an existing Boot Sector. It might render some other drive or drive partition unusable, and hard to recover. Or the existing boot sector has some kind of mistake, malware written into it.

I guess the Qubes installer may try to include some thing into existing boot that allows it to see/start Qubes. I dunno.

Which is why I just gave up trying to understand that portion. Formatted the laptop hard drive. And let the Qubes installer just do what it wanted. Which is likely not the best approach. It is an install recipe that has worked for me.

Likely Solenes comment is more correct as to your particular issue of not getting going. One of the philosophies of how to reply in a forum to help troubleshoot, is to only work on the most likely problem, then work Error message by issue by issue. with someone knowledgeable. Not a long diatribe on what did and did not work for some other individual. Hmm.

If you continue to post here. Likely I will not chime in. and let you go -issue by issue with someone who will be waiting for you to post your results.

Enjoy your day.

15

That is correct - you don’t need these extensions, they just make hypervisor more powerful, efficient, add functionality, and improve security if your use case relies on operating certain hardware that must be passed through, which is most certainly the case if you are going to connect to the internet. It is described in the documentation.

It does, but you are probably correct about transparency. User chooses what templates to install (and what features to set up) during initial setup. If we assume that user is aware of what a template is, it should be easy for them to guess what is going to happen. It is not the same with tor.

That is really cool, and I am kind of envious. On the other hand, qubes and arch are quite a polar opposite - one is very sophisticated type-1 hypervisor with a lot of automation that relies on hardware virtualization technology for sizeable part of its functionality and another is a minimalist do-it-yourself distribution.

I’m not completely sure why it relies on UI so much, but I can bet that it is due to the fact that you need to interact with virtual machines somehow without compromising dom0. This is the reason why you cannot connect a console directly to a vm: QubesOS starts management domain to give you a console. As far as I can see this is not possible with a text-based interface: even if you connect dom0 to a management domain and management domain to the target domain, you are still passing tty to dom0.

I may be wrong here, but this what hardware troubleshooting is. You get good hardware support once all weird failures are gone. Qubes is not in a good state: just check 2k issues on github, some of them are stuck there for years. Don’t run it unless you have the intent to bear the consequences and contribute to solve them, just mad, or have someone to help you.

16

Don’t know if there is a specific issue with your motherboard, but the X670e chipset and 7950X CPU does work with Qubes OS.

I think the problem you experienced could be related to sys-usb.
Hardware reset during installation and boot of R4.2 on Ryzen 9 7950X

It seems fairly common for AMD motherboards to have a multiple USB controller, and some are used for system components like BT or Wi-Fi. If you automatically create sys-usb during installation all controllers will be passed to sys-usb, on AMD motherboards this can result in a system crash at boot when sys-usb comes online.

2 Likes
17

Thanks, I’ve subscribed to the topic.