Cannot install QubesOS, always booting to a black screen

[machineassociate]

Motherboard:

“AMD TRX40 ATX motherboard sTRX4 for 3rd Gen Ryzen Threadripper-series processors with 16 power stages”

or

“Prime TRX40-Pro”

CPU

“AMD Ryzen™ Threadripper™ 3970X”

Graphics car:

Nvidia Quadro RTX 4000
“GPU Device Id: 0x10DE 0x1EB1”

I am (new to linux) currently running Linux Manjaro 23.1.0 on an NVM, I have Linux MX on a SATA SSD, and do not mind erasing Linux MX and install Qubes, just for testing first, and if everything works with Qubes, erase everything and start using Qubes.

I tried to follow:
User documentation / Choosing your hardware
qubes-os [dot] [org]/doc/system-requirements/

AMD-V with RVI, and AMD-Vi

AMD-V:

grep -o -w -m 1 “svm” /proc/cpuinfo

Should return:

svm

returns:

svm

AMD-V with RVI:

sudo dmesg | grep “Nested Paging enabled”

Should return:

Nested Paging enabled

returns:

[ 12.119832] kvm_amd: Nested Paging enabled

sudo dmesg | grep “Nested”

returns:

[ 12.119830] kvm_amd: Nested Virtualization enabled
[ 12.119832] kvm_amd: Nested Paging enabled

AMD-Vi:

sudo dmesg | grep “IOMMU enabled”

Should return:

IOMMU enabled

returns:

nothing

sudo dmesg | grep “IOMMU”

returns:

[ some numbers that might be important!] pci 0000:60:00.2: AMD-Vi: IOMMU performance counters supported
[ some numbers that might be important!] pci 0000:40:00.2: AMD-Vi: IOMMU performance counters supported
[ some numbers that might be important!] pci 0000:20:00.2: AMD-Vi: IOMMU performance counters supported
[ some numbers that might be important!] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ some numbers that might be important!] pci 0000:60:00.2: AMD-Vi: Found IOMMU cap 0x40
[ some numbers that might be important!] pci 0000:40:00.2: AMD-Vi: Found IOMMU cap 0x40
[ some numbers that might be important!] pci 0000:20:00.2: AMD-Vi: Found IOMMU cap 0x40
[ some numbers that might be important!] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ some numbers that might be important!] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[ some numbers that might be important!] perf/amd_iommu: Detected AMD IOMMU #1 (2 banks, 4 counters/bank).
[ some numbers that might be important!] perf/amd_iommu: Detected AMD IOMMU #2 (2 banks, 4 counters/bank).
[ some numbers that might be important!] perf/amd_iommu: Detected AMD IOMMU #3 (2 banks, 4 counters/bank).
[ some numbers that might be important!] AMD-Vi: AMD IOMMUv2 loaded and initialized

lscpu

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         43 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  64
  On-line CPU(s) list:   0-63
Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen Threadripper 3970X 32-Core Processor
    CPU family:          23
    Model:               49
    Thread(s) per core:  2
    Core(s) per socket:  32
    Socket(s):           1
    Stepping:            0
    Frequency boost:     enabled
    CPU(s) scaling MHz:  62%
    CPU max MHz:         3700,0000
    CPU min MHz:         2200,0000
    BogoMIPS:            7389,00
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm con
                         stant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f
                         16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpex
                         t perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb s
                         ha_ni xsaveopt xsavec xgetbv1 cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lo
                         ck nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl umip rdpid overflow_recov s
                         uccor smca sev sev_es
Virtualization features: 
  Virtualization:        AMD-V
Caches (sum of all):     
  L1d:                   1 MiB (32 instances)
  L1i:                   1 MiB (32 instances)
  L2:                    16 MiB (32 instances)
  L3:                    128 MiB (8 instances)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-63
Vulnerabilities:         
  Gather data sampling:  Not affected
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Not affected
  Retbleed:              Mitigation; untrained return thunk; SMT enabled with STIBP protection
  Spec rstack overflow:  Mitigation; Safe RET
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Not affected
  Tsx async abort:       Not affected

Is my hardware compatible?

I tried booting with:
Qubes release 4.2.0-rc5
Qubes release 4.2.0-rc4
Qubes release 4.1.2
When I select the “Test then install” option, I get ablack screen, and nothing happens.
I also tried the “Install” option, and nothing happened. “Black screen, waited for 5 minutes, and nothing happened”

Qubes release 4.0.4
Some text flashes on the screen, then nothing happens. “Black screen, waited for 5 minutes, and nothing happened”

I tried multiple USB cards, and booted on a USB 2.0 & 3.0, without any luck

Linux ISO to USB

find the USB:
lsblk
Unmount it:
sudo umount /dev/sdc1
Format it:
sudo mkfs.ext4 /dev/sdc

/Documents/ISO/Qubes-R4.2.0-rc5-x86_64
sudo dd if=Qubes-R4.2.0-rc5-x86_64.iso of=/dev/sdc status=progress bs=1048576 conv=fsync

I tried doing as much as I could, out of respect for everyone and myself. I really am out of ideas. Any help is much appreciated.
Thank you in advance

[apparatus]

Try to add x2apic=false in the boot command line of xen in grub:

Improve X2APIC compatibility with modern AMD platform

[machineassociate]

This leaves me with more questions than answers

(1)
add
x2apic=false

(2)
in the boot command line of xen
Edit(1) (me) no clue

(3)
in grub

(me) I am guessing by editing the file:

sudo mousepad /etc/default/grub

I should add:

x2apic=false

In which/to which line should I add it?

Should the last step be:

sudo update-grub

Edit(1) Or is it the files in the “USB stick”

Thanks for the reply

[apparatus]

Trouble installing on Lenovo Ideapad 5 Pro

Most likely this issue. You can probably solve this by adding x2apic=false to the grub item. To do this, when grub shows up, navigate to your desired installation option ( “install with kernel latest”, e.g.). Then, press e key to edit that option. Scroll down with arrow keys, and find the line beginning with “multiboot2”, and add x2apic=false to the end of that line.

Example with images, but this one is not for Xen command like options:

Autostart troubleshooting

[machineassociate]

…Scroll down with arrow keys, and find the line beginning with “multiboot2”, and add x2apic=false to the end of that line…

I do not have a line that starts with

multiboot2

Edit (1)
I’ve not installed or was ever able to install Qubes at all

[apparatus]

You need to edit the grub options for Qubes OS installer, not your another OS grub.

[machineassociate]

Thanks, that I understood better.

Still, nothing but a black screens

[apparatus]

You can try this:

Qubes 4.1 Installation Issues for Legion 7

After trying the second release of Qubes 4.2 on https://qubes.notset.fr/iso. I managed it get it installed. I used the boot arguments: console=none vga=,keep dom0_max_vcpus=1 dom0_vcpus_pin acpi=off nomodeset=0 Everything seems to be working as expected. I won’t be using Qubes for anything too important until a proper release of Qubes 4.2 comes out but I’ll post any other issues and fixes I find on here for anyone facing similar issues.

[machineassociate]

Thank you. I tried:

console=none
vga=,keep
dom0_max_vcpus=1
dom0_vcpus_pin
acpi=off
nomodeset=0

console=none
vga=keep
dom0_max_vcpus=1
dom0_vcpus_pin
acpi=off
nomodeset=0

I also tried for “Install Qubes (not the latest kernel)” with no success

This is far away from easy for my hardware !!

[machineassociate]

I booted with the verbose mode and could get these screenshots from videos I captured.
It happens so quickly, and goes to a black screen, so I could not get a log of what happened

Thie first image is the first details I get, which I do not think is that important!

These two images are of the last messages I get.

Can anyone give any tips into what’s going on?
Thanks.

[apparatus]

You need to add dom0_max_vcpus=1 and dom0_vcpus_pin to Xen command line parameters (multiboot2 line in grub).
acpi=off and nomodeset should be added to kernel command line parameters (module2 vmlinuz line).
You can also remove quiet to see the log output.

[machineassociate]

I tried everything as you posted, nothing worked
I tried that for Install Qubes, Install with latest kernel, Test media…

anyone has any tips!

Thank you in advance

[machineassociate]

Some progress:

Qubes release 4.1.2

0. Remove all SSDs, and leave just one for Qubes OS. Such that when I boot, I’ll only have the SD card & one SSD on the system.

1. Use USB 2.0

2. 16GB SD Card

3. The SSD that I used had one partition and it was “unlocated/empty”

• a.1) lsblk “or lsblk -f for more details”

• a.2) Find your SSD: min was: sda, yours could be sdb, sdc, …

• b.1) Format it with the commands: mkfs.ext4 /dev/sda

• b.2) Change “/dev/sda” with - a.2)

4.1. (still not 100% sure about the following)

4.2. When everything is ready, and after a restart, when booting, … mash F8 to get to the boot selection menu. Do not select:

UEFI: “Your SD Card”

UEFI: “Your SD Card - Partition 2”

and select:

“Your SD Card”

4.3. Press the “TAB” button when selecting “Install QubeOS” and add as @apparatus suggested

dom0_max_vcpus=1 and dom0_vcpus_pin to Xen command line parameters (multiboot2 line in grub)

and

acpi=off and nomodeset=0 should be added to kernel command line parameters (module2 vmlinuz line).

My installation now came to a close after the following message:

The following error occurred while installing the boot loader. The system will not be bootable. Would you like to ignore this and continue with installation?

Failed to set new efi boot target. This is most likely a kernel or firmware bug.

I’ll give Qubes release 4.2.0-rc5 a try, and see what happens. I am very glad that I got over a lot of hurdles, I am happy and wanted to share what I’ve been through. I am also past the black screen of the abyss.

[apparatus]

@machineassociate

The following error occurred while installing the boot loader. The system will not be bootable. Would you like to ignore this and continue with installation?

Failed to set new efi boot target. This is most likely a kernel or firmware bug.

Should be some Fedora bug and not related to Qubes OS specifically.
You can search for a fix here:

Fedora Discussion

[machineassociate]

I tried Qubes release 4.2.0-rc5 and found whay I have the black screen of the abyss

1. Boot & select USB “not the UEFI partition”
   on the xen.gz line I added:
   dom0_max_vcpus=1
dom0_vcpus_pin
vga=,keep
   on the vmlinuz line I added:
   acpi=off
nomodeset=0
   First:
   
   
   
   I get this:
   
   
   
   This is what I get:
   
   

Xen hypervisor allocated kernel memory conflicts with E820 map
(XEN) Hardware Dom0 halted: halting machine.

2. I tried:
   Boot & select USB “not the UEFI partition”
   on the xen.gz line I added:
   vga=,keep
   on the vmlinuz line I added:
   I added nothing in this line
   

   

3. I tried:
   Boot & select the UEFI partition in the USB
   on the xen.gz line I added:
   vga=,keep
   on the vmlinuz line I added:
   I added nothing in this line
   

   

Basically, the same!

I just wanted to report this. Still have not found a solution for the error in post #13

[machineassociate]

Not a single solution was of any help! And most solutions are quarter and half explained
I’ve been trying for 3 days now

1. While installing:
   

   

2. This is hwo the disk looks like after that installation:
   

   

After I unplug all external storage except for the SSD where I installed
Qubes OS, when I (F8 to get the boot menu), there is no UEFI partition! Is that normal?

3. My boot menu:
   

   

4. Reboot and select proper Boot device
   or Insert Boot Media in selected Boot device and press a key
   

   

5. When I boot from from my Manjaro’s disk, I get:
   

   

6. Which gives me:
   

   

7. Advanced options for Qubes “from the 5. step” gives me these two:
   

   

8. Both of which gave the same error, a similar one to the one above:
   

   

9. Grub for Qube “not from the Advanced options for Qube (5th image)”
   

   

10. Grub for the first of two, from “Advance Options” “from the 5. step”
    

    

11. Grub for the second of two, from “Advance Options” “from the 5. step”
    

    

My brain is cooked. I really wanted to run this OS I am heart broken.
I bought this used monster of a PC just because I wanted this Distro.
Will not give up

I also looked at:
qubes-os[dot][org]/doc/uefi-troubleshooting/#installation-finished-but-qubes-boot-option-is-missing-and-xencfg-is-empty–installation-fails-with-failed-to-set-new-efi-boot-target

step 2.

cd /mnt/sysimage/boot/efi/EFI/qubes
cp xen-*.efi xen.efi

Copy them where to?

Step 3. Create xen.cfg
An aditinal tip would be:
tocuh xen.cfg
nano notwor for me to edt, so I used:
vi xen.cfg

Btw, all of that was not necessary, because there were no problems with these files as I checked

[apparatus]

@machineassociate

I also looked at:
qubes-os[dot][org]/doc/uefi-troubleshooting/#installation-finished-but-qubes-boot-option-is-missing-and-xencfg-is-empty–installation-fails-with-failed-to-set-new-efi-boot-target

step 2.

cd /mnt/sysimage/boot/efi/EFI/qubes
cp xen-*.efi xen.efi

Copy them where to?

Step 3. Create xen.cfg
An aditinal tip would be:
tocuh xen.cfg
nano notwor for me to edt, so I used:
vi xen.cfg

It’s outdated.
To fix missing UEFI entry you need to do this:

BIOS Update That Will Break Boot

If you use uefi : get a qubes os media installation. boot to qubes os rescue mode. skip to shell. [shell] : efibootmgr -v -c -u -L "Qubes OS" -l /EFI/qubes/grubx64.efi -d /dev/nvme0n1 -p 1 reboot change -d <which drive?> and -p <which efi partition?>. if you want you can make sure first by issuing efibootmgr -v in dom0

Also the Qubes OS 4.2 is released so it’s better to install it instead of Qubes OS 4.1.2.

[machineassociate]

I did try that too, but got an “error” that I could not solve! I wrote my last post at 3 in the morning sorry, I forgot to say/add that!

A side note:
Something that I noticed after the install, there is no UEFI partition
with SATA6G_5: Crucial…!

But, let’s first talk about:
efibootmgr -v -c -u -L "Qubes OS" -l /EFI/qubes/grubx64.efi -d /dev/sda -p 1

The only way that I can access and install “unsuccessfully” is by:
Select the:
UEFI: SanDisk (15.3GB) “image above”

I then get a quick flash of
Welcome to GRUB:

I select:
Install Qubes OS R4.1.2.0 using kernel-latest …

press e to edit

on the xen.gz line I add:
dom0_max_vcpus=1
dom0_vcpus_pin

on the vmlinuz line I added:
acpi=off
nomodeset=0

When I get the “installation screen”, I press:
Ctrl + Alt + F2

then the following image:

EDIT:

I have a typo here!
it should be:
efibootmgr -v -c -u -L "Qubes OS" -l /EFI/qubes/grubx64.efi -d /dev/sda -p 1
grubx64.efi
That still did not fix the issue!

Boot0002* Hard Drive
is
/dev/sda
and is where I had installed Qubes OS

Boot0003* UEFI: SanDisk
Boot0004* UEFI: SanDisk, Partition2
is
/dev/sdb
and is what I use to install the OS:
lsblk -p
sudo dd if=Qubes-R4.1.2-x86_64.iso of=/dev/sdb status=progress bs=1048576 conv=fsync
Taken from:
https://www.qubes-os.org/doc/installation-guide/#linux-iso-to-usb

Boot0005* USB
is?

Boot0006* UEFI: Generic-USB3.0...
is
/dev/sdc
and is where I had Qubes-R4.2.0-x86_64 (I have installed 4.1.2, this is the new version!)

efibootmgr -v -c -u -L "Qubes OS" -l /EFI/qubes/grubx64.efi -d /dev/sda -p 1
Returns:
Could not prepare Boot variable: No such file or directory
Do I have to
mkdir /mnt/sda1
mount /dev/sda1 /mnt/sda1

I also tried installing 4.2.0. At the moment, I found not a single way to reach the
installation screen.
I’ll try to use another SD card and format it as fat32

[apparatus]

Try to add x2apic=false in the boot command line of xen in grub as well.

@machineassociate

efibootmgr -v -c -u -L "Qubes OS" -l /EFI/qubes/grubx64.efi -d /dev/sda -p 1
Returns:
Could not prepare Boot variable: No such file or directory

Maybe there is no such file /EFI/qubes/grubx64.efi in EFI System partition for some reason?
Try to mount the EFI System partition and check the files there:

mkdir /mnt/sda1
mount /dev/sda1 /mnt/sda1
ls /mnt/sda1/EFI/*

[machineassociate]

Here is what I did:
I see that there are the necessary files after mounting sda

I have 6 images of grub.cfg, I wont post them unless you think I should.
I looked however at the content of xen.cfg

Which seems correct for me!
https://www.qubes-os.org/doc/uefi-troubleshooting/#installation-finished-but-qubes-boot-option-is-missing-and-xencfg-is-empty--installation-fails-with-failed-to-set-new-efi-boot-target

And here are som more stuff that I tried to look at and some changes that I tried
(“Qubese OS” vs ‘Qubes OS’ in:
efibootmgr -v -c -u -L "Qubes OS" -l /EFI/qubes/grubx64.efi -d /dev/sda -p 1 ):

@apparatus

Try to add x2apic=false in the boot command line of xen in grub as well.

I’ll do that next for Qubes OS 4.2.0
sudo dd if=Qubes-R4.2.0-x86_64.iso of=/dev/sdZZ status=progress bs=1048576 conv=fsync
Thank you very much for all the help so far by the way