QubesOS on the VM -> Unsuported Hardware

User Support Hardware Issues
1

Hi there,

I am running MX Linux as my main OS,
and in the VM I want to run QubesOS, for which I am allocating 40GB of the SDD.

Unfortunately, I am encountering a problem at the beginning of the installation process.
Picture below

qubes_unsuported_hardware
qubes_unsuported_hardware1600Ɨ1077 110 KB

Quick System Info of the Hardware and MX OS

System: Kernel: 5.10.0-21-amd64 [5.10.162-1] x86_64 bits: 64 compiler: gcc v: 10.2.1 parameters: BOOT_IMAGE=/vmlinuz-5.10.0-21-amd64 root=UUID=<filter> ro quiet splash Desktop: Xfce 4.18.1 tk: Gtk 3.24.24 info: xfce4-panel wm: xfwm 4.18.0 vt: 7 dm: LightDM 1.26.0 Distro: MX-21.3_x64 Wildflower January 15 2023 base: Debian GNU/Linux 11 (bullseye) Machine: Type: Laptop System: Dell product: Latitude 7480 v: N/A serial: <filter> Chassis: type: 10 serial: <filter> Mobo: Dell model: N/A serial: <filter> UEFI-[Legacy]: Dell v: 1.19.0 date: 07/26/2020 Battery: ID-1: BAT0 charge: 16.4 Wh (52.7%) condition: 31.1/60.0 Wh (51.9%) volts: 7.2 min: 7.6 model: SMP DELL DM3WC64 type: Li-poly serial: <filter> status: Discharging CPU: Info: Dual Core model: Intel Core i7-7600U bits: 64 type: MT MCP arch: Amber/Kaby Lake note: check family: 6 model-id: 8E (142) stepping: 9 microcode: F0 cache: L2: 4 MiB flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 23199 Speed: 1420 MHz min/max: 400/3900 MHz Core speeds (MHz): 1: 1420 2: 1421 3: 1405 4: 1424 Vulnerabilities: Type: itlb_multihit status: KVM: VMX disabled Type: l1tf mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable Type: mds mitigation: Clear CPU buffers; SMT vulnerable Type: meltdown mitigation: PTI Type: mmio_stale_data mitigation: Clear CPU buffers; SMT vulnerable Type: retbleed mitigation: IBRS Type: spec_store_bypass mitigation: Speculative Store Bypass disabled via prctl and seccomp Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer sanitization Type: spectre_v2 mitigation: IBRS, IBPB: conditional, RSB filling, PBRSB-eIBRS: Not affected Type: srbds mitigation: Microcode Type: tsx_async_abort mitigation: Clear CPU buffers; SMT vulnerable Graphics: Device-1: Intel HD Graphics 620 vendor: Dell driver: i915 v: kernel bus-ID: 00:02.0 chip-ID: 8086:5916 class-ID: 0300 Device-2: Microdia Integrated_Webcam_HD type: USB driver: uvcvideo bus-ID: 1-5:2 chip-ID: 0c45:6715 class-ID: 0e02 Display: x11 server: X.Org 1.20.11 compositor: xfwm4 v: 4.18.0 driver: loaded: modesetting unloaded: fbdev,vesa display-ID: :0.0 screens: 1 Screen-1: 0 s-res: 2560x1440 s-dpi: 96 s-size: 677x381mm (26.7x15.0") s-diag: 777mm (30.6") Monitor-1: eDP-1 res: 2560x1440 hz: 60 dpi: 210 size: 309x174mm (12.2x6.9") diag: 355mm (14") OpenGL: renderer: Mesa Intel HD Graphics 620 (KBL GT2) v: 4.6 Mesa 20.3.5 direct render: Yes Audio: Device-1: Intel Sunrise Point-LP HD Audio vendor: Dell driver: snd_hda_intel v: kernel alternate: snd_soc_skl bus-ID: 00:1f.3 chip-ID: 8086:9d71 class-ID: 0403 Sound Server-1: ALSA v: k5.10.0-21-amd64 running: yes Sound Server-2: PulseAudio v: 14.2 running: yes Network: Device-1: Intel Ethernet I219-LM vendor: Dell driver: e1000e v: kernel port: f040 bus-ID: 00:1f.6 chip-ID: 8086:15d7 class-ID: 0200 IF: eth0 state: down mac: <filter> Device-2: Intel Wireless 8265 / 8275 driver: iwlwifi v: kernel modules: wl port: f040 bus-ID: 02:00.0 chip-ID: 8086:24fd class-ID: 0280 IF: wlan0 state: up mac: <filter> Bluetooth: Device-1: Intel Bluetooth wireless interface type: USB driver: btusb v: 0.8 bus-ID: 1-7:3 chip-ID: 8087:0a2b class-ID: e001 Report: hciconfig ID: hci0 rfk-id: 3 state: up address: <filter> bt-v: 2.1 lmp-v: 4.2 sub-v: 100 hci-v: 4.2 rev: 100 Info: acl-mtu: 1021:4 sco-mtu: 96:6 link-policy: rswitch hold sniff link-mode: slave accept service-classes: rendering, capturing, object transfer, audio Drives: Local Storage: total: 476.94 GiB used: 7.63 GiB (1.6%) SMART Message: Unable to run smartctl. Root privileges required. ID-1: /dev/nvme0n1 maj-min: 259:0 vendor: A-Data model: SX6000PNP size: 476.94 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s lanes: 4 type: SSD serial: <filter> rev: V9002s16 temp: 29.9 C scheme: MBR Partition: ID-1: / raw-size: 23.38 GiB size: 22.85 GiB (97.71%) used: 7.34 GiB (32.1%) fs: ext4 dev: /dev/dm-0 maj-min: 253:0 mapped: root.fsm ID-2: /boot raw-size: 1024 MiB size: 973.4 MiB (95.06%) used: 187.1 MiB (19.2%) fs: ext4 dev: /dev/nvme0n1p1 maj-min: 259:1 ID-3: /home raw-size: 444.52 GiB size: 436.47 GiB (98.19%) used: 112.2 MiB (0.0%) fs: ext4 dev: /dev/dm-2 maj-min: 253:2 mapped: 1.home.fsm Swap: Kernel: swappiness: 15 (default 60) cache-pressure: 100 (default) ID-1: swap-1 type: partition size: 7.98 GiB used: 0 KiB (0.0%) priority: -2 dev: /dev/dm-1 maj-min: 253:1 mapped: swap Sensors: System Temperatures: cpu: 35.5 C mobo: N/A Fan Speeds (RPM): cpu: 0 Repos: Packages: note: see --pkg apt: 2041 lib: 1032 flatpak: 0 No active apt repos in: /etc/apt/sources.list Active apt repos in: /etc/apt/sources.list.d/brave-browser-release.list 1: deb [arch=amd64] https://brave-browser-apt-release.s3.brave.com/ bullseye main Active apt repos in: /etc/apt/sources.list.d/debian-stable-updates.list 1: deb http://deb.debian.org/debian bullseye-updates main contrib non-free Active apt repos in: /etc/apt/sources.list.d/debian.list 1: deb http://deb.debian.org/debian bullseye main contrib non-free 2: deb http://security.debian.org/debian-security bullseye-security main contrib non-free Active apt repos in: /etc/apt/sources.list.d/mx.list 1: deb http://nl.mxrepo.com/mx/repo/ bullseye main non-free Active apt repos in: /etc/apt/sources.list.d/nordvpn.list 1: deb https://repo.nordvpn.com/deb/nordvpn/debian stable main Active apt repos in: /etc/apt/sources.list.d/signal-xenial-added-by-mxpi.list 1: deb [arch=amd64] https://updates.signal.org/desktop/apt xenial main Active apt repos in: /etc/apt/sources.list.d/vivaldi.list 1: deb http://repo.vivaldi.com/stable/deb/ stable main Info: Processes: 251 Uptime: 21m wakeups: 2 Memory: 31 GiB used: 2.41 GiB (7.8%) Init: SysVinit v: 2.96 runlevel: 5 default: 5 tool: systemctl Compilers: gcc: N/A alt: 10 Client: shell wrapper v: 5.1.4-release inxi: 3.3.06 Boot Mode: BIOS (legacy, CSM, MBR)

2

Did you read the documentation? Installation guide | Qubes OS

Thereā€™s a big note in yellow:

Note: Qubes OS is not meant to be installed inside a virtual machine as a guest hypervisor. In other words, nested virtualization is not supported. In order for a strict compartmentalization to be enforced, Qubes OS needs to be able to manage the hardware directly.

1 Like
3

VMware should work, itā€™s the only hypervisor Iā€™ve been able to use to run Qubes OS, but you need to enable nested virtualization.

Keep in mind, itā€™s mostly useful for testing/trying Qubes, itā€™s not a secure way to use the OS.

4

I have other Linux operation system which I want to use + the GRUB from that installation
I want to keep MX GRUB, and be able to load QUBES which Is installed on dedicated partition

before installation
|nvmeon1p1||ext4|1024|MiB|boot|
|nvmeon1p2||luks|24.4|GiB|MX|
|nvmeon1p3||swap|10|GiB|SwapMX|
|nvmeon1p4
||nvmeon1p5|luks|128|GiB|Qubes|
||nvmeon1p6|luks|313.54||HomeMX|

After installation
|nvmeon1p1||ext4|1024|MiB|boot|
|nvmeon1p2||luks|24.4|GiB|MX|
|nvmeon1p3||swap|10|GiB|SwapMX|
|nvmeon1p4|
||nvmeon1p5|luks|313.54||HomeMX|
||nvmeon1p6|luks|1|GiB|boot|
||nvmeon1p7|luks|127|GiB|Qubes|

I used ā€˜automaticā€™ installation, and choose which partition I want to preserve, and on which one system supposed to be installed.

Before automatic installation happened, I used custom installation option, but without success. Message that I saw was following.

You have not defined a root partition (/), which is required for installation of Qubes OS to continue.
You have not created a bootable partition

|How to install Qubes, so MX is installed, Grub from MX is installed, and I have option to choose between MX and Qubes?

Many Thanks

5

Iā€™ve been able to get Qubes up and running in a VM using the 4.2 weekly build ISO (havenā€™t tried 4.1 yet). This is with a nested vIOMMU and full HVM support.

I used kvm/qemu configured via VMM (Virtual Machine Manager) running on bare metal fedora-37, with some config changes for compatibility and to support vIOMMU usage.

I am using an AMD Zen 2 Family CPU and obviously the bare metal has virtualization and iommu enabled in the bios. I installed fedora-37 using the standard iso+Rufus+usb key, and fully updated it. I donā€™t recall if VMM was already installed or notā€¦if not pretty easy to find and download from the fedora repos using the Software application.

I created the VM using the Generic Linux 2022 option, set memory/storage/etc. and then chose the easy-to-miss ā€œCustomize Configuration before installā€ checkbox on the page that had the finish button on it. Note that sometimes when I do that the iso I chose in the wizard ends up not set and I have to set it again under the sata cdrom 1 node.

VMM needed some additional manual configs (via UI clicky or UI XML tab, depending):
_Verify Generic Linux 2022 baseline. This should default to a virtual Q35 chipset which is the only way intel/amd vIOMMU can be supported, at least from what I have read.
_Change Firmware from BIOS to UEFI, as this may be necessary for vIOMMU support.
_For now I set CPU to one socket and eight cores (1 thread each)ā€¦I can later experiment up to 32 with the current hardware. Depending on your processor layout you may want to experiment with using sockets instead of cores. From what Iā€™ve read vIOMMU can be sensitive to NUMA or quasi-NUMA issues (though perhaps only if actually passing in real hardware(?), but Iā€™m not there yet). One way may have better performance but less stability than the other, depending on hardware.
_Changed NIC to e1000 from default of virtio (the default which did not workā€¦it locked up sys-net).
_Initial setup had both usb tablet as input0 and ps2 mouse as input1. When I finally got viommu working the pointer stopped working (presumably due to sys-usb starting up). To fix that I removed the tablet device and set the ps2 mouse as input0.
_Changed video from Virtio (the default which did not work) to VGA. Upped vram to 65536 from 16384 in xml for larger screen options.
_Added RNG /dev/urandom
_In VMM Overview, where you can edit the entire XML file, you also need to add a manual clause in the devices section (closer to the bottom of the file).

<iommu model=ā€œintelā€/>

Lastly in the VM bios on first boot, disable secure boot so that the iso can actually be booted!

Qubes configā€¦.

Qubesā€™s grub needed an addl line (what i have below works, though includes some ignored bits) and then it has to be rebuilt:

sudo bash
vi /etc/default/grub
#add next line at end
GRUB_CMD_LINE=ā€œ$GRUB_CMD_LINE iommu=on iommu=pt amd_iommu=on intel_iommu=onā€
#exit vi
:wq
#rebuild grub
grub2-mkconfig > /etc/grub2.cfg

I also created the file /etc/modprobe.d/vfio.conf but am not sure if it was necessary. Contents are:
options vfio_iommu_type1 allow_unsafe_interrupts=1

B

PS The above solution integrated some random things I came across with some of the advice here:
https://www.berrange.com/posts/2017/02/16/setting-up-a-nested-kvm-guest-for-developing-testing-pci-device-assignment-with-numa/

PPS-if you havenā€™t used VMM before itā€™s really easy to miss the View/Details menu option, which is how you can continue to modify the configuration between VM restarts. View/Console brings you back to the UI of the VM.

1 Like
6

I did try to get HVM working inside a QubesOS VM using VMWare Workstation 17 under Windows 10 on AMD Zen 2 last week (and yes all hyper-v and related MS virtualization software components disabled) but got nowhere and gave up after about 10-15 hours. Got it working under Linux/kvm (see my previous post) tonight.

B

7

Someone on the forum said VMware canā€™t run Qubes on AMD CPUs, itā€™s some issues with the nested IOMMU that only works with Intel.

Good to know it works with KVM, I had tried running Qubes in Gnome Boxes, but I couldnā€™t get the nested virtualization to work.

1 Like
8

Yeah Boxes and VMM are just, basically, 90% configurators for the libvirt xml, esp. now that remote server connectivity stuff has moved from Boxes to Connections.

I suspect I could just port over the xml I now have in VMM to boxes and itā€™ll just work, perhaps with some minor boxes metadata added.

B

9

Made an edit in the longer reply above to be sure that the iommu xml tag isnā€™t hidden by the message parser.

B