MacOS VM on qubes

franc01s · December 25, 2020, 5:49pm

Has anybody succeeded in installing a MacOS VM (or other) on qubes? What was your recipe?

alzer89 · January 27, 2021, 4:07am

Tried the following:

Booting qube from different versions of Niresh ISO (Catalina, Mojave, Lion)
RESULT: UNSUCCESSFUL
SeaBIOS loaded correctly, but Qube halted before any GUI loaded.
Creating HVM from ready-made MacOS VMDK image
RESULT: UNSUCCESSFUL
SeaBIOS said no bootable device found.
Booting from Hackintosh Yosemite ISO
RESULT: UNSUCCESSFUL
GUI loaded correctly, but then kernel panic occurred.
Trying to create Xen file equivalent from Kholia’s OSX-KVM project
RESULT: UNSUCCESSFUL
Lack of sufficient knowledge about Xen. Still a work in progress.

MY MOTIVATION FOR DOING SO:
It would be nice to be able to have it alongside my Windows 10 qube that I keep around for:

when my family asks me for MacOS technical support. (GNU+Linux satisfied all my daily driver needs wonderfully)
development purposes
testing malware safely
showing people my single laptop running multiple OSes simultaneously

Legality issues aside (I was doing this on a MacBook Air 4,2, before anyone brings it up ), I’m sure there would be enough demand out there for this. Especially now that Big Sur likes to ‘phone home’ almost as much as Windows…

fsflover · January 27, 2021, 7:24am

Emily · December 16, 2021, 12:52pm

Do you get it to work?

I’m wondering if this script would translate to a Qubes standalone:

ppc · December 16, 2021, 12:54pm

maybe no, since the “hardware” is different

ckN6QwSZ · June 8, 2022, 12:02pm

Same here. I believe Kholia’s project to be the most promising approach. We need Qubes to use a UEFI firmware in the hackintosh HVM to boot, though.

edit: so, first step is to replace the hvmloader against OVMF_CODE.fd.

ckN6QwSZ · June 8, 2022, 1:01pm

After placing OVMF_CODE.fd in dom0:/usr/libexec/xen/boot/ and changing hvmloader to OVMF_CODE.fd with

virt-manager (hvm-mac - overview - xml - change and apply)

I started the HVM with

qvm-start hvm-mac --cdrom=dispXXXX:/home/user/OpenCore.iso

HVM starts, but it looks like qvm-start reverts the change to hvmloader, so I’m back to SeaBIOS.

ckN6QwSZ · June 8, 2022, 1:46pm

qvm-features hvm-mac uefi 1

(credits: Enabling UEFI boot in qube - #2 by kommuni )

qvm-start hvm-mac --hddisk=dispXXXX:/home/user/OpenCore.img

aaaand OpenCore works!

ckN6QwSZ · June 7, 2022, 9:58pm

Hi there, I know this could be a huge waste of time, but it still might be fun and I couldn’t resist. As for all Hackintosh projects this is for educational purposes only.

In a fedora-35-dvm and after increasing private storage max size to 10GB for dispXXXX:

wget https://github.com/kholia/OSX-KVM/raw/master/fetch-macOS-v2.py
chmod +x fetch-macOS-v2.py
./fetch-macOS-v2.py
[..]
Choose a product to download (1-5): 4
[..]
qemu-img convert BaseSystem.dmg -O raw BaseSystem.img

In dom0:

qvm-create -P poolhd0_qubes --property=virt_mode=hvm --property=kernel='' --label blue --standalone hvm-mac
qvm-start hvm-mac --cdrom=dispXXXX:/home/user/BaseSystem.img

So far, so easy.

Now follows the more difficult part. One has to boot via OpenCore and to attach OVMF_CODE (firmware) and OVMF_VARS (nvram) to the HVM. This works for Qemu/KVM (plus libvirt) and should be doable for Qemu/Xen as well.

Anybody interested in joining me in this ride?

Kholia’s github repo (kholia) contains an opencore.qcow2 and a script which sets up the environment for Qemu/KVM: OpenCore-Boot-CD.sh.

PS: worst part might be a few cpu flags which Xen does not pass through as far as I can tell.

ckN6QwSZ · June 8, 2022, 10:55am

First problem I encountered is that neither BaseSystem.img, BaseSystem.iso, OpenCore.img or OpenCore.iso are recognized as bootable devices (passed with --cdrom or --hddisk alike).

So, let me rephrase my above post into a few questions:

how can I pass arguments (an environment) to an Xen-HVM like it is done in OpenCore-Boot-CD.sh (1) for Qemu/KVM?
what is Qubes equivalent to qemu-system-x86_64 "${args[@]}"?

I have taken a look into /usr/lib/python3.8/site-packages/qubesadmin/ in dom0 and tried to understand how qubes starts Qemu/Xen VMs under the hood. Did grep -Ri qemu and grep -Ri xen and was vim-ing through some of the files, but did not really get a grip on how HVMs are started.

anybody out there who would like to share his or her knowledge about the starting process of HVMs?

These are the parameters which attach the devices to Qemu/KVM:

[user@dispXXXX ~]$ grep -i drive Downloads/OSX-KVM/OpenCore-Boot-CD.sh
-drive if=pflash,format=raw,readonly,file=“$REPO_PATH/$OVMF_DIR/OVMF_CODE.fd”
-drive if=pflash,format=raw,file=“$REPO_PATH/$OVMF_DIR/OVMF_VARS-1024x768.fd”
-drive id=OpenCoreBoot,if=none,snapshot=on,format=qcow2,file=“$REPO_PATH/OpenCore/OpenCore.qcow2”
-device ide-hd,bus=sata.2,drive=OpenCoreBoot
-device ide-hd,bus=sata.3,drive=InstallMedia
-drive id=InstallMedia,if=none,file=“$REPO_PATH/os.dmg”,format=raw
-drive id=MacHDD,if=none,file=“$REPO_PATH/mac_hdd_ng.img”,format=qcow2
-device ide-hd,bus=sata.4,drive=MacHDD

An important step I believe is to exchange the SeaBIOS to OVMF_CODE.fd which seems to be a tianocore firmware port (2) for Qemu.

ckN6QwSZ · June 8, 2022, 11:32am

This is a start:

https://wiki.osx86project.org/wiki/index.php/Snow_Leopard_Server_on_Xen

In qubes-os the config-files for the VMs are not located in /etc/xen/, however, I found them in /etc/libvirt/libxl/ and that are libvirt-files which can even be edited with virsh edit hvm-mac.

alzer89 · June 8, 2022, 4:32pm

As per the macOS EULA, I have my Apple hardware on standby ready to try this

ckN6QwSZ · June 8, 2022, 4:44pm

Topics have been merged and my last three posts are older then my first three posts.

Anyway, I got OpenCore to run and BaseSystem to boot like this

qvm-features hvm-mac uefi 1
qvm-block a --persistent hvm-mac dispXXXX:loop0
qvm-block a --persistent hvm-mac dispXXXX:loop1
qvm-start hvm-mac

after making the images available in dispXXXX like this

sudo losetup -f /home/user/git/OpenCore.img
sudo losetup -f /home/user/git/BaseSystem.img

Kudos and Thanks to @tzwcfq who also suggested to create a seperate template like this

which might suite the usecase better, since we have to add a lot more then opencore and the installer medium to get the HVM Hackintosh to work.

Next step is to convert as much args from OpenCore-Boot(-CD).sh into /etc/qubes/templates/libvirt/xen/by-name/hvm-mac.xml as possible.

ckN6QwSZ · June 8, 2022, 6:08pm

And that is clearly a pain in the a… - might be easier to take another machine, install qemu, kvm, libvirt, setup a macOS-VM and transfer the macOS.xml to qubes to see which parts of the kvm xml can be used.

For instance to transfer

-cpu Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check

to the existing

  <cpu mode="host-passthrough">
    <feature policy="disable" name="vmx"/>
    <feature policy="disable" name="svm"/>
    <feature policy="require" name="invtsc"/>
  </cpu>

one probably has to take a big dive into qemu’s and libvirt’s manuals.

https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html

At least

  <cpu mode="custom">
    <model name="Penryn"/>
    <feature policy="disable" name="vmx"/>
    <feature policy="disable" name="svm"/>
    <feature policy="require" name="invtsc"/>
    <feature policy="require" name="ssse3"/>
    <feature policy="require" name="sse4.2"/>
    <feature policy="require" name="popcnt"/>
    <feature policy="require" name="avx"/>
    <feature policy="require" name="aes"/>
    <feature policy="require" name="xsave"/>
    <feature policy="require" name="xsaveopt"/>
    <feature policy="require" name="check"/>
  </cpu>

doesn’t work out of the box.

edit: this cpu-config is supported though…

  <cpu mode="host-passthrough">
    <feature policy="disable" name="vmx"/>
    <feature policy="disable" name="svm"/>
    <feature policy="require" name="invtsc"/>
    <feature policy="require" name="ssse3"/>
    <feature policy="require" name="sse4.2"/>
    <feature policy="require" name="popcnt"/>
    <feature policy="require" name="avx"/>
    <feature policy="require" name="aes"/>
    <feature policy="require" name="xsave"/>
  </cpu>

tzwcfq · June 8, 2022, 6:26pm

Try:

  <cpu mode="host-model">
    <model>Penryn</model>

ckN6QwSZ · June 8, 2022, 6:32pm

Works.

And writing the config to /etc/qubes/templates/libvirt/xen/by-name/hvm-mac.xml to make changes permanent works, too.

tzwcfq · June 8, 2022, 6:45pm

Also you can add vendor id if it’ll make a difference:

  <cpu mode='host-model'>
    <model vendor_id='GenuineIntel'>Penryn</model>

ckN6QwSZ · June 8, 2022, 6:54pm

I’ll do that. And

are accepted now, too.

Do you have an idea how to get

  -device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc"
  -drive if=pflash,format=raw,readonly=on,file="some_dom0_path/OVMF_CODE.fd"
  -drive if=pflash,format=raw,file="some_dom0_path/OVMF_VARS-1024x768.fd"

inside of the xml? I tried a few times and that just got erased by virt-manager without a failure notice.

ckN6QwSZ · June 8, 2022, 6:59pm

Hold on… kholia has got a .xml file in his repo…

github.com

kholia/OSX-KVM/blob/master/macOS-libvirt-Catalina.xml

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<!--
	macOS libvirt XML configuration.

	Run "virt-xml-validate macOS-libvirt-Catalina.xml" to validate this file.

	To install this file, you may place it at ~/.config/libvirt/qemu/
	and run: virsh define macOS-libvirt.xml.

	This configuration has been tested in Ubuntu 20.04 with stock QEMU-KVM.

	Move/rename images and loader/nvmram files and paths as you wish.

	!!! Don't forget to replace CHANGEME with your values !!!

	Adjust memory and currentMemory to 3145728 if you want only 3 GiB.

	Consider removing some cpu features if your hardware lacks support.

This file has been truncated. show original

tzwcfq · June 8, 2022, 7:00pm

libvirt: Domain capabilities XML format