Setup OpenBSD Qube with usb passthrough, audio and 1920x1080 resolution

After using OpenBSD as the main OS for 2-3 years I wanted to give Qubes OS another shot, but I still wanted to have Qube with full desktop experience of OpenBSD.

So here is my configuration of how I set up my OpenBSD qube with full 1080p resolution, and with audio and usb support. This config allows me (and possibly you) to use OpenBSD qube in full screen (desktop like experience) using gop, which is not available in OpenBSD Qube using SeaBIOS.

Setup (Qubes OS 4.2)

OVMF fixup

Before we can do anything, we need to fix uefi enabled HVMs.
As of 4.2, Qubes comes shipped with ovmf binaries that are not compatible with xen. Starting with edk2-stable202108, ovmf developers removed Xen support and moved it to the OvmfXen.

Qubes does not include those binaries compatible with xen, and instead, 4.2 is currently built with binaries from ovmf-edk package that does not support xen.

This causes uefi enabled HVMs on 4.2 to get stuck on Guest has not initialized the display (yet).

To fix that, we need to add compatible binaries ourselves by installing edk2-ovmf-xen package.
Luckily for us, it is available in fedora 37 repo. So in dom0, type the following:

$ sudo qubes-dom0-update edk2-ovmf-xen

This should add xen compatible ovmf binaries to /usr/share/edk2/xen/.

Next, we should replace the old ovmf.bin in /usr/libexec/xen/boot/ with the new ovmf.bin file from /usr/share/edk2/xen/. But before that, we should copy the existing ovmf.bin before replacing it (just in case).

$ sudo cp /usr/libexec/xen/boot/ovmf.bin /usr/libexec/xen/boot/ovmf.bin.old
$ sudo cp /usr/share/edk2/xen/OVMF.bin /usr/libexec/xen/boot/ovmf.bin

Now, all uefi enabled HVMs should work.

Note: This change will not affect any other qube, only HVMs with uefi feature enabled.

Download OpenBSD .iso file

First, we need to download the installation image for OpenBSD.

Go to OpenBSD’s snapshot directory, and download installX.iso, SHA256 and SHA256.sig files. Where ‘X’ represents the version number.

We won’t be using the current version of OpenBSD, which is 7.4. Instead, we are going to use snapshots.
Why? Because snapshot .iso images include EFI boot loaders that allow them to boot on EFI machines (no more hacky USB installs. yay!).
More info about that on mailing list

Verify the installation image

We need to check if the downloaded installX.iso file is valid. To do so, we need signify, and signify-openbsd-keys packages.
To install them, in the terminal of your appvm type the following (I’ll be using debian based appvms):

$ sudo apt install signify-openbsd signify-openbsd-keys

This should install the signify tool and add all required public keys used to verify the installation image.

Now that we have downloaded all of the required tools, we need to verify the installation image. This can be done with the following two lines:

$ sha256sum -c --ignore-missing SHA256
$ signify-openbsd -Cp /usr/share/signify-openbsd-keys/openbsd-x-base.pub -x SHA256.sig installX.iso

(Skip this if you are using appvm based on debian-12. openbsd-signify-keys package is out of date and it does not contain recent keys)

Output for the first command should be minirootX.iso: OK, and for the second command it should be Signature Verified minirootX.iso: OK

Creating qube

Go to Menu > Qube Manager > select “New Qube”

  • Name: openbsd-vm
  • Type: StandaloneVM (fully persistent)
  • Template: (none)
  • Networking: default

When a “Boot openbsd-vm from device” dialog shows up, choose your installX.iso file and click ok.
Then we need to halt the qube by either killing it from “Qubes Manager” or by typing “reboot” when OpenBSDs bootloader prompt shows up.

Enabling audio and usb support, and enabling uefi

Audio and usb can be enabled using the following two commands in dom0:

$ qvm-features openbsd-vm audio-model ac97
$ qvm-features openbsd-vm stubdom-qrexec 1

And uefi:

$ qvm-features openbsd-vm uefi 1

Creating custom libvirt config

We need to modify libvirt configuration for this qube, as xbf driver causes kernel panic on OpenBSD when booting under UEFI.
This will make disks show as SATA disks instead of IDE disks when xbf driver is not loaded.

First, we need to make /etc/qubes/templates/libvirt/xen/by-name/ if it does not exist. In dom0 type:

$ sudo mkdir -p /etc/qubes/templates/libvirt/xen/by-name/

Then we need to copy the main config template (This copy will be used only when running vm named openbsd-vm):

$ sudo cp /usr/share/qubes/templates/libvirt/xen.xml /etc/qubes/templates/libvirt/xen/by-name/openbsd-vm.xml

Then find the following text (~line 123):

                <disk type="block" device="{{ device.devtype }}">
                    <driver name="phy" />
                    <source dev="{{ device.path }}" />
                    {% if device.name == 'root' %}
                        <target dev="xvda" />
                    {% elif device.name == 'private' %}
                        <target dev="xvdb" />
                    {% elif device.name == 'volatile' %}
                        <target dev="xvdc" />
                    {% elif device.name == 'kernel' %}
                        <target dev="xvdd" />
                    {% else %}
                        <target dev="xvd{{dd[counter.i]}}" />
                        {% if counter.update({'i': counter.i + 1}) %}{% endif %}
                    {% endif %}

                    {% if not device.rw %}
                        <readonly />
                    {% endif %}

                    {% if device.domain %}
                        <backenddomain name="{{ device.domain }}" />
                    {% endif %}
                    <script path="/etc/xen/scripts/qubes-block" />
                </disk>

And change it to this:

                <disk type="block" device="{{ device.devtype }}">
                    <driver name="phy" />
                    <source dev="{{ device.path }}" />
                    {% if device.name == 'root' %}
                        <target dev="sda" />
                    {% elif device.name == 'private' %}
                        <target dev="sdb" />
                    {% elif device.name == 'volatile' %}
                        <target dev="sdc" />
                    {% elif device.name == 'kernel' %}
                        <target dev="sdd" />
                    {% else %}
                        <target dev="sd{{dd[counter.i]}}" />
                        {% if counter.update({'i': counter.i + 1}) %}{% endif %}
                    {% endif %}

                    {% if not device.rw %}
                        <readonly />
                    {% endif %}

                    {% if device.domain %}
                        <backenddomain name="{{ device.domain }}" />
                    {% endif %}
                    <script path="/etc/xen/scripts/qubes-block" />
                </disk>

Installing OpenBSD

I won’t explain how to install OpenBSD here. You’re gonna need to do that by yourself.
But I will show the basic configuration that needs be to done so you can have a bootable system with proper resolution.

Install configuration

First, boot openbsd-vm with installX.iso (Qubes Manager → right click on openbsd-vm → settings → advanced → “boot qube from CD-rom”)

When you get to bootloader, type the following:

	machine gop 22
	boot -c

machine gop sets output of gop to specified mod/resolution (in our case, 1920x1080). Change this to number that represents resolution you want. see machine gop for all modes
boot -c causes the kernel to go into boot_config. Here we’ll disable some stuff.

When in boot_config, type the following:

	disable xbf
	quit

disable xbf disables xbf driver that causes panic on boot.
quit continues the boot.

Now, continue with installation.

First boot

On the first boot, when we get to the bootloader screen, we need to repeat the previous steps.

	machine gop 22
	boot -c
	disable xbf
	quit

It should now boot.
(If for some reason, qube boots back into installation, in dom0 type: qvm-block detach openbsd-vm vm-name:loop0)

Making changes persistent

After we boot into the system, we need to make our changes permanent so we don’t need to apply them every time we boot our qube.
This can be done using config.

Open your terminal and switch to root using su. Then type:

# config -e -o /bsd.new /bsd

Now, we can generate modified kernel with disabled xbf:

	disable xbf
	quit

And finally, we want to edit /etc/boot.conf so we can use that modified kernel and automatically set gop to specified mode.

	machine gop 22
	boot bsd.new

And, we’re done!

Old setup (for Qubes OS 4.1)

Old Tutorial for Qubes OS 4.1
  1. Create OpenBSD installation USB with image that includes sets. Qube VM won’t boot from .img file directly.
  2. Create a new Standalone VM named openbsd-vm.
  3. Enable audio and usb support (stubdomain).
	qvm-features openbsd-vm audio-model ac97
	qvm-features openbsd-vm stubdom-qrexec 1
  1. Enable UEFI in OpenBSD qube
	qvm-features openbsd-vm uefi 1
  1. We need to modify libvirt configuration for this qube, as xbf driver causes kernel panic on OpenBSD when booting under UEFI. This will make disks to show as SATA disks instead of IDE disks when xbf driver is not loaded.

    4.1. Copy default config file.

    sudo cp /usr/share/qubes/templates/libvirt/xen.xml /etc/qubes/templates/libvirt/xen/by-name/openbsd-vm.xml
    

    4.2. Modify below config segment in openbsd-vm.xml:

		from this:
		
                <disk type="block" device="{{ device.devtype }}">
                    <driver name="phy" />
                    <source dev="{{ device.path }}" />
                    {% if device.name == 'root' %}
                        <target dev="xvda" />
                    {% elif device.name == 'private' %}
                        <target dev="xvdb" />
                    {% elif device.name == 'volatile' %}
                        <target dev="xvdc" />
                    {% elif device.name == 'kernel' %}
                        <target dev="xvdd" />
                    {% else %}
                        <target dev="xvd{{dd[counter.i]}}" />
                        {% if counter.update({'i': counter.i + 1}) %}{% endif %}
                    {% endif %}

                    {% if not device.rw %}
                        <readonly />
                    {% endif %}

                    {% if device.domain %}
                        <backenddomain name="{{ device.domain }}" />
                    {% endif %}
                </disk>

		to this:
		<disk type="block" device="{{ device.devtype }}">
                    <driver name="phy" />
                    <source dev="{{ device.path }}" />
                    {% if device.name == 'root' %}
                        <target dev="sda" bus="virtio" />
                    {% elif device.name == 'private' %}
                        <target dev="sdb" bus="virtio" />
                    {% elif device.name == 'volatile' %}
                        <target dev="sdc" bus="virtio" />
                    {% elif device.name == 'kernel' %}
                        <target dev="sdd" bus="virtio" />
                    {% else %}
                        <target dev="sd{{dd[counter.i]}}" bus="virtio" />
                        {% if counter.update({'i': counter.i + 1}) %}{% endif %}
                    {% endif %}

                    {% if not device.rw %}
                        <readonly />
                    {% endif %}

                    {% if device.domain %}
                        <backenddomain name="{{ device.domain }}" />
                    {% endif %}
                </disk>
                
  1. Start qube and enter OVMF menu. Attach install USB to VM, and boot from it.
  2. When in bootloader, type:
	machine gop 22 -> sets output of gop to 1920x1080. Change this to number that
		represents resolution you want. see `machine gop` for all modes
	boot -c	-> Enter boot_config 
  1. When in boot_config (mode after entering ‘boot -c’), disable xbf driver with 'disable xbf' then 'quit'.
  2. Continue with the installation as normal (i wont explain that here.)
  3. On first boot, repeat 6. and 7 as changes we did were not permanent (yet).
  4. Next we need to make kernel modifications permanent. this can be done with entering config -e -o /bsd.new /bsd command as root
    10.1. Type disable xbf and then quit. This will generate new modified kernel with disabled xbf driver.
  5. After that we need to tell bootloader that we want to use a modified kernel with the automatic setting of gop. To do that, put the following in /etc/boot.conf:
    	machine gop 22
    	boot bsd.new
    
  6. Thats it. You should now have Fully working OpenBSD Qube with working USB passthrough, audio and correct resolution for full screen usage.
  • Known problems:
    • Was unable to use microphone.
    • There is unhandled scsi interrupt that happens when installing OpenBSD and when shutting down a Qube. It does not cause any problems and it is not shown under normal use.
8 Likes

While this particular solution is not something I want to pursue, the details shared are insightful and helpful. Thank you @Xuni for posting this.

1 Like

No problem :slightly_smiling_face: :+1:

Thanks! <3

2 Likes

Trying this out on Qubes 4.2. I’m getting “Guest has not initialized the display (yet)” displayed on console on step 5.

I’ve tried changing the video-model from vga to cirrus and virtio etc. The only two that work are vga and cirrus and both show the same screen message. Nothing comes up in /var/log/libvirt/libxl/libxl-drver.log

Any suggestions?

Here are the logs (/var/log/xen/console/guest-OpenBSD73-dm.log):

[2023-09-20 22:25:48] + qemu -sandbox 'on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny' -chardev 'pipe,path=/tmp/qmp/qemu,id=m' -mon 'chardev=m,mode=control' -chardev 'socket,server,nowait,path=/tmp/qemu.qmp,id=m2' -mon 'chardev=m2,mode=control' -xen-domid 117 -no-shutdown -nodefaults -no-user-config -name OpenBSD73 -display none -device 'VGA,vgamem_mb=16' -boot 'order=dc' -device 'usb-ehci,id=ehci' -device 'usb-tablet,bus=ehci.0' -device ac97 -smp '2,maxcpus=2' -device 'rtl8139,id=nic0,netdev=net0,mac=00:16:3e:5e:6c:00' -netdev 'type=tap,id=net0,ifname=vif117.0-emu,br=xenbr0,script=no,downscript=no' -display 'qubes-gui,domid=0,log-level=0' -machine 'xenfv,suppress-vmdesc=on' -m 984 -drive 'file=/dev/xvda,if=ide,index=0,media=disk,format=host_device,cache=writeback' -drive 'file=/dev/xvdb,if=ide,index=1,media=disk,format=host_device,cache=writeback' -drive 'file=/dev/xvdc,if=ide,index=2,media=disk,format=host_device,cache=writeback' -audiodev 'pa,id=qemupa,server=unix:/tmp/pa.sock' -device 'nec-usb-xhci,id=xhci'
[2023-09-20 22:25:48] qemu: -chardev socket,server,nowait,path=/tmp/qemu.qmp,id=m2: warning: short-form boolean option 'server' deprecated 
[2023-09-20 22:25:48] Please use server=on instead 
[2023-09-20 22:25:48] qemu: -chardev socket,server,nowait,path=/tmp/qemu.qmp,id=m2: warning: short-form boolean option 'nowait' deprecated 
[2023-09-20 22:25:48] Please use wait=off instead 
[2023-09-20 22:25:48] + tee /proc/self/fd/2
[2023-09-20 22:25:48] + echo '{"execute": "qmp_capabilities"}'
[2023-09-20 22:25:48] {"execute": "qmp_capabilities"} 
[2023-09-20 22:25:48] {"QMP": {"version": {"qemu": {"micro": 0, "minor": 1, "major": 6}, "package": ""}, "capabilities": ["oob"]}} 
[2023-09-20 22:25:48] audio: Device ac97: audiodev default parameter is deprecated, please specify audiodev=qemupa
[2023-09-20 22:25:48] E: [vchan-sink] module-vchan-sink.c: .[1;31msource cork req state =0, now state=1.[0m 
[2023-09-20 22:25:48] E: [vchan-sink] module-vchan-sink.c: .[1;31msink cork req state =0, now state=1.[0m 
[2023-09-20 22:25:48] qubes_gui/init: 573
[2023-09-20 22:25:48] qubes_gui/init: 582 
[2023-09-20 22:25:48] qubes_gui/init: 584 
[2023-09-20 22:25:48] qubes_gui/init[611]: version sent, waiting for xorg conf [2023-09-20 22:25:48] {"return": {}}  
[2023-09-20 22:25:48] + '[' '!' -e /tmp/qemu.qmp ]
[2023-09-20 22:25:48] + '[' -e /proc/self/fd/4 ] 
[2023-09-20 22:25:48] + '[' -e /proc/self/fd/3 ] 
[2023-09-20 22:25:48] + true
[2023-09-20 22:25:48] + printf '==== Press enter for shell ====\n' 
[2023-09-20 22:25:48] ==== Press enter for shell ==== 
[2023-09-20 22:25:48] + read
[2023-09-20 22:25:48] + vchan-socket-proxy '--mode=server' '--reconnect-marker=1' --verbose 0 device-model/117/qmp-vchan /tmp/qemu.qmp 
[2023-09-20 22:25:48] br0: port 2(vif117.0-emu) entered blocking state 
[2023-09-20 22:25:48] br0: port 2(vif117.0-emu) entered disabled state 
[2023-09-20 22:25:48] device vif117.0-emu entered promiscuous mode 
[2023-09-20 22:25:48] br0: port 2(vif117.0-emu) entered blocking state 
[2023-09-20 22:25:48] br0: port 2(vif117.0-emu) entered forwarding state 
[2023-09-20 22:25:48] qubes_gui/init[622]: got xorg conf, creating window 
[2023-09-20 22:25:48] qubes_gui/init: 632 
[2023-09-20 22:25:48] configure msg, x/y 1600 328 (was 0 0), w/h 640 480
1 Like

I edited your post so it is easier to read. I hope that’s fine.

That’s fine. Thanks for doing that.

1 Like

How do you do that? When I start the qube, a qube HVM window is displayed with the message Guest has not initialized the display (yet) inside, like @thewanderer

Hmmmm… that’s weird. It works on my PC running Qubes OS 4.1

Maybe there are some regressions on 4.2?
I’ll try and reproduce it on 4.2 when I get time to upgrade (and test it).

here are some images that show it working. (forgot to post them in the first post…):

btw. I’m currently busy (learning, finding job etc.), but when I get time I can make video tutorial if anyone is interested?

maybe, I didn’t try on 4.1

:+1:

I found my old thinkpad. I’ll try and install 4.2 on it and see whats wrong.
Until then, I edited my first post and added note about this tutorial not working on Qubes OS 4.2.

Thank you very much :pray:

1 Like

Yes thank you. I would also like to get OpenBSD on Qubes.

2 Likes

Ok, so I installed Qubes OS 4.2 on my old laptop.
And from my limited testing, it looks like booting HVMs via UEFI boot in 4.2 is broken (by setting qvm-features <vm-name> uefi 1).

I’ll report this issue on Github. Until then, If anyone wants OpenBSD HVM with more features, it is better to use Q4.1.

1 Like

This tutorial was updated and changed to work with Qubes OS 4.2.
(It still contains the old unchanged tutorial for 4.1 below if somebody needs it)

1 Like

Thanks a lot for the guidance.

Here are a few updates with Qubes 4.2.3 and OpenBSD 7.6

OVMF fixup

File is not available anymore in current edk2-ovmf-xen package, need to get it from 4.1 package.
You can’t have large display without it.

[user@dom0 ~]$ ls /usr/libexec/xen/boot/
hvmloader  qemu-stubdom-linux-full-kernel  qemu-stubdom-linux-kernel  xen-shim
ovmf.bin   qemu-stubdom-linux-full-rootfs  qemu-stubdom-linux-rootfs
/usr/libexec/xen/boot/ovmf.bin

/usr/share/edk2/xen/:
OVMF.fd
[user@dom0 ~]$ rpm -qi edk2-ovmf-xen
Name        : edk2-ovmf-xen
Version     : 20230524
Release     : 3.fc37
Architecture: noarch
Install Date: sam. 23 nov. 2024 21:53:15
Group       : Unspecified
Size        : 2138759
License     : BSD-2-Clause-Patent and OpenSSL
Signature   : RSA/SHA256, lun. 26 juin 2023 13:40:03, Key ID f55ad3fb5323552a
Source RPM  : edk2-20230524-3.fc37.src.rpm
Build Date  : lun. 26 juin 2023 12:55:24
Build Host  : buildhw-x86-10.iad2.fedoraproject.org
Packager    : Fedora Project
Vendor      : Fedora Project
URL         : http://www.tianocore.org
Bug URL     : https://bugz.fedoraproject.org/edk2
Summary     : Open Virtual Machine Firmware, Xen build
Description :
EFI Development Kit II
Open Virtual Machine Firmware (Xen build)
[user@dom0 ~]$ rpm -ql edk2-ovmf-xen
/usr/share/edk2
/usr/share/edk2/xen
/usr/share/edk2/xen/OVMF.fd
/usr/share/licenses/edk2-ovmf-xen
/usr/share/licenses/edk2-ovmf-xen/LICENSE.openssl
/usr/share/licenses/edk2-ovmf-xen/License-History.txt
/usr/share/licenses/edk2-ovmf-xen/License.OvmfPkg.txt
/usr/share/licenses/edk2-ovmf-xen/License.txt
/usr/share/qemu
/usr/share/qemu/firmware

Getting ovmf.bin from https://yum.qubes-os.org/r4.1/current/host/fc32/rpm/xen-runtime-4.14.6-4.fc32.x86_64.rpm per UEFI enabled HVMs hang on "Guest has not initialized the display (yet)" ¡ Issue #8625 ¡ QubesOS/qubes-issues ¡ GitHub

[user@disp9516 Downloads]$ mkdir xen-runtime-4.14.6-4
[user@disp9516 Downloads]$ cd xen-runtime-4.14.6-4
[user@disp9516 xen-runtime-4.14.6-4]$ rpm2cpio ../xen-runtime-4.14.6-4.fc32.x86_64.rpm | cpio -idmv
./etc/bash_completion.d/xl.sh
./etc/logrotate.d/xen
./etc/sysconfig/xencommons
./etc/xen
[...]

recover on dom0

dom0$ qvm-run --pass-io dispXXX `cat /home/user/Downloads/ovmf.bin` > /tmp/ovmf.bin
dom0$ file /tmp/ovmf.bin
dom0$ sha256sum /tmp/ovmf.bin
dom0$ sudo cp /usr/libexec/xen/boot/ovmf.bin /usr/libexec/xen/boot/ovmf.bin.20241124
dom0$ sudo cp /tmp/ovmf.bin /usr/libexec/xen/boot/ovmf.bin

OpenBSD install process

Small comment, network autoconf/dhcp does not work. You must set IP, dns and default route statically.
Still need to disable xbf