Hello,
I am trying to install Qubes OS on a new computer with one of the newly released ryzen 7000 CPU.
Does anyone is currently trying the same thing ?
It seems it require to have a linux kernel >= 6.0 and to have a xen version at least >= 4.15.
The complex part is to upgrade the xen version since pretty much everything depend on xen.
If someone have tips on how to upgrade the xen server version it would be great.
If not I will send update if I succeed in making progress.
My current path is to take the qubes-vmm-xen, update the xen version from 4.14.5 to 4.16.2, update the patchs file to work with 4.16.2. This part should be quite simple but I expect that trying to make it work with the other qubes component will be extremely long and complex. and then build the iso. Will see.
I am trying to get Qubes working with a Ryzen 6000 series and am having the same issues as you - Xen. You are not alone in this, I hope for both our sakes you uncover something.
If you are actively trying to patch/dev things we can try to chat / team up
on matrix you can reach on at neowutran@neowutran.ems.host
I expect this task to be long and complex
Seems like I need to upgrade libvirt & dom0.
Trying to upgrade dom0 to fedora 37, xen to 4.16.2 and libvirt to 8.6.0.
Xen 4.16.2 is compiling successfully with all the qubes patch now.
Libvirt 8.6.0 is compiling but didnât integrated the qubes patch yet (will be time consuming, code base is largely different from 6.6.0 )
I have been attempting this for the last few days, and havenât managed it so far.
Had resigned to put it on hold for now, until I saw this post.
Iâve tested multiple kernels and ISO versions, although some of the issues Iâve had may have been caused by mobo manufacturer / nvme.
I may reach out to you on matrix, however I agree it seems it will be a large effort, with new zen hardware (from reports) having compatibility issues with other OSâs and windows just now.
I am having some success.
Was able to patch enough things to build an iso with:
(
xen 4.16
dom0: fedora 37
libvirt 8.6
lorax fc37
and recent anaconda
)
I can confirm that I donât have the xen error saying that CPU 25 is not known.
However, a lot of work is still needed before being able to use Qubes ( things donât go past the installer), things I have noted:
Updating the vmm-xen patches for debian
Updating all the core-libvirt patches
Updating the remaining patches for anaconda (the installer)
Fix dependencies issues between anaconda and blivet ( probably just need to upgrade blivet)
Probably more things I donât think about
Fixing bugs
Fixing bugs
Will post my builder config + iso here a bit later.
If people have the same hardware issue as me, time, and some software engineering skills, i wonât say no to some help
Libvirt patches have been ported (except 2 because I belive they wonât be needed)
Found that the python code QubesOS is using is not compatible with modern python versions.
More specifically âPython 3.11 Passing coroutines is forbidden, use tasks explicitlyâ
I donât known if it is easier to patch all the qubesos code or to find workaround to downgrade python version. Or maybe instead of using fc37 as dom0, use fc36 instead.
Choose to patch the python code to use explicit task.
Was able to create the iso, run the installer (non-efi), and boot into qubes.
dom0 is fedora 37, xen 4.16.2, it seems to work.
However, a lot of work is still needed:
need to update anaconda patch
some configuration is needed to make the installer work in efi mode
need to update the vmm-xen patches for debian vm
Once it is done and confirmed that everything work, will need a lot of cleanup work in the commits and try to make this qubes os fork as easy to keep updated with upstream as possible
and then work with upstream to upgrade everything for a new future release.
In the current state it is only to show the progress I am making and for devs willing to help finishing this project. Some work is still needed before reaching a âusableâ state, and lot of work to clean up everything once the âusableâ state have been reached
Great work and progress neowutran, thanks for updating along the way.
I tried for another 2 days, attempting integrate 4.16 also.
Youâve come pretty far, well done. How stable is the build youâve managed to boot into?
iirc python 3.11 is a major change under the hood, I saw some benchmarks of 50% performance gains in certain tasks (i think more C implementations), so not too surprised thereâs some issues there.
What kernel did you use for dom0 build?
I think a lot of these AM5 boards have issues just now. Have you tried adding x2apic=false for the iommu issue? This seemed to help in some of my tests.
Currently my main difficulty is patching anaconda for the partitioning setup. A lot have changed between the fedora 37 anaconda and the fedora 32 anaconda.
Patch need to be rewritten from scratch. ( speaking about ~3 or 4 blocking patch of less than 100 lines combined. But that still some work to do. i wonât say no to help on that subject).
Once anaconda have been patched, then I expect everything to work correctly ( well, just the fedora vm, didnât ported the patch for debian vm )
For the python3.11, issue have been fixed, explicitly using tasks instead of coroutine seems to be quite simple and basically, you just need to hunt for âwait(XXXXâŚ)â and check that no coroutine is passed inside a âwaitâ.
âHow stable is the build youâve managed to boot into?â
Well, dom0 doesnât crash and I see no error in dom0. Other than that, cannot be used because of the partitioning issue and thin pool name (that are normally setup by anaconda, but havenât ported the required patch)
Update: qmemman is crashing (core dump). https://github.com/QubesOS/qubes-linux-utils/blob/master/qmemman/meminfo-writer.c Xen api have probably changed, need to read the 4.16 doc and update the calls
For the AM5 motherboard, havenât tried any additional options. I just have contacted the asus support, now I need to fill a detailled technical IOMMU bug report for their engineering team.
When you have time, can you try the iso (or build it yourself) to check that you can finish to install the thing and reach dom0 ?
You may be able to temporarily work around that by setting dom0 min/mem values equal on your startup command line and (once domU VMs can launch) disabling memory sharing for all of them (temporarily).
Struggling with the anaconda addons, I need to find a way to modify and test it quickly (recompiling the iso after each modification is ⌠), but documentation is lacking on the testing part.
For the qmemman crash settings dom0 min=max value doesnât seems to have a noticable impact. For the core dump it seems (a lot more debugging is needed to confirm) to crash here xen/tools/python/xen/lowlevel/xs/xs.c at master ¡ xen-project/xen ¡ GitHub . To support python 3.11 I need to add a patch to add âPY_SSIZE_T_CLEANâ.
From the documentation:
For all # variants of formats (s#, y#, etc.), the macro PY_SSIZE_T_CLEAN must be defined before including Python.h. On Python 3.9 and older, the type of the length argument is Py_ssize_t if the PY_SSIZE_T_CLEAN macro is defined, or int otherwise.
qmemman is fixed now (normally), it was indeed my patch that was incorrect.
So some progress.
Now when I start a new VM, vm is unresponsible, no way to send command to it.
Logs indicate that multiples things went wrong
Something about clock / timer (xen_timer / hrtimer). TSC clocksource doesnât seems to work. âMarking clocksource âtscâ as unstable because the skew is too large âŚâ âOverride clocksource tsc is unstable and not HRT compatible - cannot switch while in HRT/NOHZ modeâ âSwitched to clocksource xenâ. Then lot more logs/trace about issue with clocksource.
Then it crash with things related to disk âQubes initramfs script here:â ⌠â/dev/xvdd: Canât open blockdevâ âEXT4-fs (xvdd): mounting ext3 file system using the ext4 subsystemâ. Some more logs and nothing else
Now need to understand what is means.
The file system errors happens on standard QubesOs, so will focus on the clocksource issue
VM react when passing command through sudo xl -v console fc37, but everything is spectaculary slow.
After a very long time, qrexec is working too.
On the clock issue TSC is detecting my clock speed as ~200Mhz, while the correct value is 4500Mhz+.
Was able to launch a xterm windows, but speed is not good
Continuing to make some progress.
The TSC issue was hardware related, with another computer I doesnât have the issue. Still waiting for a reply from the asus engineering department.
Next issue is PCI passthrough.
When I try to pass the network card to sys-net, libxl crash with the following message:
âlibxl_qmp.c:1838:qmp_ev_parse_error_messages: Domain 4:Offset 0x000e:0x49090000 expands past register size (1)
libxl_pci.c:1830:device_pci_add_done: Domain 4: libxl__device_pci_add failed for PCI device 0:1:0 (rc -28)
libxl_create.c:1973:domcreate_attach_devices: Domain 4:unable to add pci devicesâ
Same error but when using PV instead of HVM:
âxen_pt_config_reg_init: Offset 0x000e mismatch! Emulated=0x0080, host=0x49090000, syncing to 0x49090000â
