Cannot get locally built xen packages to install in dom0

User Support
#1

i am running R4.1, i made some modifications to the vmm-xen component of qubes, and i am having difficulty installing the built packages in dom0.

i have copied the packages to dom0, but attempts to dnf update with the newly built packages, e.g. xen-libs give the error

“problem: the operation would result in removing the following protected packages: qubes-core-dom0”

i suspect that there is some larger set of components i need to build in order to successfully update in dom0.

i would appreciate it if someone can tell me how to go about updating these packages so i can test them.

#2

Maybe this can help?

#3

i have already used qubes-builder to fetch source code with “make get-sources”, then “make vmm-xen” to build after making modifications. this completed successfully, with the packages showing up in qubes-src/vmm-xen/pkgs/dom0-fc32/x86_64/ .

however, now that i have the packages built, i don’t see how i am supposed to update to these packages to test them in dom0. most *nix oses have ports that build and can be trivially updated using their package managment tools, but qubes will not let me upgrade individual packages in dom0 because they conflict with the meta package “qubes-core-dom0”.

do i need to build more components, so that i can upgrade the meta package qubes-core-dom0?

#4

based on the qubes-builder doc page (Qubes builder | Qubes OS), i can build a fresh iso with all these changes by doing a full build.

this means a lot more build time, having another drive to install onto and/or hw, and having to migrate a working qubes system to the new configuration after doing a fresh install, which is a pain in the ass.

1 Like
#5

Yeah, I tried installing Xen 4.16.2 in dom0.

The kernel module amdgpu has problems with resetting GPUs, especially when they’re virtualised. I’d have issues where Xorg would “implode” on resume from S3 sleep.

After tinkering with upgrading Xorg in dom0, and still having the issue persist, I wanted to see if the Xen project had figured that out in a later version.

So I compiled it in a Fedora 32 VM, moved it to dom0, installed it, remade grub.cfg, and rebooted.

It borked everything… :rofl:
(All of the Qubes daemons fail on boot, and it gets stuck on systemd)

Thank goodness it was only a testing machine :grimacing:

If you don’t mind “borking” your machine, the --force option might help…

I agree. But on the plus side, you will save on your heating costs with all the compiling you’re doing :stuck_out_tongue:

#6

You will probably break your dom0 :fire: , so do it only if you know what you do. Me, I will try the same process as described in this chapter for a template.

#7

i am simply patching the existing installed version of xen, not upgrading to new code.

from the sound of it, the qubes builder doc is very misleading:

“You can also build selected component separately. Eg. to compile only gui virtualization agent/daemon”

what is the point of telling users they can build selected components if they cannot install or test those components?

i have been running qubes for 7+ years at this point, and i have never needed to build any packages previously. that nobody seems willing or able to give me an answer to this basic question about building these packages is very disappointing.

#8

Well forgive us, but none of us appear to have actually attempted what you’re attempting. We’re doing our best to figure it out with you, though…

Unfortunately, it is not as up-to-date as it should be. I’ll also be the one of the first to admit that it’s written by the community, based on their own needs and past experiences, which is why, without some outside perspective, while it may be written with the best intentions, it might not appear “complete” to a third party…

Are you sure there isn’t an RPM package somewhere in the qubes-src directory…?

Failing that, it will have been built somewhere, some maybe create a tarball and copy it into dom0…?

I will have a look myself, but it might take a week or two…

#9

if you take the time to read what i have already posted, i have made it quite clear what i have already attempted. everything i will post in this summary below is included above.

  • i am attempting to install and test a modified version of the vmm-xen component of qubes
  • after making some modifications to the sources in the qubes-builder repo, i was able to get it to fetch sources with “make get-sources”
  • i patched the vmm-xen component and successfully built the corresponding packages with “make vmm-xen”
  • i copied the rpms i built to dom0 and attempted to install them
  • i could not install the rpms that were built because of the aforementioned conflicts with the qubes-core-dom0 metapackage “problem: the operation would result in removing the following protected packages: qubes-core-dom0”

since it is clear that it is not possible to build and update the vmm-xen component by itself via packages built from qubes-builder using the normal fedora pkg manager (dnf), there is clearly some missing tribal knowledge required to update these packages. i am seeking this tribal knowledge, which boils down to “how do qubes developers build, install, and test patches to vmm-xen?”.

the fact that qubes rides on top of xen is fine most of the time, but it is a shitshow when xen does not support the hardware properly. i am stuck on a laptop that alternates between no fan spinning and 3500+ rpm spinning with no middle ground as a result of this missing hw support in xen. see this issue for reference, which suggests patching vmm-xen as a resolution

while i have really enjoyed running qubes for the past 7+ years, the fact that joanna effectively abandoned the project to work on golem (or whatever she has pivoted to since) has not gone unnoticed by myself and many other users. i’m not asking the world here, i just need access to some undocumented tribal knowledge so i can avoid wasting many hours of my own time and/or money on new hardware.

that nobody is either willing or able to read this and point me in the right direction speaks to the extent that qubes is becoming / is a ghost town. i’ve posted on the user mailing list and posted on the forum, wtf else am i supposed to do? post this on the dev list?

Moderation policies
#10

I apologise. You did make it very clear. It was late, I was fatigued, and I was just trying to help.

@adw, any idea on how best to go about this, and who would be able to provide insight?

#11

Some ideas I will explore if I was on the same problem:

  • I will try to find a related issue in the issue list (also in the closed issues).
  • If I can’t find a related issue then I will create an issue with a lot of details
  • If I find a related issue, I will give details of my case, I will try the possible workaround from the issue. I noted you found the #4604, did you comment this issue?
  • if only 2-3 files should be changed in dom0, I will do a backup of the original files and copy the new ones (from my build), and test if it solve the problem
  • I will compare the official installed rpm(s) (available in the dom0 repository) with the custom built rpm(s) (with the help of rpm commands like : rpm -qlvp <my.rpm>, rpm -qvp --info --scripts --provides --requires --conflicts <my.rpm>, see the rpm man page)
  • I will check the dependencies and package versions (may be you should increment the release version number of your built package, may be you should install a group of linked rpms, …)
  • I will check how the Qubes-OS team publishes the new versions of the related component (git upstream with another xen repo? Apply a patch? one or many built rpms? Exlore the repository history, the updates-status/issues)
  • I will also post the commands I tried (ex: the rpm installation you tried) and the full output of the commands (see copying clipboard text from dom0 official doc)
#12

Not sure. Maybe @Demi or @unman might be able to provide some insight about this.

1 Like
#13

apologies for snapping off so hard earlier.

i avoid sharing the exact hardware i use for various reasons. the problem that i have run into is that intel 8th gen and later i7 cpus use HWP to do cpu frequency scaling, xen lacks support for this hw, and the result is i have several otherwise-useful laptops that run poorly with qubes. i have run qubes on many other laptops that did not have this cpu frequency scaling problem and that hw held up pretty well. on these i7 cpus that don’t scale frequency, i see excessive battery use, fan activity going from 0 to 3500+ rpm, and increased wear and tear on the heatsink fan.

this hw is several years old at this point and i am surprised to see xen not have support for it. i checked the 4.16.0 and 4.16.1 release notes and see no mention of HWP support being added, which is disappointing. the issue i cited (4604) describes the exact problem i experience and i was able to successfully build a patched vmm-xen component per the instructions included, but it appears there is some dance required to properly update these pkgs in dom0.

qubes is the only context in which i use xen, and my impression is that xen is more tailored to run on rackmount and whitebox machines than laptops, which may explain the lack of HWP support for intel laptop cpus. i am really surprised to see a common intel hw extension for cpu frequency scaling without support in xen, particularly 3+ years after it was added to production hw. the only things i could find that mention this support were

https://lore.kernel.org/xen-devel/20210308210210.116278-1-jandryuk@gmail.com/T/
Xen project Mailing List (search for hwp here)

short of episodically hand-building a patched vmm-xen when R4.1 xen packages get updated, it looks like i’m pretty much screwed with this lack of HWP support in xen.

does this mean that qubes will always have this problem on more recent intel laptops?

is there any chance this patch would get reviewed and incorporated into qubes’ vmm-xen patches?

#14

I have Intel i9-12900K and cpu frequency scaling works for me on Qubes 4.1 current-testing with kernel-latest.
When I change cpufreq governor to performance then I’m getting the maximum cpu frequency (5 GHz) and have the same performance as bare-metal debian in tests:

xenpm set-scaling-governor performance
#15

That’s up to @marmarek, but unless it causes problems, I see no reason why it should not.

#16

Hmm - I suspect that if Joanna were still active in Qubes, you would
have got short shrift. She wasn’t noted for patience.
You should remember that no one owes you an answer - every contributor
to Qubes has a life outside, and other factors may prevent them
from giving you the answer you seek.
I have been unable to contribute for a while now, and it could be that
other users are under pressure too.

It’s getting late now. I’ll read over this thread and post my thoughts
in the morning.

1 Like