Then you can boot from some LiveCD and mount the dom0 there to copy the files.
But it’s better to debug the issue on Qubes OS installed on another disk first.
Thanks for this, figured out what versions of packages were there before.
Are these qubes specific packages? Trying to get them in a Fedora LiveCD but they don’t exist. Where can I download them?
Yes, they are Qubes-specific.
You can download them from the repository:
Index of /r4.2/current/dom0/fc37/rpm/
You can check the signature of downloaded package like this:
Anyone got an AppVM or Debian standalone solution for Mullvad VPN? - #16 by unman
Tried:
- boot into Fedora live disk
- from here: Index of /r4.2/current/dom0/fc37/rpm/
I downloaded: xen-hvm-stubdom-linux-4.2.9-1.fc37.x86_64.rpm AND xen-hvm-stubdom-linux-full-4.2.9-1.fc37.x86_64.rpm - mount my Qubes OS encrypted harddrive
- copy downloaded packages to home directory
- reboot into Qubes
- ran
rpm -K <each package name>
to verify signatures - ran
sudo dnf localinstall <each package name> - qvm-start sys-net
Got same results.
Tried downgrading xen, xen-hypervisor, xen-libs, xen-licenses and xen runtime to 4.17.3-4, but it gave me error:
The operation would result in removing the following protected packages: qubes-core-dom0
So not sure how to proceed.
I guess last thing to try is to downgrade the Xen packages:
[user@dom0 ~]$ dnf list installed | grep 4.17.3-5
python3-xen.x86_64 2001:4.17.3-5.fc37 @qubes-dom0-cached
xen.x86_64 2001:4.17.3-5.fc37 @qubes-dom0-cached
xen-hypervisor.x86_64 2001:4.17.3-5.fc37 @qubes-dom0-cached
xen-libs.x86_64 2001:4.17.3-5.fc37 @qubes-dom0-cached
xen-licenses.x86_64 2001:4.17.3-5.fc37 @qubes-dom0-cached
xen-runtime.x86_64 2001:4.17.3-5.fc37 @qubes-dom0-cached
But there could be some problems with dependencies which could require you to downgrade some other packages as well.
Tbh it’d be much easier if you could install test Qubes OS on a separate drive and update the packages there without updating just Xen packages (install a specific Xen version that worked for you and exclude the Xen packages from the updates).
Yes, qubes-core-dom0 (see above)
Well, that’d be a pain to go through all the package dependencies and downgrade them as well.
You’ll try to downgrade qubes-core-dom0 and it could require another 10 packages to downgrade and these 10 packages could require another 10 packages each to downgrade.
Maybe there is a way to list all the dependencies that you need to downgrade as well to downgrade specific package, but I don’t know of it.
Maybe you can get the list of installed packages since the last working ones from dnf history and just downgrade them all.
But this list could be quite big.
But you can try to downgrade
python3-xen.x86_64 2001:4.17.3-5.fc37 @qubes-dom0-cached
That you’ve missed as marmarek suggested, maybe that’d be enough.
1 Like
That did it. Including python3-xen in the list of packages I was downgrading for xen allowed it to work. Now everything is booting again.
Guess I just won’t update dom0 for a bit.
Thanks so much for walking me through this process today!
You can exclude the Xen packages from dom0 updates by editing the /etc/dnf/dnf.conf in dom0 and adding exclude=pkg1 pkg2 ... there:
Got the solution from Tonux599 on github:
I added
GRUB_CMDLINE_XEN_DEFAULT="$GRUB_CMDLINE_XEN_DEFAULT spec-ctrl=ibpb-entry=no-pv"
to the end of /etc/default/grub and updated with grub2-mkconfig -o /boot/grub2/grub.cfg and then reboot. Then updated Xen from 4.17.3-4 to 4.17.4-2 and other xen-hvm-stubdom* packages from 4.2.9 to 4.2.12, rebooted again, and now all my Qubes boot with latest Xen.
NOTE This is opening up a big security hole as marmarek explains:
So, I’m afraid there is not much hope for this old-ish system… The only way to make the system kinda-usable has a tradeoff with security here, by disabling the mitigation for PV domains (which should mean just stubdomains, make sure you don’t have any really untrusted PV qubes) with
spec-ctrl=ibpb-entry=no-pv. It does mean that stubdomain will be able to mount the attack, potentially leaking memory of any other VM (so isolation of sys-net/sys-usb and any other HVM becomes weaker). If that is not an acceptable risk, blame AMD for making buggy CPU, and replace with something newer…
And the Qubes team has declined to attempt to address this bug in the future.