Updating pre-4.1 templates is breaking them

User Support 4.1
#1

I recently upgraded to 4.1 with a clean and restored my templates from backup, which went smoothly. Now I’m finding that the templates are using old versions of the Qubes tools from the 4.0 repo. This is causing weird issues like not being able to move files between qubes, meno options not working, and being unable to install new templates. I’ve tried to upgrade a couple of the templates with the packages in the 4.1 repos but both of them have been a complete disaster.

Qube: sys-firewall
Template: fedora-35-minimal-firewall
I originally created this one on Qubes 4.0 where it was based on fedora-34-minimal-firewall, which is the minimal template with just the packages to be firewall and be the updatevm. I updated this to Fedora 35 on Qubes 4.1, and it seemed to work fine except for being unable to install new templates. I found the bug report talking about this, which is where I found out about this issue. I then changed the 4.0 to 4.1 in the repo conf and did the distro-sync. Everything seemed to work and I can install templates, but the firewall part no longer works. Every qube that uses it as the networking vm (most of them) cannot reach any websites. Disaster. Reverted.

Qube: i2p
Template: debian-11-minimal-i2p
This template I created on Qubes 4.0 with debian-10-minimal to isolate I2P. After restoring it on Qubes 4.1 I upgraded it to Debian 11. It kind of works, but moving files is very glitchy. Once I realize all the qubes-* packages are still 4.0, I change that to 4.1 and bullseye. Again, the install seems to go smoothly, but the template no longer starts at all. There’s an error about not being able to contact qrexec and to check the log, but the log says to check systemctl, and - how would I? Disaster again. The template is completely trashed.

What is the right way to upgrade the templates after upgrading Qubes??? There’s nothing in the docs on this for either Qubes or the Templates. Which versions of the debian and fedora templates even have the packages for 4.1?

#2

I don’t know what’s the right way to upgrade your old Qubes 4.0 templates from backup to Qubes 4.1 but I’d just install fresh templates from Qubes 4.1 repo and made the same changes to them that I did for templates in backup.

You can connect to your template console from dom0:
qvm-console-dispvm yourtemplatevm
I guess you’ll have qrexec timeout error so you need to temporary increase timeout to look for errors in console:
qvm-prefs yourtemplatevm qrexec_timeout 3600

#3

Thanks, I was able to get the console open and see the error message on the Debian template:

/usr/lib/qubes/qrexec-agent: error while loading shared libraries: libqrexec-utils.so.2: cannot open shared object file: No such file or directory

#4

In latest version qrexec-agent should use libqrexec-utils.so.3 library. It seems that qubes-core-qrexec package somehow didn’t get updated. The library comes with libqrexec-utils2 package and maybe you don’t have it as well.
You can try to reinstall them:
sudo apt-get install --reinstall qubes-core-qrexec libqrexec-utils2

#5

Unfortunately it appears that’s not possible to do with qrexec missing/broken.

Screenshot_2022-06-03_19-47-43
Screenshot_2022-06-03_19-47-43507×695 18.9 KB

#6

Then it’d be easier to just install fresh templates from Qubes 4.1 repo and make the same changes to them that you did for templates in backup.

#7

That’s not feasible in the slightest. I have over 20 different templates, some of which were not trivial to make. Also there’s no way I can justify committing the time to do that if I can’t trust they won’t break again in some equally absurd way. I’m going to remake this ONE because I need it and it seems totally lost. Then I’m going to try some more of the Fedora based templates, of which most of my templates are. I really hope the others will go more smoothly, this is one of the killer features of Qubes. I really need it to work.

#8

You can try to use template from Qubes 4.1 for updatevm and all of it’s upstream networking VMs and check if you’ll be able to update your old templates with this setup.

#9

I’ve found that updating the repo line when I’m upgrading the minimal templates SEEMS to work for the templates I’ve done already, though most of those are all the minimal template + one app or framework.

After running dnf clean all as the documentation says, run sed -i 's/4.0/4.1/g' /etc/yum.repos.d/qubes-r4.repo

I’m not sure about some aspects of the more important and complicated templates: firewall and sys-net.

On Qubes 4.0 and fedora-34-minimal-sys-net, I am not able to make backups or copy from network share to another qube. I haven’t been able to test that one Qubes 4.1 and Fedora 35 yet.

Firewall is dead and won’t pass any traffic, so I’ll switch that back to the Fedora 35 template shipped in Qubes 4.1.

I’m going to do the same to any other templates that give me the least bit of trouble. Just keeping all these templates up to date takes an exorbitant amount of time. Any attack surface reduction is just reduced with an attack on my time. Adding the troubleshooting and figuring out what magic package is needed to make things work takes more time than I have. I have a suspicion this will lead me back to having just the fedora-35 as the basis for the majority of my qubes. Remaking qubes is faster than fixing them, but I have to draw the line and limit that where it’s absolutely vital.

Qubes needs to improve handling upgrades of the templates if minimals are going to be so touted. I’ve managed full stack apps that were less work!

#10

minimal templates are not “touted” - the docs explicitly say that they
are intended only for advanced users.

Most people who advocate their use repeat this in the forum over and over.