Hey people, I can get updates directly via dnf (both check-updates --refresh and install them), and I have notification about updates being available, but updater fails to apply them (GUI application fails as well):
Possibly adding --log=DEBUG to qubes-vm-update and looking at /var/log/qubes/VMNAME.log could reveal more details and make it possible to resolve the issue.
How come I have notification that there is an update, I already have forced dnf to refresh, but running updates does nothing, until I shut down the vm?
If update algorithm uses whatever current data there is, it must’ve noticed available updates, and if it’s refreshing, it also must’ve noticed available updates.
But if it uses some other information, like the result of the last refresh, what would be the use case for --force-update without --force-upgrade?
p.s. Checking for updates when package information is stale, I guess?
In order to understand the issue, we have to understand the Template updates availability notification mechanism as well as the update itself.
The updates availability notification is controlled via two simple bash files within the template.
/usr/lib/qubes/upgrades-installed-check
/usr/lib/qubes/upgrades-status-notify.
The 2nd bash scripts simply runs the 1st one. It detects the template package manager. Checks for updates (dnf --refresh, apt update, …) and finally informs the 1st bash script via true or false. Then the 1st bash scripts tells dom0 to flip templates updates-available feature for that template. Finally Qui Domains widget (in dom0 or GUIVM) reacts to the feature change and shows the pop-up notification.
It is possible to manually run the 2nd script within the template and monitor its output.
The qubes-vm-update mechanism is different. We have to look back at history a little bit here. I believe at some point of time (Qubes OS 4.1), there used to be a different update mechanism via salt which was not perfect. So the Qubes team started developing a new update mechanism (mostly developed by Piotr Bartman). All of the files for this update mechanism are in dom0. And qubes-vm-update actually copies the content of /usr/lib/python3.1x/site-packages/vmupdate/agent to the template at /run/qubes-update/. Then the vmupdate.py executes it on the template and starts to communicate with it. The agent/worker has plugins for different package managers. But overall they do similar tasks such as refreshing, upgrading, clearing cache, reporting back update progress (not supported by all package managers).
Looking at their source for advanced users (I consider you as one) could provide excellent insight as how it is done under the hood. Once you understand the main points, it becomes very easy to understand it.
It is sometimes necessary to do upgrade tasks which could not be achieved via the package manager (dnf, apt, pacman, …). Examples are fixing the notorious SELinux bug or pipewire fix for archlinux or enabling bookworm bookworm backports. In that case, there is a special directory in the agent/source/plugins for such tasks.
That is a very polite way of saying “go read the code you lazy prick”, thanks!
And so I did. And I have started losing context somewhere around /usr/lib/python3.11/site-packages/vmupdate/agent/source/dnf/dnf_api.py. I’m not very solid about stuff.
Regardless, as far as I can tell, pgk_mng.upgrade runs upgrade from ./source/common/package_manager.py. There, refresh does nothing. It seems to be not implemented. Same goes to get_action in upgrade_internal.
I’m missing the part where it calls upgrade_internal from dnf_api.py (I suspect that this is what it really does, because upgrade_internal in common/package_manager.py doesn’t seem to be doing anything).
Because of that I don’t know how is it even possible to have a running template with updates available, and yet never apply them until reboot. I guess I should look into dnf.base.upgrade_all() for that.
Still, this UX decision is concerning. If user wants to “update all targeted VMs even if no updates are available”, it means user expects there to be an update, even if the system isn’t aware of it, therefore it must refresh package information to be useful.
Also I believe that this program is missing an important functionality: force update, force package information refresh, but don’t ignore errors.
I want to look more into this issue in the future, but no guarantees on that.