@NicholasTrust contacted me directly, off channel, for this issue. Replying here, but this is not a new issue. Agreeing with @unman with most probable cause being lack of space under template, where normally apt refuses to apply update if forecasted consumed space is bigger than actual available space. But that might have been gone wrong.
This issue should be renamed to “Template doesn’t boot after update” for prosperity and usefulness in the future for others.
I would advise getting safe belts first:
qvm-clone debian-11 debian-11-fixed
qvm-console-dispvm as of 09 february 2023 gives:
Usage: /usr/bin/qvm-console-dispvm [--autostart] [--] vmname
Connects to VM console throught DispVM using the qubes.ShowInTerminal RPC service.
With --autostart, start the VM first.
So if the RPC service will still be listening into broken (to be fixed) template named debian-11-fixed, it would be
qvm-console-dispvm --autostart debian-11-fixed which will open a terminal in dispvm to the broken template.
And then resume from previously given instructions.
That would be running df -h from the dispvm terminal.
On a template:
/dev/mapper/root here is the template OS layer filesystem. You want to make sure there is enough free space there and if that was the cause. You can increase that root filesystem space from Qubes Settings of that template, there named System storage max size where Private storage max is maximum size of /home (qubes persistence filesystem). Again doing changes at that layer might require a reboot reboot (df -h might infirm this) but changes need to be applied to be useful.
Once you can open a terminal normally under debian-11-fixed (which would mean you fixed the template, hence its hypothetical name), you might want to create a new qube using that template first and see if that qube boots normally. Then delete that testing qube.
Then you can use the Qubes Template Manager to have all you qubes point to now fixed debian-11-fixed template. Then Qubes manager to rename debian-11 to debian-11-backup, and debian-11-fixed to debian-11. And then qvm-remove debian-11-backup later on, requiring qvm-prefs debian-11-backup installed_by_rpm False otherwise qvm-remove debian-11-backup will fail, saying that it was installed by rpm…
As usual, you need to have all qubes depending on those templates shutdown. I use qvm-shutdown --force --all --wait from dom0 because I know what i’m doing, but you could point to individual qubes directly through: qvm-shutdown --force NameOfQubeDependingOnTemplate1 NameOfQubeDependingOnTemplate2.
Make sure everything works as expected by pointing one qube, or creating a new testing qube pointing to your repaired template first. Otherwise causing more issue then anything else.
You could also just redownload debian-11 template from the qvm-template-gui tool from dom0.
In the future, when you see that something broke, that a template is not providing OS for a qube, you can easily revert the template to its prior state. Qubes provides that safeguard for all qubes and templates. The proper way to do this preventively would be to create a new qube using that template in question. If that doesn’t boot normally, you can clone the template in its current state to a backup (qvm-clone debian-11 debian-11-backup) and then call qvm-volume revert debian-11:root and retry booting your test qube using that template.
Hope things are clearer here. Note that in case a Template or a qube is consuming space reaching the limits of defined maximum System or Private storage, QubesOS is warning the user of such case on top right corner of xfce, asking the user to take preventive action, permitting to click the warning pointing directly to that Template/qube settings to increase assigned maximum space.
@adw : qvm-volume revert should get more publicity for a while now. Is rollbacking, directly in qubes manager, a planned feature? A hook on failing to start a qube, or a failed template update should most probably guide the user into reverting to prior of such update?
I see qvm-volume revert NameOfQube:root to be really useful for templates for rollback purposes, with no bad consequcnes. If issue like this one are opened frequently (forum/github), then QubesOS has a UX problem, where this issue shows lack of knowledge to be able to resolve easily and becoming an emergency situation. It could be a “lay back and relax, Qubes OS got your back: do you want to rollback failed update?” kinda situation instead if we really wanted to, as well as a way to promote QubesOS usage for newbies/not-so-techsavvy users who fear of breaking things.