How do I restore my fedora template's private volume?

Long story short, I wanted to organize files, so naturally I moved things from one VM to another(hundreds of gigs.) then after I closed said VMs. and suddenly the thinpool metadata shot over to 90%+. this shouldn’t have happened cuz I was just moving things around.

After that I was trying to find ways to solve that problem, but ultimately I gave up. after deleting a lot of the built-in VMs like the Debian template, sys-whonix, whonix-gw, whonix-ws. it was still at 90.3%, I even deleted my fedora 36 template VM(as I thought I did. but it was more complicated than that.) Somehow I deleted the private volume of the fedora template but not the root volume cuz all the internal VMs like sys-net rely on the fedora template.

now I can’t open sys-net, sys-firewall, or any VM, probably because of the incomplete fedora template. and when I try to increase the fedora private volume thru GUI it says:
ERROR: Basic tab:
Logical volume vm-fedora-36-private not found in volume group qubes_dom0

What do I need to do to restore or make a new fedora private volume?

I really wish the qubes team can make a function that recovers deleted VMs so we dont have to lose our data. setting 10% space aside and once you are over 90% you’re suddenly forced to delete stuff. And theres no warning or anything. that is just not a good practice

There are processes to try to mitigate this risk and there’s a
warning.

This happens because Qubes maintains snapshots of the private data - you
can see this in /dev/qubes_dom0
So when you moved “hundreds of gigs” of data, that was present in the
target “private” volume, removed from the source “private”, and still
present in the source “private-back”.
This is mentioned in the docs

In this sort of situation you can remove those weighty snapshots,
manually or by setting “revisions_to_keep” to 1, and recycling the
source qube.

I never presume to speak for the Qubes team.
When I comment in the Forum or in the mailing lists I speak for myself.

In this sort of situation you can remove those weighty snapshots,
manually or by setting “revisions_to_keep” to 1, and recycling the source qube.

during the salvage campaign ive deleted so many vms and I don’t want to fuck this up so plz guide me thru all this. I don’t even know if this is salvageable or not. so here are the things Ive noticed so far and some additional questions

  1. after entering qvm-volume info fedora-36:private it shows that there’s 0 revisions_to_keep, none available revisions for revert. and there’s no vm-fedora-36-private in /dev/qubes_dom0 now what do I do to recover fedora 36? without it the whole system is pretty much useless

  2. can I recover vms that I already deleted? cuz there’s no snapshots of them in /Dev/qubes_dom0

So when you moved “hundreds of gigs” of data, that was present in the target “private” volume, removed from the source “private”, and still present in the source “private-back”.
This is mentioned in the docs

  1. when will the “moved data” in the source “private-back” disappear? after a reboot? after a second shutdown of the source VM?

anyone knows how to fix the fedora private volume problem? I really want to be able to use my computer again

Did you consider (and being able) to rename fedora template, then to install new one, to remap all the qubes to it and to try to use it?

You’ll need to set sys-net etc to use Debian templates if that is to
work.

I cant. as I mentioned in the post, i deleted Debian template in order to get the thinpool down to 90%

how? the only template VM I have rn is the crippled fedora one. that’s why I wanted to “recover” the private volume

if I could do that I wouldn’t asked in the first place lol. Debian template was deleted to save space

If you have installation media you can pull the fedora template from
there.
If you have this on USB, and you had a sys-usb, then you will have to
reboot and remove the section from boot parameter that says
“rd.qubes.hide_all_usb”
Mount the install media in dom0 - the fedora template package is under Packages
If you used DVD, then this is likely attached to dom0 anyway.

sadly I don’t have sys-usb on or DVD drive.

I have a USB drive and I can burn fedora onto it. is this enough?

can I mount my USB to dom0?

Sorry, I just didn’t believe someone deleted all the templates expecting Qubes to function afterward, without any backup obviously.

It’s not recommended, but I don’t see any alternative to this. I’d pull out data and reinstall Qubes afterward, keeping at least one minimal aside unused.

Yes, this is your only option.
Don’t write fedora to the stick, write the Qubes installer image.

I’ve already told you what to do.
Interrupt the boot process, and edit the boot parameter - delete the
section of the line that says “rd.qubes.hide_all_usb”.
Complete the boot.
Insert the USB stick - it will be attached to dom0 with the risks
that involves.
Mount the stick at /mnt
In a terminal, cd to /mnt/Packages
Install the template with qvm-template install /mnt/Packages/TEMPLATE_NAME - you’ll have to check this, but
autocompletion will help.

You may be able to use “reinstall” instead of install - if not, you will
have to purge the existing broken template. I would take a backup first.

I never presume to speak for the Qubes team.
When I comment in the Forum or in the mailing lists I speak for myself.

I thought I could backup templates from another qubes system then restore them in the current one.

plus the 90% threshold wasn’t helping. I panicked and I needed more space!

gosh this sounds like demanding, and scary. are there detailed instructions on these steps that you mentioned?

is this still salvageable?

by the way there’s some good news, i can actually use some of my standaloneVMs, albeit without internet. the trick is just uncheck sys-net as netVM since it relies on the fedora template to work.

and one of these standaloneVMs is a fedora one. can I somehow make use of it?

can I somehow copy my fedora standaloneVM’s private volume to the fedora template one? they’re pretty much the same thing

second opinions? please! I really want to use my qubes normally.

@unman can I mount my thumb drive with no sys-usb?

I chose to not have sys-usb during initial setup of the system. in fact I combined sys-usb and sys-net together so I only have sys-net. since it relies on fedora so I can’t open sys-net

If you take the step I suggested (edit the boot parameter - delete the
section of the line that says “rd.qubes.hide_all_usb”), then the drive
will be connected to dom0.