I just posted another thread about how my qubes install got nuked. I managed to back up my files to reinstall qubes in case I can’t figure out the rescue system. However all my data seems to be stored in little .img files in /var/lib/qubes. Will these .img files work after a reinstall if I just create VM’s with the same names and then dump the old .img files in the new /var/lib/qubes, or will it just cause more errors and surprises? If not, how do I open them to retrieve my data? I really need my data back or I’ll be spending the whole week figuring out how to configure every little linux thing from scratch… this sucks.
@Quben: welcome to the Qubes OS community. Please see our code of conduct to get a feel for how we like to interact here.
I’ve taken the freedom to edit your post to make it adhere to said CoC and would like to encourage you to participate in this forum without excessive swearing or behavior that would “reasonably be considered inappropriate in a professional setting”.
I sincerely hope you are getting the help you need with your technical issue.
1 Like
Ah yes, I wonder where my sense of conduct went. Oh right, it’s inside one of these .img files that I’m stuck here waiting for information about before I can do a damn thing with this system.
Come on one of y’all has got to know what to do with these
Then I’m assuming your old installation didn’t use the default installation layout (LVM Thin) but probably Btrfs?
That should work if your new installation uses a compatible installation layout.
And if the old installation was already R4.1 and not a previous Qubes release, you should even be able to get away with swapping out the whole /var/lib/qubes directory tree, which would recreate all your VMs automatically. Untested:
- Shut down all VMs in the new Qubes installation
systemctl mask --now --runtime qubesdsudo mv -T /var/lib/qubes /var/lib/qubes.out-of-the-waysudo cp -Ta --sparse=always /mnt/your-backup/var/lib/qubes /var/lib/qubes- Reboot
Step 4 depends on your backup having preserved proper file ownership and permissions.
Btrfs? No, why? Pretty sure everything disk wise was on default both times
Anyway your fix didn’t work. There’s a problem. For some reason, the backup I made of /var/lib/qubes ended up being way larger than the Qubes partition it originally came from. I made a tar with tar -cvf and it created a 350GB file while the Qubes partition wasn’t larger than 60GB. So now when I try to restore, there isn’t enough space. It must’ve been using some sort of compression that didn’t translate into the tar archive.
I had limited success restoring only the app vm’s in the sense that I got my old qubes layout to show up in Qubes Manager, but I still couldn’t launch any qubes (not even sys-net). I was trying to work from there and link the old appvm’s to new templatevm’s but then I kinda ran into a snag… I may have accidentally deleted every copy of the new templatevm’s entirely. I think that copy command of yours isn’t meant to be ran within the same disk lol, somehow it made symlinks out of everything and the backup I made was destroyed. So now since I have to reinstall Qubes again, I’ll restore to the old install and give anaconda rescuer another shot before installing again.
Is there no way at all to just “mount” the individual images? I can probably reconstruct everything I need in fresh qubes if I can just get access to the old files
If the old installation had been using defaults then there would have been no .img files. By default, VM data is stored in LVM volumes.
Can you post two excerpts from your old system’s /var/lib/qubes/qubes.xml to get to the bottom of this:
- The
<pools>section - From the outermost
<properties>section, every line that starts with<property name="default_pool
Yeah the tar archive should have been created with --sparse.
If you extract those ~350 GB in it to a temporary location (running tar as root, so that ownership is preserved), the copy process to the final location can shrink your data to the original size.
???
private.img is a raw ext4 disk image that can be mounted as-is. Don’t do this in dom0 though: Not just the files stored inside of the filesystem image, but the image structure itself can be malicious. I’d copy or attach each .img to its own individual DisposableVM (with no netvm assigned), and mount it there.
I’m back in the old install now so I’ll try creating a tar with --sparse and see if that works. I didn’t even think about extracting to a temporary location first but I’ll try that this time. I’ll report back when I’m back in the new install… for like the third time today…
For some reason I was working under the assumption mount only worked with block devices… Apparently either I was wrong or these .img files count as block devices because I just mounted one. Thanks for the precaution too - I was gonna do all of this inside dom0 otherwise lol
I guess I should be set now. When I get back to the new install (5 fudging hours from now) I’ll either extract the sparse’d tar properly (preferably), OR manually copy the contents of each .img file and rebuild all my vm’s from that. I’ll let you know how it goes. Thanks!!
Hopefully you’ve chosen an alternative installation layout (Btrfs, or “Standard Partition” with XFS) that’s compatible with your situation, so you don’t have to reinstall a fourth time.
When you pass a regular file to mount, it automatically sets up a loop device.
Ahh good to know! Thanks
So yeah I’m still struggling. I “successfully” extracted the new sparse’d tarball to /var/lib/qubes, however, the partition got corrupted and Qubes refused to boot. I booted off the live usb and did a fsck and it found a bunch of errors, and 1/3rd of my /var/lib/qubes was gone (according to du -sh). Right now I’m trying a copy with rsync and I’ll try to unmount the volume properly this time when it’s done - that may have been what caused the problem the first time.
One thing I’m worried about is every time I mount the /dev/mapper/qubes_dom0-root it automatically detects and adds all the VM images to /dev/mapper. So me copying stuff into /var/lib/qubes may be conflicting with whatever mounting mechanism those are using. Would you know how to turn this off?
Edit: also, it turns out I had to resize the /var/lib/qubes partition before doing any of this. It defaults to 20GB and expands with use within Qubes, but I had to resize it manually to fit all my old files without it complaining about running out of space (shoutout to this guy)
EDIT 2: Phew, my filesystem was not fully corrupted. Qubes booted up fine with the default /var/lib/qubes. Back to attempting to copy stuff… grr…
Okay what the hell… I just installed Qubes fresh, resized the dom0 pool to 100gb, rebooted the system to make sure it persisted, ran a fsck, all good. Then I booted into the live usb and tried to copy everything with cp -Ta and this is what I got along with a corrupted filesystem: https://i.ibb.co/LZxCBWF/IMG-20221011-114522074-HDR.jpg (note: /mnt/root is qubes_dom0-root and /mnt/wdb is the location of the backup)
The total backup size is 66gb (properly sparse’d) so it should’ve fit with no problems, and as you can see I copied it elsewhere in /var/lib as to not disrupt the volumes mapped to /dev/mapper from /var/lib/qubes. What the hell is going on?
I’m calling guinness records about getting the most Qubes installs in a single week. God damn this is ridiculous. I feel like Qubes relies too much on its own broken backup/restore function, seeing how all other recovery methods have failed explosively
@rustybird I finally have some good news. I gave up trying to mess with the /var/lib/qubes directory structure (which I’m now convinced is CURSED) and instead focused on mounting individual images, and I have now successfully restored one of my old qubes. I was worried I would lose my command history (I’m too lazy for scripts rofl) but it is all there. Even my custom terminal size and colors are back. I’m so happy lol. I should be able to restore the rest of them this way. Thanks again for all your help!!
One thing I’m still tripping hard about is exactly how the .img files relate to their equivalents in /dev/mapper. The qube I restored actually didn’t have any .img files in its folder, even after I saved one file within that qube. I had to mount it via /dev/mapper/qubes_dom0-vm--my--qube--private. Maybe this discrepancy has something to do with why I couldn’t copy files to /var/lib/qubes earlier?