Modern Filesystem with SNAPSHOTS

bawdyanarchist · July 19, 2024, 5:14pm

I know this probably belongs on Github, but maybe being a small voice here will also help, (and maybe a good test post before consuming time resources of devs on Github).

We BADLY need a block-level snapshots filesystem in Qubes. One of the beautiful things about Qubes, is using VMs for development environments and testing out software, packages, configurations, etc. Standalones, passthru, HVM, you name it.

But the problem is that, while experimenting, sometimes you’re going to make a mistake, or something is going to fail. Some of those missteps can be fatal to even booting a VM, and either you can spend the next 5 hours of your life troubleshooting and desperately trying to get back to the state you were just in 3 minutes ago … OR … if we had a proper, modern filesystem that supported snapshots like any sane environment should … you would just rollback to the last known good state.

If you think “I dont need that, I just use Qubes for normie stuff” … Well, imagine deleting a file, only to realize that you actually need it. Snapshots allow you to recover files as well.

I would jokingly say, it’s a crime against Unix for a beast as capable and awesome as Qubes to not have a filesystem with block level snapshots.

I have already tried Rudd-O ZFS on Qubes, and I cant make it work. I have asked him about it in Github as well, no response (I’m sure he’s a busy guy). I am aware of these threads:

github.com/QubesOS/qubes-issues

Switch default pool from LVM to BTRFS-Reflink

opened 09:46AM - 22 Mar 21 UTC

DemiMarie

T: enhancement P: default C: storage

**The problem you're addressing (if any)** In R4.0, the default install uses …LVM thin pools. However, LVM appears to be optimized for servers, which results in several shortcomings: - Space exhaustion is handled poorly, requiring manual recovery. This recovery may sometimes fail. - It is not possible to shrink a thin pool. - Thin pools slow down system startup and shutdown. Additionally, LVM thin pools do not support checksums. This can be achieved via dm-integrity, but that does not support TRIM. **Describe the solution you'd like** I propose that R4.3 use BTRFS+reflinks by default. This is a proposal ― it is by no means finalized. **Where is the value to a user, and who might that user be?** BTRFS has checksums by default, and has full support for TRIM. It is also possible to shrink a BTRFS pool without a full backup+restore. BTRFS does not slow down system startup and shutdown, and does not corrupt data if metadata space is exhausted. When combined with LUKS, BTRFS checksumming provides authentication: it is not possible to tamper with the on-disk data (except by rolling back to a previous version) without invalidating the checksum. Therefore, this is a first step towards untrusted storage domains. Furthermore, BTRFS is the default in Fedora 33 and openSUSE. Finally, with BTRFS, VM images are just ordinary disk files, and the storage pool the same as the dom0 filesystem. This means that issues like #6297 are impossible. **Describe alternatives you've considered** None that are currently practical. bcachefs and ZFS are long-term potential alternatives, but the latter would need to be distributed as source and the former is not production-ready yet. **Additional context** I have had to recover manually from LVM thin pool problems (failure to activate, IIRC) on more than one occasion. Additionally, the only supported interface to LVM is the CLI, which is rather clumsy. The LVM pool requires nearly twice the amount of code as the BTRFS pool, for example. **Relevant [documentation](https://www.qubes-os.org/doc/) you've consulted** `man lvm` **Related, [non-duplicate](https://www.qubes-os.org/doc/reporting-bugs/#new-issues-should-not-be-duplicates-of-existing-issues) issues** #5053 #6297 #6184 #3244 (really a kernel bug) #5826 #3230 ― since reflink files are ordinary disk files we could just rename them without needing a copy #3964 everything in https://github.com/QubesOS/qubes-issues/search?q=lvm+thin+pool&state=open&type=issues Most recent benchmarks: https://github.com/QubesOS/qubes-issues/issues/6476#issuecomment-1689640103

renehoj · July 19, 2024, 5:20pm

You can revert the file system if something goes wrong.

bawdyanarchist · July 19, 2024, 5:32pm

That’s a huge help, thank you.

bawdyanarchist · July 19, 2024, 8:16pm

That’s a huge help, thank you. Is this a block level snapshot?

In many cases, it’d be very useful to be able create snapshots inside a VM as well. I’ve tried installing ZFS in Debian VMs, but because you’re layering a ZFS filesystem on top of an already ext4 filesystem, you lose a lot of performance.

Also, do you know of any native encryption solution for VMs? Whether that’s from the perspective of dom0, or from inside the VM. A security focused containerized distro should have the native capability to encrypt those containers as well.

solene · July 19, 2024, 8:22pm

It’s not really a snapshot, more a bunch of derived disks that are not merged within the main volume and could be discarded if you want to restore an older version.

It’s not as flexible as snapshots but provide pretty similar feature