I still think that btrfs with bees ( GitHub - Zygo/bees: Best-Effort Extent-Same, a btrfs dedupe agent ) is the direction one should look into if you really need dedupe on Qubes today.
btrfs on Qubes uses the file-reflink driver so the above will likely work.
I can’t speak to whether you might need to modify the dom0 kernel to complete the solution but if you do you might be able to get Qubes development team to take a PR.
B
@brendanhoar Really interesting. Will have to test. Modified OP to reflect this as the best solution for users cloning qubes and specializing templates. (Not sure how to mark a post as solution, but above reply would be the most practical one (if working) @deeplow )
It was in the General Discussion category, which doesn’t have solutions. I’ve moved it now to the User Support and marked that post as the solution.
Dedup in ZFS has gotten much faster and RAM saving. As in /actual/ dedup, like Bees for btrfs but fully online. They replaced the use of the DDT (dedup table) with an FDT structure (fast dedup table) that is way more efficient than the DDT.
Cloning and reflink aren’t dedup. Dedup is when you make a copy of existing data and the fs intelligently detects that then stores only one copy. When you reflink or clone, you are achieving space savings over having two copies, but that happens because no second copy is actually being made at any point.
TLDR commonly cited space saving technologies don’t qualify as dedup.
@Rudd-O I would love ZFS to be fully supported inside of QOS installer, but scattered seems to point to the conclusion that licensing problems are still not completely resolved for that to happen? What is the state of the situation?
Nothing on the Qubes OS side to ship ZFS with the installer.
However the installer I wrote is in fact now functional and fully automatic with Qubes OS 4.2.