OpenZFS in selected qubes

User Support General
1

Is it possible to use OpenZFS as a file system only in select qubes? How would one do this? Are there any caveats?

2

Yes, itā€™s possible.

What exactly is your aim? Do you want the qubeā€™s root partition on zfs? Just /home? Something else?

1 Like
3

Thanks for your answer! Itā€™s a vault/storage qube, I want to be able to store my files safely and avoid corruption and bitrot.

4

A zfs filesystem on /home then. I havenā€™t done something like that before but if you ask me, this is how Iā€™d approach it.

  1. create a new template, and in it:
  2. install ZFS
  3. ensure you have a compatible kernel for your ZFS version (this is a common sticking-point for Debian guests-- install if needed)
  4. (add pvgrub2-pvh support if needed)
  5. create appvm based on the template
  6. create new partition table on appvmā€™s private volume:
    partition 1: ext4 of 50 MB or so
    partition 2: unformatted, using the remainder of the disk
  7. (in template still:) modify /etc/fstab: change /dev/xvdb to /dev/xvdb1, and remove /rw/home entry
  8. possibly need to alter qubes-mount-dirs.service so that it doesnā€™t try to create/mount filesystems we donā€™t want ?
  9. load up appvm and create a zpool on the unformatted partition
  10. create new zfs filesystem for /home, with the option -o copies=2
  11. set up bind-dirs for /etc/zfs/zpool.cache, so that it exists on xvdb1 (itā€™s unique to the appvm)
  12. pray that bind-dirs happens before zfs-import-cache.service in the startup process, and if not then need to go delay zfs-import-cache.service in the template.

I think that about covers it?

If your only concern is durability of your data, you may want to also consider btrfs or xfs, since those are also checksumming filesystems but have the benefit of being more integrated with linux, and could therefore be easier to set up.

2 Likes
5

I do exactly this, use ZFS in vault qube for avoiding corruption and bit rot.

I take a different approach than what @likeafox laid out. I leave the appvm running whatever it normally uses and pass an HBA card through to the VM for hosting all my ZFS pools. In my case I am using Debian-13 with the OpenZFS packages installed from backports (ZFS 2.4).

In my application, I am hosting torrents on my ZFS array. After much testing and rebuilding, I settled on a 1MB record size, with a separate EXT4 drive for downloading to. The torrents get moved to the ZFS array after completion, avoiding fragmentation. I also use high endurance SATA SSDā€™s as special drives to speed up metadata.

Previous to using the special drives, scanning a folder holding ~90TB of files over NFS took ~15 minutes before the directly listing would appear in the media client. After moving to the SSD special drives, the client can now populate the listing in about 15 seconds.

I also enabled the new fast de-dupe, which gives me block level de-duplication. This dove tailed nicely with the use of a download drive (moving torrents to or from breaks hard links and BRT references) and the de-duplication tables get stored on the special drive. So the de-duplication happens transparently with no slow down.

5 Likes
6

@KB3452 hello! It is very interesting! Maybe you could create a separate topic and write a guide? That would be great for beginners

1 Like
7

My answer is about converting the qubeā€™s filesystem to zfs, which seemed to be what OP wanted to do. If you can get away with only using zfs on external media though, thatā€™d greatly simplify configuration for sure.

Also, hang on a minute, did you say youā€™re running torrents in your vault qube?

1 Like
8

Yes, that was what I was trying to convey, all be it poorly, that your method was for the VMā€™s filesystem.

Yes, I used the default vault AppVM as my torrent box. Well mainly the name. I have replaced most of the underlying bits with a template tailored for ZFS and Bittorrent.

If you are thinking, that is a bit ill advised, hehe, I am not your typical Qubes user. I mainly run Qubes to keep malware at bay and to have the entire box using a VPN, just incase anything leaks.

I also pass a few NICs into my vault, and have multiple VPN tunnels running directly on vault, with a separate BT client bound to each VPN tunnel.

So not a typical setup for a ā€˜vaultā€™ qube. :grinning:

9

Hey, if it works for you, then great. I just want to be clear, for the sake of everyone else who might read this, that the qube name ā€œvaultā€ in your case, is a complete misnomer. There is nothing vault-like about your vault, haha