ZFS in Qubes OS

Use at your own risk!

Beware: Dragons might eat your precious data!

Install ZFS in Dom0

Install DKMS style packages for Fedora (defunct in 0.6.2 due to spl/issues/284)

Fetch and install repository for DKMS style packages for your Dom0 Fedora version http://zfsonlinux.org/fedora.html:

disp1# wget http://archive.zfsonlinux.org/fedora/zfs-release-1-1$(rpm -E %dist).noarch.rpm
dom0# qvm-run --pass-io disp1 'cat /home/user/zfs-release-1-1.fc18.noarch.rpm' > /home/user/zfs-release-1-1.fc18.noarch.rpm
dom0# sudo yum localinstall /home/user/zfs-release-1-1.fc18.noarch.rpm
dom0# sudo sed -i 's/$releasever/18/g' /etc/yum.repo.d/zfs.repo
dom0# sudo qubes-dom0-update @development-tools
dom0# sudo qubes-dom0-update zfs

Install DKMS style packages from git-repository

Build and install your DKMS or KMOD packages as described in http://zfsonlinux.org/generic-rpm.html.

Prerequisites steps in AppVM (i.e. disp1)

Checkout repositories for SPL and ZFS:

mkdir ~/repositories && cd ~/repositories
git clone https://github.com/zfsonlinux/spl.git
git clone https://github.com/zfsonlinux/zfs.git

Revert changes in SPL repository due to this bug: https://github.com/zfsonlinux/spl/issues/284

cd ~/repositories/spl
git config --global user.email "user@example.com"
git config --global user.name "user"
git revert e3c4d44886a8564e84aa697477b0e37211d634cd

Installation steps in Dom0

Copy repositories over to Dom0:

mkdir ~/repositories
qvm-run --pass-io disp1 'tar -cf - -C ~/repositories/ {spl,zfs}' | tar -xpf - -C ~/repositories/

Installing build requirements for SPL and ZFS DKMS modules:

sudo qubes-dom0-update dkms kernel-devel zlib-devel libuuid-devel libblkid-devel lsscsi bc autoconf automake binutils bison flex gcc gcc-c++ gdb gettext libtool make pkgconfig redhat-rpm-config rpm-build strace 

Configure and build SPL DKMS packages:

cd ~/repositories/spl
./configure --with-config=user
make rpm-utils rpm-dkms

Configure and build ZFS DKMS packages:

cd ~/repositories/zfs
./configure --with-config=user
make rpm-utils rpm-dkms

Install SPL and ZFS packages (i.e. version 0.6.2):

sudo yum localinstall \
    ~/repositories/spl/spl-0.6.2-1.qbs2.x86_64.rpm \
    ~/repositories/spl/spl-dkms-0.6.2-1.qbs2.noarch.rpm \
    ~/repositories/zfs/zfs-0.6.2-1.qbs2.x86_64.rpm \
    ~/repositories/zfs/zfs-dkms-0.6.2-1.qbs2.noarch.rpm \
    ~/repositories/zfs/zfs-dracut-0.6.2-1.qbs2.x86_64.rpm \

Configure ZFS

Automatically load modules



for module in spl zfs; do
    modprobe ${module} >/dev/null 2>&1

Make this file executable.


Tame the memory-eating dragon (i.e. 512 Mb zfs_arc_max):


options zfs zfs_arc_max=536870912

Setup a zpool with ZFS datasets

You can create a ZFS dataset for each AppVM, ServiceVM, HVM or TemplateVM or just use a pool as your backup location.

Move your existing directory to a temporary location, or the ZFS mount will overlay your directory.

Beware: VMs on a ZFS dataset aren’t working, if your ZFS installation deserts you.

So keep netvm, firewallvm and your templates on your root file-system (preferably on a SSD).

zpool create -m none -o ashift=12 -O atime=off -O compression=lz4 qubes mirror /dev/mapper/<cryptname1> /dev/mapper/<cryptname2>
zfs create -p qubes/appvms
zfs create -m /var/lib/qubes/backup-zfs qubes/backup
zfs create -m /var/lib/qubes/appvms/banking qubes/appvms/banking
zfs create -m /var/lib/qubes/appvms/personal qubes/appvms/personal
zfs create -m /var/lib/qubes/appvms/untrusted qubes/appvms/untrusted
zfs create -m /var/lib/qubes/appvms/work qubes/appvms/work

Have fun with zpool and zfs.

Tips and Hints

Backup your data

You’re depending on an huge amount of code for this file system, keep this in mind and backup your precious data.

Encrypt underlying devices

dom0# cryptsetup -c aes-xts-plain64 luksFormat <device1>
dom0# cryptsetup luksOpen <device1> <cryptname1>

With the use of cryptsetup a keyfile can be specified to decrypt devices.

dom0# head -c 256 /dev/random > /root/keyfile1
dom0# chmod 0400 /root/keyfile1
dom0# cryptsetup luksAddKey <device1> /root/keyfile1

Decrypt devices on boot

Add your devices to /etc/crypttab.

<cryptname1> <device1> <keyfile1>
<cryptname2> <device2> none

Specifying a keyfile is especially useful, if ZFS should be ready during boot.

Further Reading

This document was migrated from the qubes-community project
  • Page archive
  • First commit: 08 Dec 2020. Last commit: 08 Dec 2020.
  • Applicable Qubes OS releases based on commit dates and supported releases: 4.0
  • Original author(s) (GitHub usernames):
  • Original author(s) (forum usernames):
  • Document license: CC BY 4.0
1 Like

I gave obsolete information here that would confuse people

Quite the opposite: R4.2 now has a dedicated “zfs” Qubes OS storage driver, and the very latest ZFS filesystem prerelease version (2.2.0-rc3) added reflink support on Linux too so theoretically it should also be compatible with the “file-reflink” driver on both R4.1 and R4.2.

1 Like

That’s great!

To confirm: “qubes/storage/zfs.py” does the VM creation/cloninig/management using ZFS, but the installer will still not have ZFS modules on it. So you’ll still need a non-zfs root, but once qubes is actually going, you can format the rest ZFS and use that.
Is that correct?

I think so.