Zfs incompatible with kernel 6.5

OK so I see in qube settings that one can select “provided by qube” which seems to be what you’re talking about.

There’s no way, apparently, to make that the default choice in global settings (apparently selecting none() there is not the same thing…though I could be wrong about that.)

That’s a minor inconvenience, though since I can just set debian-12-minimal to use the kernel provided by the qube, and as it gets cloned that should persist in the clones.

provided by qube = none
But “provided by qube” means that you need to make all your qubes to use HVM virtualization.
The default virtualization is PVH for qubes without attached PCI devices.
It’s better to install grub2-xen-pvh package in dom0 and set default kernel to pvgrub2-pvh so the qubes with default PVH virtualization will be able to use the kernel installed in them.

1 Like

A very, VERY important detail. I don’t want to use HVM unless I absolutely have to.

But now I have a different problem: Templates using pvgrub2-pvh won’t start.

Did you follow the guide above to install kernel and grub in your templates?
Minimal templates don’t have them installed by default.

1 Like

Yeah, I finally realized that “provided by qube” doesn’t mean anything yet!

Working it now.

I’m basically planning to clone debian-12-minimal to debian-12-minimal-kernel, installing the grub stuff there, then set that qube’s kernel to pvgrub2-pvh, and then cloning THAT to make my templates. And of course doing the requisite install in dom0. (I’m making a README file for the next debian or qubes version.)

OK…so doing all that gives me a “sys-base” template (minimal + kernel + bare minimum utilities such as the one that lets you do passwordless sudo) that is 600MB bigger than before at 2.2GB…but at least I CAN run it to install the software. And it’s still much smaller than debian-12-xfce which is 11 GB. Strictly speaking I should now go regenerate all of my templates since this kernel has been updated.

Although I don’t like the extra steps and extra space, apparently this is the only way to have an up-to-date kernel that’s appropriate to debian 12.

New issue: of course my HVMs (like sys-net) won’t work with pvgrub2-pvh, but rather must go with “provided by qube”. No surprise there when one thinks about it.

However, when I use “provided by qube” networking won’t work, so I have to go back to 6.1 from dom0.

OH and VERY DANGEROUS make sure sys-usb isn’t set up to use pvgrub2-pvh. I locked myself out with that one…took me hours to get back in including mounting in troubleshoot mode to kill my restart-usb cron job.

Is there any limitation to installing ZFS within a minimal Qube and using it there versus within DOM0? That would work for me.

Incidentally, this is the method I had been recommending the entire time, including the config example in my first post.


Limitation isn’t a useful way to think about the difference here. There’s not much overlap, they’re two completely different use cases, really.

ZFS in dom0 will let you manage your Qubes system’s storage using a ZFS-backed pool.

ZFS in a qube will let you use ZFS’s tools from within the qube for myriad kinds of user-level tasks on supplementary storage (like qube-attached USB drive, network storage, etc).

Ideally, since debian templates are supported right out of the box, debian kernels would be offered via the same mechanism as the fedora ones. Instead, in order to use a debian kernel, we have to bloat our “minimal” templates by installing it in the template.

Or am I still missing something?

It sounds like you pretty much have it figured out now.

I’m not sure that isn’t already the case. I assume the dom0 package kernel-61-qubes-vm will give you the kernel Debian uses, or something very closely analogous to it. The only reason I can’t say for sure is because I haven’t tested that package myself. Contrary to what @apparatus said earlier, its current version appears to be 6.1.111.

Template bloat is an issue that affects Qubes very generally, and has generated a lot of discussion on this forum. Hopefully in the future we’ll have more sophisticated template management tools that can address this issue in a more fundamental way, so we never have to cringe when installing things in a template. I get the concern, and I just try my best not to worry about it too much, because in the end bloat doesn’t harm system performance or security very much.

1 Like

You’re right, there seems to be separate packages for older LTS kernels that are still supported (even kernel-515-qubes-vm for kernel 5.15), I’ve missed this and only checked the latest LTS kernel package kernel-qubes-vm versions.