Apparently the version of zfs in our debian-12 repository (2.1.11-1) isn’t new enough to get along with kernel 6.5.x. (2.1.13-1 is needed.) I had to reset the kernel to 6.1.x to get zfs to install.
Any chance of getting 2.1.13-1 made available via a simple apt-install?
You can install newer version of zfs from bookworm-backports.
I gather this is not under Qubes OS’s control?
That sounds a lot harder than changing the kernel. And I suspect given the way I do things, I’m less likely to stop doing that when it’s no longer necessary.
Also, Debian isn’t using 6.5 yet, so if I’m using Debian, perhaps I shouldn’t be either?
Yes, it’s configured in template itself.
It’s not really something hard to do:
Instructions
Add backports repository in /etc/apt/sources.list and then you can install packages from it like this:
apt-get install -t bookworm-backports <package>
If you only want to use stable Debian packages then yes.
Thanks for the info.
Given my automation, I will surely forget to undo that when it’s no longer necessary. And I probably won’t even know when it’s no longer necessary.
The template I install zfs on is a clone of a different template…which is using the (QubesOS) default kernel. So, whenever I generate this it will fail…but I left myself a note that I should change the kernel if it fails. Until such time as the newer version of zfs is available by default. Then things will work with “normal” configuration. The worst I can forget to do is remove the note, but it’s in a place where it will be hard to miss.
Almost four months later and this is STILL an issue.
I want to like zfs, but I am really glad at this point I never made it my basic filesystem.
I’m trying to understand the issue you are having. Do I have it right that you want to install zfs in a Debian-12 qube? If so, what is the issue with using Linux 6.1 in your qube? After all, that is the Linux version that ships with Debian 12 and is well-supported for security updates. Linux 6.1 is what I use in my qubes that use zfs, and so far have had no issues.
My setup looks like this:
The template we want to install zfs in does not have a kernel installed yet (and mine began as a minimal template). In the template run these commands:
apt-get install --no-install-recommends dkms
cat << '__EOF__' > /etc/dkms/zfs.conf
# only build for kernels installed with apt
BUILD_EXCLUSIVE_KERNEL=`/usr/bin/python3 -c 'import apt ; print(*([pkg.shortname[12:] for pkg in apt.Cache() if pkg.is_installed and pkg.shortname.startswith("linux-image-") and "." in pkg.shortname] or ["NOPENOPENOPE"]),sep="|")'`
__EOF__
apt-get install zfs-dkms
apt-get install linux-image-amd64 linux-headers-amd64 grub2 qubes-kernel-vm-support
grub-install /dev/xvda
Then, when you create an appvm based on that template, in the qube settings set to either Mode: HVM, Kernel: (provided by qube) or Mode: PVH, Kernel: pvgrub2-pvh
I certainly can get it to install, with a 6.1 kernel. But the latest kernel is a 6.5 kernel (IIRC) and there are at least three older versions of THAT on my system. And exactly one 6.1 kernel. In other words 6.1 is looking woefully out of date from what I can see. I’m actually wondering if something in the system is eventually going to just delete that kernel off my system automatically after several more updates to 6.5.
What would be better, however, would be if Debian would just distribute zfs 2.1.13 (which will work with 6.5 kernels) in the first place without having to go to a “test” repository to get it. It has been months now; when are they going to do it?
The long and short of it is, I have an automated process to build my VMs and it’s frustrating to have to do something “special” for the templates that use zfs (which is not all of them by any means). Right now, that’s going to an older kernel for them and just them.
How did you install the 6.1 kernel? Was it:
I simply switched the template to use that kernel in settings. (It’s available on my system–and I hope it remains so until Debian starts distributing the new zfs.)
I honestly don’t know how one would make it available if it wasn’t present in the list. Until this happened–months ago, with no end in sight–I’ve never needed to mess with the kernel setting.
Let’s get you familiar with Debian’s release cycle.
Debian 12 is Debian’s current stable release. The idea behind Debian’s stable releases is they’re a collection of package versions that are known to work well together, and will continue to do so reliably and predictably for the lifetime of the release, without needing to change system configuration.
To that end, before a stable release, packages for that release undergo a version “freeze”, which means as a matter of policy they don’t receive any more feature updates. Updates to packages are limited to targeted patches for fixing serious bugs, increasing the reliability and security of the system. These updates don’t increment major version numbers. That means Debian 12 will never receive a more modern zfs, nor kernel, nor most other packages. But to call the packages out-of-date is misleading, because the package versions are right where they are supposed to be from Debian’s perspective. Debian is actually very timely about issuing security updates to its packages.
So to answer your question,
Feature updates happen at major releases, the next one being Debian 13. Debian 13 will have a newer zfs, kernel, etc… It’s reasonable to expect Debian 13 will come out a year from now, but no release date has been set yet. At that time you’ll be able to switch to Debian 13, or to remain on Debian 12, whichever you prefer. Debian 12 will be usable for a long time to come.
From debian.org:
Debian 12.0 was initially released on June 10th, 2023.
The Debian 12 life cycle encompasses five years: the initial three years of full Debian support, until June 10th, 2026, and two years of Long Term Support (LTS), until June 30th, 2028.
The 6.1 kernel itself will also get support by the Linux kernel team until Dec 2026:
https://www.kernel.org/category/releases.html
So kernel 6.1 is not going away anytime soon. 6.1 is the kernel provided by Debian 12; it’s the intended kernel to be used with its other packages, and as I said above, out-of-date isn’t the most accurate way to describe it.
All that said, the most Debian way of solving your problem is indeed to use kernel 6.1.
Sounds like you are using a kernel provided by dom0. Be aware there are broadly two ways to manage kernels, that is, managed by dom0, or by the template/qube itself. More info here: Managing qube kernels | Qubes OS
Debian packages generally expect a coherent system populated by other Debian packages. The pitfall of using a kernel provided externally by dom0 is that it breaks that coherence enough to matter in the 1% of edge cases like the one you found yourself in. The zfs package provided by Debian expects the kernel provided by Debian.
You resolved this by making dom0 provide the kernel version the zfs package expects. That works, but you could also install Debian’s default kernel inside its template, like in the setup example in my earlier post. The latter method maintains the full coherence provided by Debian stable’s repo, as well as keeps the template/qubes configuration more self-contained (no need to leave yourself notes to unbreak it if it breaks again, because it simply will not break without deliberate config changes).
So, if I read you correctly, the upshot is that I should just use the 6.1 kernel for all Debian-12 templates, not just the ones that use zfs.
That’s certainly doable and probably the easiest solution (and it seems to be the appropriate solution). Every template I make (other than Micro$haft windows and one fedora one that I need to develop software I run on dom0) is ultimately based on debian-12-minimal, which I do absolutely nothing to other than keep it updated; if I change its kernel via settings, everything downstream will just have that kernel and I don’t have to intervene when the intermediate template upon which all zfs-using qubes are based, is created.
What I was afraid would happen, just happened.
In spite of the fact that kernel 6.1.xxx is supposed to be used with Debian-12, at least according to @likeafox above, AND that I was actually USING that kernel in all of my debian templates…the latest update to dom0 seems to have deleted it off my system. (Honestly do you think you could have checked to see if it was IN USE before deleting it?)
So now the only way I can build zfs into my templates is to use a testing repository, or reinstall the kernel (and it will probably get deleted now, after every dom0 update, a gratuitous pain in the ass).
I’m very glad I never actually started using ZFS. I could have been completely crippled by this move.
I’ve been unable to find any sign that ANY kernels other than 6.6.36, 42, and 48 are available even with --action=list kernel-qubes-vm.
Since the docs don’t consider the possibility you might want to install an old kernel, the only other repository I know to check is qubes-dom0-unstable and it has nothing in it, apparently.
To be clear what I want to know is what I need to do to put a 6.1 kernel back where it can be selected in the settings gui as the default kernel to use for qubes–given that it apparently doesn’t exist on my system any more. (Presumably dom0 wouldn’t be using that template)
You can still install the 6.1 kernel provided by dom0, but it’s outdated and it won’t be updated anymore (last version is kernel-qubes-vm-6.1.75 from 28-Jan-2024 04:24) so it’s better to use the 6.1 kernel provided by the qube from debian (currently linux-image-6.1.0-25-amd64 (= 6.1.106-3)).
To install the old kernel version in dom0 just specify the version directly:
sudo qubes-dom0-update kernel-qubes-vm-6.1.75
Thanks!
Funny, downloaded 93 MB installed 928M…basically 900% bloat on install!
Still…this situation just seems absurd. This is the kernel that debian-12 is designed to use, yet it’s no longer being updated?
(Oh I did raise the limit on the number of versions that would be retained, so it will be a while before I get burned again.)
6.1. kernel is designed to be used in debian. You’re using kernel provided by dom0 which is Fedora so it’s not designed to use 6.1.
Use the kernel provided by the qube itself to not have these problems.
I don’t understand what you mean by “the kernel provided by the qube itself.”
My debian qubes are all debian-12-minimal (not debian-12-xfce) based, if that makes a difference.