Qubes OS not using kernel version reported by update details

Hi,

I have just updated all qubes. In the Details section of Qubes Updater I saw that the kernel was updated from 5.15.52-1.fc32.qubes.x86_64 to 5.17.something. So, I thought it is a good idea to reboot the whole system after everything has been updated. However, after rebooting uname -rv still show the old kernel version (testing in dom0 and fedora-36).

In fedora-36 qube I notice that rpm -q kernel outputs:

kernel-5.17.9-300.fc36.x86_64
kernel-5.18.13-200.fc36.x86_64
kernel-5.18.16-200.fc36.x86_64

regardless of the fact that in the same qube uname -r shows 5.15.52 as mentioned above. Another strange thing to me is the fact that the update Details did not show 5.18.xx but 5.17.xx kernel.

My questions are:

  1. Is it normal the update process not to update the kernel in the qubes (especially dom0 which AFAIK propagate to other qubes)?

  2. Are there any additional/manual steps required to update the kernel to the updated one? (e.g. some dracut runs or something else)

  3. Or is the not updated kernel version deliberate (a special security measure or for some other reason)?

  4. Why fedora-36 shows kernel several versions none of which are in use?

  5. Why fedora-36 shows a kernel version (5.18) which Qubes Updater details did not show?

  6. What is the right action in this situation?

2 Likes

I’m sorry no one replied before.
Template based qubes get their kernel from one stored in dom0 - those
kernels are updated with qubes-dom0-update, and appear in the kernel
selection list in Qube Settings.
This process is completely separate from any kernel updates that are
installed as part of the standard update procedure in templates, which
then are installed in template based qubes. These kernels are
installed but not used.
If you want to use a standard kernel you set kernel to NONE in Qube
Settings. The qube will now boot and use one of the locally installed
kernels.
You can read more about this here

If the update process has installed a new kernel-qubes-vm package in
dom0
, that kernel should be used in newly booted qubes.
If the update process has installed a new kernel package in dom0, that
kernel should be used on a rebooted dom0.

If the update process in a template installs new kernels, those will
not be used in template based qubes.

No, this should be done automatically.

No

These are kernels installed locally. They will not be used unless you
choose to use them by setting “kernel” to NONE in the settings, so the
qube will use local kernels.

Because this is locally installed.

Read the docs.
The only thing you mention that requires action is to confirm that you
installed a new kernel in dom0 with qubes-dom0-update, and that the new
kernel was not used on reboot. This would be a bug and should be
reported at GitHub.
I have never seen this happen: if you can show this did happen, you should
provide a good deal of evidence including attached update logs, and
details of your system, when submitting a bug report.

I never presume to speak for the Qubes team.
When I comment in the Forum or in the mailing lists I speak for myself.
1 Like

+1

Thank you for replying! I have read the doc you suggest before posting but couldn’t seem to find the details you kindly provided.

I hope you don’t mind these extra clarification questions too:

Is there any disadvantage in setting kernel to NONE in Qube Setting, thus making it a different version from the one in dom0?

In case it is better for some reason to have all qubes running the same kernel version (as dom0), is it good and is there a way to disable kernel updates in domU templates and somehow update template kernels only explicitly when needed? (with the idea not to waste traffic for software that won’t be used anyway)

What kernel is (was) set in Qube Manager - System - Global Settings - Default kernel used by qubes at the time of your question?

I’d say yes. You wouldn’t use kernel specially crafted by Qubes’ devs.

If your dom0 kernel is not the newest available, you likely selected the current one manually some time back.

Unfortunately Qubes OS uses the upstream Fedora /etc/default/grub config which will always re-use the kernel you selected last. IMHO this may even pose a security risk for users who don’t notice that. Anyway a bug is open at qubes-issues.

I’d say yes. You wouldn’t use kernel specially crafted by Qubes’ devs.

Are you saying that Qubes’ devs craft kernels for each distro which can run in a qube (fedora, debian, and the other community templates)?

Also:

  • What does this special crafting include?
  • How does this crafting improve anything (thus making it an advantage to use an older kernel)?
  • Why do these crafted kernels lag behind latest stable kernels? Is that not a disadvantage?
  • What if one installs (creates a template) using a distro which is not available for Qubes? How will that “play” with the dom0 kernel version?

and (quoting myself):

Is there a way to disable kernel updates in domU templates and somehow update template kernels only explicitly when needed? (with the idea not to waste traffic for software that won’t be used anyway)? Would you recommend this?

If your dom0 kernel is not the newest available, you likely selected the current one manually some time back.

It is 5.15.57-1. Not sure why it was 5.15.52 after rebooting right after the update.

Unfortunately Qubes OS uses the upstream Fedora /etc/default/grub config which will always re-use the kernel you selected last.

I have not selected anything manually or explicitly.

IMHO this may even pose a security risk for users who don’t notice that.

Could you elaborate, please?

Anyway a bug is open at qubes-issues.

Would you share a link, please? Thanks.

Sorry I hope you won’t get it wrong. You have to read more and then to ask so much questions for which answers are all over the places. I have no doubt someone will now jump in instead of me.

I would very much like to get it right and I am certainly not willing to waste anyone’s time. I have read the related docs and searched the web for these answers but could not find them. It took me a quite a few hours but I don’t mind reading the right info from the places you mention. Unfortunately, I could not to find them, so it would be great if you share some links.

Qubes provides dom0 kernels that are used in all distros, excepting (I
assume) Whonix . I don’t use Whonix so cant confirm this. The kernels
are not distro specific.

You can see the source in GitHub.
I haven’t checked for a while but I do not think that the kernel-vm packages
have any particular “crafting”, other than being packaged to be loaded
in to a qube.

It allows for easier administration of the system.

If you are desperate you can always try the kernel-latest-qubes-vm
package, or even pull that from the testing repository for the very
latest.

If you have created a new template it will use the same mechanism for
loading kernels as the rest.
If you install a standalone then you set kernel to “None” and load the
standard kernel from that distro.

You can also build a custom kernel and package it in dom0 for use by any
templates. All that is in the docs

This isn’t Qubes specific and you may find mechanisms per distro.
In Debian, you can mark packages as hold:
sudo apt-mark hold <package-name>

In Fedora?
There used to be a plugin to lock dnf updates to a specific version, but
I think that disappeared a few years ago.
You can try dnf mark install kernel-VERSION which should hold that
package version on the system, but that hasn’t always worked for me.
Or define an exclusion like this:
exclude=package PACKAGE_NAME in /etc/dnf/dnf.conf

None of that is Qubes specific, except you must do it in the template.

1 Like

Thank you!

One last question:

Will holding/locking the kernel-VERSION in templates (to avoid wasting traffic) also block proper updating/propagation of the kernel version from dom0 (when it updates) to the templates? Or is the process of kernel version propagation from dom0 to templates completely independent from in-template kernel updates (assuming template kernel is set to “default” and not customized in any way)?

Example of what I mean:

Date 1:

dom0 - 5.15.57
fedora - 5.15.57
debian - 5.15.57
On the same date one locks/holds kernel versions in templates fedora and debian.

Later date 2:

dom0 gets a kernel update to (say) 5.20 and templates get all package updates (except for the previously locked/held packages of kernel) as in a normal update.

What will happen to template kernels then? Will they remain 5.15.57 or will they also update to 5.20?