A little bit off-topic but how can I replace the uefi boot art? The original plymouth is ok for me but that uefi art really not my flavor.
Well, dom0 in Qubes essentially is Fedora, so it should be identical 
You’ll need to alter your BIOS for that. Plymouth won’t be able to help you with that, unfortunately.
Electricity goes through your machine → BIOS (MBR/UEFI) → GRUB → Xen → dom0 kernel & initramfs → systemd → plymouth → a bunch of other stuff → X display manager → XFCE greeter
That’s the general boot sequence (paraphrasing for simplification)
1 Like
You probably can, but check the documentation for motherboard and/or firmware.
On ThinkPads, you need to download the latest firmware and place an image file in the update directory called LOGO1.BMP/JPG/PNG and the firmware update tool write the image to flash. Not all types of color encoding and compression are supported, and there are limitations in image size and file size.
@alzer89 is probably correct on this. Much depends on how plymouth is integrated into the qubes os .ISO image. Based on what @renehoj has said, a basic installation feature such as using xorg vs wayland, can differ between distributions.
The plymouth team extended the program to allow incorporating the ACPI table that enables communication to the UEFI for redrawing the firmware boot screen. This allows plymouth to “hijack” the firmware power-on screen.
Whether you could take advantage of this feature likely depends on how plymouth is installed on your system.
Update 2022.10.25 : I’m not certain that the plymouth can use Wayland spec for the display server. All instances of plymouth may use X11/xorg.
Well, you guys are correct, too. I just wanted to clarify to save you guys some time. @Jim and @renehoj, I’m absolutely loving the boot splash screens you guys are coming up with, and I would love to be able to help you get them into Qubes OS as quickly as possible (for those who wish to use them)
Oh I wish that plymouth could do this… 
It’s the other way around. The plymouth team were trying to mimic what a Windows OEM install looks like:
- BIOS loads, with a boot splash (usually a manufactuer’s logo)
- The bootloader then grabs the splash from the BIOS binary, and displays it in the same place as it was in the BIOS
The BIOS technically never stops executing the entire time the machine is powered on. It’s always there, and all the other software (bootloader, kernel, userspace) are “stacked” on top of it.
That’s actually why people call it a software stack 
…but everyone in this thread already knew this. I’m just putting it for anyone else who stumbles across this thread
- The progress radial is shown underneath the boot splash, and the OS boots.
Like this:
If the BIOS doesn’t have a boot splash image that it can understand, Windows will just show the Windows logo:
The current ISOs of Fedora and Ubuntu do this if the machine has a boot splash in the BIOS.
Qubes OS technically could do this too, if you wanted to configure it that way:
Well, there’s only really one place you can install plymouth (for now) 
No software can change, override, or kill any software further down its stack (unless the software further down the stack allows it to, of course).
This is why Linux kernel panics actually freeze/hang the machine. They wouldn’t be so serious if systemd could just restart the kernel 
So, because of where plymouth is in the boot stack, it cannot change anything the BIOS has already done, because the BIOS executes before plymouth.
So your options to get a custom bootsplash are:
OPTION 1: Compile a custom BIOS with your boot splash in it and flash it to your motherboard
You actually replace the boot splash image in the actual BIOS with an image/animation of your choice (if you’re lucky enough to have a machine that the Coreboot team have figured out the inner workings of ).
ADVANTAGES: This will mean that you get the same boot splash image from the moment you power the machine on.
DISADVANTAGES: You will brick your motherboard if you don’t know what you’re doing
Big companies actually do this on their work machines. They’ll replace the OEM logo with their company logo in the BIOS, which is actually a pretty baller thing to do 
OPTION 2: Configure plymouthd to show a custom splash image as soon as it loads
Basically, this allows you to show a custom boot splash image/animation as soon as plymouthd is started, which is usually immediately after the init system (OpenRC, sysvinit, systemd, etc.) is started.
This is currently what the stock Qubes OS install does.
ADVANTAGES: It means you don’t have to mess with the BIOS binary on the chips soldered to the motherboard.
DISADVANTAGES: It means that you’ll see TWO boot splash images. The one in the BIOS, and the custom one you told plymouthd to show…
I’m trying to rework the boot animation using the same method with a sprite and the refresh function.
Using the two-step plugin works on my desktop PC, the animation is smooth with no distortion both during boot and shutdown. It’s only on my laptop that can’t render it perfectly.
I wasn’t able to change the luks password prompt text, and I don’t think you can add a script file when using two-step. It would also be better not having to install any plugin at all, and just use plymouth as shipped with the standard installation.
It’s possible to rotate the sprite, which could be a solution for systems that don’t have enough resources to do frame based animation. I was thinking about making a nice qube vortex images, that could be used as a single image rotating sprite.
1 Like
That’s very interesting. In terms of the “plugins”, there’s no real way around that, they’re written in C and an integral part of the plymouth source files. Charles Brej added the plymouth configuration file feature (theme_name.plymouth) so you could set call-value combinations for the plugin and avoid hard core scripting. For fedora, these types of files are located in the /usr/share/plymouth/themes directory (I’m not sure where in qubes dom0 / adminVM).
Are using the “script” plugin to write your image-sprite animation? Along with the two-step plugin, the script plubin is one of the most popular plugins used. The script plugin may already be installed on your system if you have the “breeze” theme installed.
The script plugin documentation is currently under construction, and is very rough. Currently, it consolidating published articles and creating tables for the raw functions from source files.
However, you may be able to find useful information for your efforts in the Create Plymouth Themes section. The specific link for the Charles Brej 4 part tutorial is here. (Note: if the page doesn’t load, hit refresh).
I’m looking forward to seeing your animation.
2 Likes
great information in your post. nice pictures for clarity.
To clarify what I was saying, plymouth currently has the capability to “hijack” the uefi and to use the associated buffer. However, that capability may not be available in qubes due to the architecure design that incorporates a hypervisor and two separate instances of fedora running in virtual machines (AdminVM/Dom0 & GuiVM).
To go further down the rabbithole for clarification, plymouth is designed to work with Direct Rendor Management (DRM) Kernel Mode Setting (KMS) drivers. The basic strategy is to use the KMS drivers to set up an optimal native display “mode” that includes virtual terminals, buffers and allows a user space client direct access to graphics hardware. All this occurs prior to the root file system being mounted. Details:
Optimal Native Display “Mode”
The optimal display mode is enabled using kernel command line parameters that communicate with the KMS drivers, configure the initial RAM disk and enable user space features. The specific parameters are set by developers so they function with a specific linux distro. On my system, some kernel command line parameters are set in the bootloader files (e.g., Grub 2).
The Display Server
After the optimal native display “mode” is established, control of the virtual terminals and buffer content is handed over to plymouth. Plymouth, in turn, runs a display server that ultimately leads to what is displayed on the screen.
Of note, is that a given linux distro might use a specific display server spec (Wayland, X11) that operates in a different way from other servers. For example, Wayland specced display servers combine the display server and compositor and provide buffers for each client. X11 specced display servers, in contrast, include both a display server and separate compositor. In addition, the entire frame buffer is rendered by the compositor.
To make the architicture more complex, developers could use XWayland that allows an x11 specced display server within a Wayland client. I’m running fedora 36 on a Lenovo R400 and from what I can tell, it uses the XWayland option.
The Unique Qubes Architecture
Plymouth was initially designed around running a single “real” instance of Red Hat and fedora, though later adjusted for other linux distros running a single “real” instance.
Given that the qubes architecure runs a “real hypervisor,” along with two instances of fedora running in virtual machines (AdminVM, GuiVM), the developers task of configuring the use of the KMS drivers, kernel command line parameters and type/manner of the display server seems like a challenging task.
This challening architecture raises interesting questions for how exactly plymouth operates within the AdminVM and the extent to which plymouth interacts with the Xen Hypervisor and GuiVM, the processes they run and the resources they use (cpu, buffers etc.). It was my impression that these were the sorts of issues that @renehoj was referring to.
Update 2022.10.25 : I’m not certain that the plymouth can use Wayland spec for the display server. All instances of plymouth may use X11/xorg.
1 Like
That’s great information. The info on the Admin API and qrexec were helpful in understanding the relationship between the AdminVM, admin processes and other VMs.
1 Like
I’ve remade the theme only using the script module, it should be usable with the standard qube installation.
On my laptop it has the same issue as the two-step plugin, but it works just fine on my desktop PC
Fantastic clarifications as well. I think that once this develops enough, we might actually have enough for a few pages in the official documentation!
That one I’m not 100% sure about, but what I do know is that on some of my testing machines that run Qubes OS, VMs have been able to read (but not write to) the bare metal BIOS.
Pure speculation, but maybe it depends on the hardware configuration?
I will play around with some of my HVMs to see if I can get TianoCore to read directly from the BIOS and get the OEM splash…
That’s the only way to fly
Correct, and Qubes runs multiple instances of DRM and KMS.
That sounds a lot like what a stock Qubes OS dom0 already has in the majority of cases.
This would likely depend on hardware configuration, I would imagine.
Some GPUs (and other PCI devices, for that matter) register themselves as multiple PCI devices to the kernel, and they don’t particularly like to be separated and compartmentalised. AMD integrated GPUs are a good example of this, because they don’t like to be separated from the other devices that “cryptographically verify the integrity of the hardware in some way”, essentially causing a resume from S3 sleep to “lock up” the entire GPU 
But in any case, there’s only one way to find out. Please let me know if you need access to any testing machine (actual hardware, not virtualised) via KVM over IP. I’d love to help you guys out in any way I can.
So then, I’m guessing this would all be able to be taken care of by the dom0 initramfs, right?
It would be interesting to see what hardware the Xen hypervisor doesn’t actually give to dom0. Xen definitely does keep some hardware for itself, but whether that would impact whether plymouth can get read access to that buffer, well there’s only one way to find out
On some of my Qubes OS machines (basically anything made by GPD, because they used tablet display panels that are hardwired in portrait orientation in their hardware), the framebuffer needs to rotated 90° at boot, and GRUB doesn’t seem to have any difficulties doing that.
Link to kernelopts:
1 Like
cool animation. the issues on the laptop could be as you said, a resource issue. I imagine your desktop is more powerful, so it may be able to compensate.
If it is a resource issue, the solution may be efficiency, such as reducing file size. If you’d like to test smaller files, I reduced your .pngs by about 80%. (link here). The quality isn’t real good, but would test the idea that smaller file sizes improve performance.
Another means of gaining efficiency would be to ensure that there is no bloat in the grub config files or initramfs in dom0. If you want to crazy about it, I just read some documentation that suggests that using a “generic” dom0 kernel can create bloat.
Oh, and I did just read that there have been issues with intel integrated graphics with VT-d, should your laptop not have discrete graphics.
cheers.
That’s good to know. From the comments in this thread, and from the documentation I’ve read, plymouth operating in dom0 is very similar to how plymouth operates on non-qubes systems. Of course, plymouth running in fedora in dom0 is operating on a virtual machine, with the optimal display environment (e.g., DRM KMS drvers use) being established by a bare metal bios, and likely bare metal bootloader.
For qubes, plymouth is likely taking a performance hit in that its operation uses both bare metal and virtual elements, in addition to the added complexity of Xen. For example, there have been issues with integrated graphics processing and VT-D (see post above for link). I think it’s possible that these factors are leading to the difficulties in gettting smooth plymouth animations with Qubes.
You raise many interesting points relating to the architecture and boot process (initramfs), so I’ll get back to you on those.
1 Like
I found the SetBootProgressFunction, it tells you the time from the boot started, which can be used to add a delay to the animation.
Slowing down the animation reduces the distortion, it doesn’t completely remove it, but it makes it does make it a lot better.
I also added a text based progress indicator, that show the plymouth progress as a percentage.
Nice solution that supports availability of resources as being a central issue.
The installation instructions at GitHub are interesting, as they are similar to running straight up fedora (vs Qubes with a VM). The source files, theme configuration files and even the theme directories are also similar. That’s good to know.
some questions, if you have the time:
-
Native Script Functions: where are you finding the “script functions” (e.g., SetBootProgressFunction)? Are you getting those from the plymouth function libraries (e.g., script-lib-plymouth.script)?
-
"animation.delay = 0.05;": Do you know the frame per second rate (fps) based on the 0.05 value? If the value is seconds, it looks like perhaps 20 fps?
-
Optimizing .PNG Files: The animation-xx.png files look nicely optimized with a small file size. How were you able to do that?
That’s my understanding. I’m more enthusiast than developer, but my understanding is that plymouth starts running during the initramfs process (plymouth daemon is called, files for the X display server are loaded etc.).
The only documentation I’ve found references initrd, that to my understanding is different from initramfs, though that was for fedora 11. I’m running fedora 36 and it uses initramfs.
That’s a really good question. There could indeed be some type of wrinkle to the qubes architecture (Xen hypervisor) that distrupts the intended use of buffers, though delays, restricting access etc., or access to hardware.
Thx. I’ll likely take you up on that offer in the future. Do you have a T430 in your line up?
Dom0 is running Fedora, using plymouth in dom0 is the same as any other Fedora.
There is some basic documentation it lists the different functions you can hook
The delay does not give you perfect fps, you don’t really know when the function will be called again, but it works to slow done the animation.
I don’t know how the images a compressed, I found them online.
1 Like
I’d say you’re more of an “enthusiastic developer” 
Ok, here goes:
*deep breath in*
Think of your computer as an office.
Filing Cabinets/Storage Cupboards/Shelves (Hard Drive)
Summary
- You keep your documents/stationery/work tools here when you’re not using them
- You might lock this up (LUKS encryption)
- You can’t do anything useful with a file while it’s here. You have to move (copy) it to your desk (RAM) to be able to work with it.
- Sure, you could stand at the filing cabinet and try to work on files, but it’s slow and uncomfortable, and you might make lots of mistakes, so you take everything out and bring it all to your work desk.
- In computers, when you edit a file, the entire file is copied into RAM, and the changes stay in the RAM while the document is open. The changes are written to disk when you close the document (or tell the program to actually save your changes). This is because RAM is exponentially faster and more durable than block storage, and you don’t want programs to be unnecessarily slow and non-responsive
Work Desk (RAM)
Summary
- When you want to do any work, you have to take out any documents/tools (software/binaries) out of their storage and place them on your desk
- The number of things you can do simultaneously is limited only by how big your desk space is (just like your RAM size).
- Bigger desk = more stuff can be out of storage at the same time.
- You can’t stack objects on top of each other on your desk. So if you run out of desk space, you need to put stuff away before you can bring out more stuff. (Just like RAM)
Employees (CPU)
Summary
- Does all the calculations. Pretty self-explanatory.
Office Corridors (PCI Lanes)
Summary
- Wider corridors means more people/stuff can move through them at the same time (PCI lane bandwidth)
Why are you telling me this…?
Summary
You know how the Linux kernel is monolithic, right? The kernel contains all drivers. The kernel is a single binary (usually called vmlinux or vmlinuz if it’s compressed) containing everything needed to boot a computer and do stuff.
The bootloader (in the case of a vanilla Qubes OS install, GRUB) will load the entire kernel into RAM, and it stays in RAM for the entire time that your machine is powered on.
The problem with a monolithic kernel
Summary
- A fully-loaded binary kernel (ie one with everything compiled into
vmlinux) is ridiculously large (somewhere around 1.5GB-2.5GB). Imagine losing 2.5GB of your RAM simply by powering it on!
Kernel modules are the answer!
Summary
You probably don’t need the parts of the kernel that allow you to use printers/joysticks/gamepads/peripherals when they’re not plugged in, right?
So, kernel modules allow the Linux kernel code base to be split into two parts:
- Stuff you need to run the entire time your machine is powered on
- Stuff for hardware that’s on your motherboard (CPU microcode, etc.)
- Stuff you only need running when the hardware is present
- Most USB device drivers
- Peripherals
- Anything that won’t cause your machine to implode if you stop running it
This means:
vmlinuxis a lot smaller- Loads faster
- Takes up far less RAM
vmlinuxcan be stored on a separate partition- You can encrypt your other files
- You can store the modules separately from the kernel
- Modules can be added/removed/edited without causing the kernel to panic
- All “mission-critical” kernel code is in
vmlinux, and never stops running
- All “mission-critical” kernel code is in
But what if some important stuff isn’t in vmlinux, how does it load?
That’s what an initrd/initramfs does. For all intents and purposes, they perform the same function, but with different methodologies.
Summary
The initial ramdisk:
- Contains any kernel modules you want to be loaded at boot time with the kernel
- LUKS decryption
- GPU drivers
- Drivers to be able to interact with your root filesystem drive (the drive where your OS is).
plymouth, if you want a boot splash
If you’ve ever tried to boot from an SD card, you’ll know that horrible feeling of watching your system unable to find the rootfs and hanging because you forgot to include the kernel modules 
So if you want anything to load at the time you load the kernel, you have to put it in the initial ramdisk. This includes your custom plymouth themes.
Why wouldn’t I just put it straight into vmlinux? What’s the difference?
- You can only put kernel code into
vmlinux, not any firmware, config files, customplymouththemes, other binaries, etc. - If you put it in the initial ramdisk, you can unload it after you boot the machine, freeing up your RAM.
Most of them are “rescued” machines. Ones that have been retired from corporate use or thrown away, and I’ve repaired them myself. Sometimes you get lucky and find someone selling a fully-functional machine that they erroneously believe “doesn’t work”, and you get a good machine for an insanely cheap price.
One of them actually hosts a Qubes OS mirror.
I have several T450, and I have T430 and X230 on order. I’m always looking for extra machines. The goal is to allow anyone working on Qubes OS access to a variety of actual hardware to test their stuff on 
Cool. Sounds like you’ve got a nice collection of Lenovo’s.
just to follow up, I’ve got an email out to a plymouth developer on “known issues” with Qubes. They haven’t specifically mentioned it, but I think they’re insanely busy with the Fedora rollout, so it may take a while to hear back.
What are the known plymouth issue with Qubes?