Qubes in tmpfs 🤫

51lieal · November 12, 2022, 2:16pm

haven’t try again, maybe in the next time.

xuy · December 20, 2022, 8:07am

Also, because Qubes manager will not show a in-RAM created VMs after PC reboot.
Therefore, change the default pool parameters to save new VMs as files (instead as LVM volumes) will be more user-friendly. This is an example parameters for opt00 file-pool:

qvm-pool add opt00 file -o dir_path=/opt
qubes-prefs --set default_pool opt00
qubes-prefs --set default_pool_private opt00
qubes-prefs --set default_pool_root opt00
qubes-prefs --set default_pool_volatile opt00

If you need to remove the wrongly created VMs as LVM Volumes you can use lvdisplay, lvremove commands. It’s probably a way to clone (copy) a VM instead of removal. Anyway, it’s best to avoid a compromising names of your virtual machines.

anon92454060 · March 25, 2023, 8:57pm

I have tried this and after step 3 and reboot the system boots just as it did before.

On step 1 do I only need to add rd.break at the end of the line and leave the rest of the line as it is?

xuy · March 26, 2023, 3:16am

Try cat /proc/cmdline to find out booted kernel command line parameters.

Try sudo find /boot -name "*.cfg" to search the grub config location.

stant, you should edit the line as shown in the example in first post

anon92454060 · March 27, 2023, 3:04pm

I have been told 40GB ram is not enough for running qubes in ram.

xuy · March 28, 2023, 2:44pm

The way the “Qubes OS in tmpfs” implies that AppVMs are simply stored on an encrypted disk (SSD). The main trick is the SSD is encrypted with a detached header. Without the detached header file, your disk will be the same as empty disk. The file is stored somewhere else from the SSD.

We can make the task easier and don’t boot Qubes in RAM to make traceless/stateless system:

You can just mount the SSD`s filesystem containing your secret AppVMs files above the default file system containing public AppVMs files. So you will get two versions of the same AppVMs: secret and public.

When you’ve done your secret daily work, you should stop all you secret AppVMs and unmount the disk (SSD filesystem). Then the public (non-secret) AppVMs becomes available on the lower (layer) file system directory. So you’re not leaving any trace of your work outside the encrypted SSD. You just need to hide your SSD`s detached header file.

If human right violators will force you to decrypt the system, you will boot to your Qubes OS, show you AppVMs and say that (secret;) SSD doesn’t contain any data. You can store your detached header file (16MB size) in a cloud file hosting service or somewhere else.

In general terms you will need to:

Encrypt you SSD with a detached header [cryptsetup]
Create a pool to store an AppVMs files (choose a storage directory) [qmv-pool]
Mount the encrypted SSD to the AppVMs files storage directory [mount]
Create an AppVMs in this pool [qvm-create -P]

All needed terminal commands are available on this thread.
This method doesn’t change the OS system requirements. You just need to use non-compromising names for your AppVMs. And disable the swap, I guess.

I must warn you, there may be a slight difficulty in changing the parameters of both (secret and public) machines from Qubes Manager, private storage size, for example.

xuy · May 20, 2023, 3:00pm

For RAM saving, you can use a compressed RAM-based block device - zram kernel module instead of tmpfs. There is approximately 45% compression ratio for root directory (/) :

Demonstration:

user@dom0 ~ % zramctl

NAME       ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo-rle        10G   7G  3.1G  3.2G       8 /

user@dom0 ~ % zramctl -h
...
DATA  uncompressed size of stored data
COMPR  compressed size of stored data
...

Setup actions:

Add + lines, remove - line in our (mentioned earlier) file /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh :

-               mount -t tmpfs -o size=100% none /sysroot
+               modprobe zram
+               echo 10G > /sys/block/zram0/disksize
+               /mnt/usr/sbin/mkfs.ext2 /dev/zram0
+               mount /dev/zram0 /sysroot

Add line to our file /etc/dracut.conf.d/ramboot.conf :

add_drivers+=" zram "

Regenerate the /boot/initramfs… and reboot.

user@dom0 ~ % sudo dracut --verbose --force

qubesuser1234 · October 30, 2023, 12:11am

Hello ! I’ve tried to use the script of @xuy in the last version of Qubes OS 4.2.0-rc4 and nothing appear. When i start my system i don’t have the option “Boot to ram” perhaps it’s because i didn’t setup a disk password in the installation of qubes ?

Do i need to reinstall Qubes OS 4.2.0-rc4 with disk password encryption for the script running correctly ?

qstateless · November 7, 2023, 2:52pm

News… Qubes Stateless (RAM-based Qubes in tmpfs/zram, like Tails OS) is working in Qubes 4.2.

Easy and full Step-by-Step instructions provided here in this post to implement a basic Qubes Stateless mode in 4.1 or 4.2. Super credit to @xuy!

Qubes Stateless: Overview...

Qubes Stateless runs your entire Qubes Dom0 environment as Disposable / Amnesic / Ephemeral / Anti-Forensic in RAM, where, unless you purposefully save things in specific alternative ways, your Qubes environment may reset your changes upon restart. This is similar to how Tails OS works, but with the more powerful security & capabilities of Qubes OS.

At each boot, you have the option to run your Dom0 session in Stateless mode or Persistent (normal) mode. Qubes Stateless runs Dom0 from RAM, and you can optionally create and run other user qubes in RAM using the “varlibqubes” storage pool. Your persistent boot storage drive and other attached persistent storage drives will also be available and writable to you during a Stateless session, including any pre-existing user qubes stored on these drives.

Qubes Stateless is an “advanced hybrid state system” where you have both Stateless and Persistent spaces to work with in the same OS environment, and you can optionally control where individual qube volumes operate from (Stateless storage or Persistent storage). This gives us great power and flexibility for controlling the level of statelessness of our qubes.

For those wondering how much hardware system RAM is required to run Qubes Stateless, I’d say 8GB minimum to work and 16GB for basic comfort. Here are my general feelings on amounts of RAM:

8GB RAM = Minimal Running
16GB RAM = Basic Comfort - A basic Qubes Stateless system can run comfortably on 16GB RAM if number and intensity of AppQubes remains towards lower end.
32GB RAM = Extra Comfort - Only needed if your number or intensity of AppQubes demands more.
64GB RAM = Luxury Comfort - Only needed if your number or intensity of AppQubes demands more.

Below are the exact step-by-step commands & instructions for a basic RAM-based Qubes Stateless, primarily derived from @xuy’s posts…

These steps implement the following aspects & components:

Basic RAM-based Qubes from xuy’s post #1.
Dracut Automation Module from xuy’s post #13.
Disable Dom0 Swap Permanently in /etc/fstab.
Disable Dom0 Swap Temporarily from xuy’s post #18.
ZRAM Compression from xuy’s post #28.
Unified grub.cfg location for both UEFI and legacy boot (#7985) → R4.2 Release Notes.
Minor command & code style adjustments for greater newbie ease.

Qubes Stateless: Step-by-Step…

WARNING: Although these Qubes Stateless instructions are made easy enough for almost any Qubes user to follow, Qubes Stateless does involve the modification of your Dom0 system files. Therefore, as a precaution, it is highly recommended that you first perform a full backup of your qubes and data before proceeding, in case your system accidentally gets corrupted or won't boot. No warranty is provided.

In Dom0 of Qubes OS 4.1 or 4.2, open a command line Dom0 Terminal, and perform the following commands & instructions:

+ Step 0. Make a Save Qube:

Optional (Recommended):

qvm-create -l blue --prop=netvm= save

Further explanation:

To give yourself a convenient option for permanently saving data files while operating in RAM-based Stateless mode, I suggest you create a persistent vault AppQube named “save”, so that you can easily copy/move files that you want to save, to the “save” qube, before your computer shuts down, restarts, crashes, etc, and you lose important files while working in the RAM-based Stateless mode.

If you create a “save” qube while in the normal persistent boot mode, then this “save” qube will be available and permanently retain any files sent to it, on your Qubes storage drive, even before, during, and after you are in the RAM-based Stateless mode.

WARNING: If you don’t understand what you’re doing with an advanced RAM-based stateless configuration of Qubes OS, then it can be very easy to lose data. Ensure that you understand and get familiar with (by testing) the different modes and data saving capabilities of this RAM-based system, and take extra precaution to setup and test reliable methods for saving your important data beyond a stateless session.

+ Step 1. Disable Dom0 Swap:

sudo nano /etc/fstab

Using nano, type a # character in front of the line containing “/dev/mapper/qubes_dom0-swap”, to comment it out and disable it, like this:

#/dev/mapper/qubes_dom0-swap ...

Press Ctrl + O to save fstab file.
Press Ctrl + X to exit nano editor.

That will permanently disable your Dom0 Swap after restart, which is likely best for operating Qubes Dom0 in Stateless mode.

If you still want to frequently use Qubes in Persistent mode and have Dom0 Swap enabled, then you can alternatively skip this permanent disable step, and rather temporarily disable Dom0 Swap when you startup Qubes Stateless by remembering (each and every time) to manually run the following Dom0 command…

sudo swapoff -a

+ Step 2. Edit GRUB to Increase Dom0's Max RAM:

sudo nano /etc/default/grub

Within the GRUB_CMDLINE_XEN_DEFAULT line, make a partial modification of this line, from dom0_mem=max:4096M to now be dom0_mem=max:10240M → Just change the text “4096M” to “10240M” (without quotes).

Press Ctrl + O to save grub file.
Press Ctrl + X to exit nano editor.

Note: If you have something like 32GB of RAM or greater in your system, and you want to run several fully stateless user qubes within Dom0 RAM space, then you may want to consider increasing this “10240M” Dom0 RAM maximum value, in order to allow for more Dom0 RAM space where you can store more user qubes as fully stateless. For example, with 32GB of system RAM, you may choose to allocate “20480M” (20GB) or even greater if you wish.

+ Step 3. Regenerate with New GRUB Configuration:

If you are using Qubes 4.2, you should run this single command:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

If you are using Qubes 4.1, you should instead run these two commands:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg

+ Step 4. Make New Directory for Dracut Automation Module:

sudo mkdir /usr/lib/dracut/modules.d/01ramboot

+ Step 5. Make New Dracut Script File module-setup.sh:

sudo touch /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
sudo nano /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

Edit the newly created module-setup.sh file and type-in exactly the following 11 lines of code:

#!/usr/bin/bash
check() {
return 0
}
depends() {
return 0
}
install() {
inst_simple "$moddir/tmpfs.sh" "/usr/bin/tmpfs"
inst_hook cleanup 00 "$moddir/pass.sh"
}

Press Ctrl + O to save module-setup.sh file.
Press Ctrl + X to exit nano editor.

Perform the following verification steps for your module-setup.sh file:

a. Verify that your module-setup.sh file has the exact correct contents with a file hash checksum…

sudo sha256sum /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

…should show output of:

cb3e802e9604dc9b681c844d6e8d72a02f2850909ede9feb7587e7f3c2f11b8a /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

If this output shows as incorrect, run the previous nano edit commands to correct the file contents and then re-verify this file again. Repeat until correct. Note that only an exact duplication, character by character, of the same file contents from these instructions will produce a successful file hash match, where even extra blank spaces or extra blank lines will cause a changed file hash mismatch. Being very precise counts!

b. Verify that your module-setup.sh file is owned by root…

sudo ls -l /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

…should show output of: “root root” for /usr/lib/dracut/modules.d/01ramboot/module-setup.sh.

If this output shows as incorrect, run the following command and then re-verify this file again…

sudo chown root:root /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

+ Step 6. Make New Dracut Script File pass.sh:

sudo touch /usr/lib/dracut/modules.d/01ramboot/pass.sh
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/pass.sh
sudo nano /usr/lib/dracut/modules.d/01ramboot/pass.sh

Edit the newly created pass.sh file and type-in exactly the following 6 lines of code:

#!/usr/bin/bash
command -v ask_for_password >/dev/null || . /lib/dracut-crypt-lib.sh
PROMPT="Boot to RAM? (y/n)"
CMD="/usr/bin/tmpfs"
TRY="3"
ask_for_password --cmd "$CMD" --prompt "$PROMPT" --tries "$TRY" --ply-cmd "$CMD" --ply-prompt "$PROMPT" --ply-tries "$TRY" --tty-cmd "$CMD" --tty-prompt "$PROMPT" --tty-tries "$TRY" --tty-echo-off

Press Ctrl + O to save pass.sh file.
Press Ctrl + X to exit nano editor.

Perform the following verification steps for your pass.sh file:

a. Verify that your pass.sh file has the exact correct contents with a file hash checksum…

sudo sha256sum /usr/lib/dracut/modules.d/01ramboot/pass.sh

…should show output of:

a2750fa31c216badf58d71abbc5b92097e8be21da23bbae5779d9830e2fdd144 /usr/lib/dracut/modules.d/01ramboot/pass.sh

If this output shows as incorrect, run the previous nano edit commands to correct the file contents and then re-verify this file again. Repeat until correct. Note that only an exact duplication, character by character, of the same file contents from these instructions will produce a successful file hash match, where even extra blank spaces or extra blank lines will cause a changed file hash mismatch. Being very precise counts!

b. Verify that your pass.sh file is owned by root and is executable…

sudo ls -l /usr/lib/dracut/modules.d/01ramboot/pass.sh

…should show output of: “-rwxr-xr-x” and “root root” for /usr/lib/dracut/modules.d/01ramboot/pass.sh.

If this output shows as incorrect, run the following commands and then re-verify this file again…

sudo chown root:root /usr/lib/dracut/modules.d/01ramboot/pass.sh
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/pass.sh

+ Step 7. Make New Dracut Script File tmpfs.sh:

sudo touch /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
sudo nano /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

Edit the newly created tmpfs.sh file and type-in exactly the following 21 lines of code:

#!/usr/bin/bash
read line
case "${line:-Nn}" in
[Yy]* )
mkdir /mnt
umount /sysroot
mount /dev/mapper/qubes_dom0-root /mnt
modprobe zram
echo 10G > /sys/block/zram0/disksize
/mnt/usr/sbin/mkfs.ext2 /dev/zram0
mount /dev/zram0 /sysroot
cp -a /mnt/* /sysroot
exit 0
;;
[Nn]* )
exit 0
;;
* )
exit 1
;;
esac

Press Ctrl + O to save tmpfs.sh file.
Press Ctrl + X to exit nano editor.

Perform the following verification steps for your tmpfs.sh file:

a. Verify that your tmpfs.sh file has the exact correct contents with a file hash checksum…

sudo sha256sum /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

…should show output of:

d9e85c06c3478cc0cf65a4e017af1a4f9f9dd4ad87c71375e8d4604399f5217d /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

If this output shows as incorrect, run the previous nano edit commands to correct the file contents and then re-verify this file again. Repeat until correct. Note that only an exact duplication, character by character, of the same file contents from these instructions will produce a successful file hash match, where even extra blank spaces or extra blank lines will cause a changed file hash mismatch. Being very precise counts!

b. Verify that your tmpfs.sh file is owned by root and is executable…

sudo ls -l /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

…should show output of: “-rwxr-xr-x” and “root root” for /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh.

If this output shows as incorrect, run the following commands and then re-verify this file again…

sudo chown root:root /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

+ Step 8. Make New Dracut Config File ramboot.conf:

sudo touch /etc/dracut.conf.d/ramboot.conf
sudo nano /etc/dracut.conf.d/ramboot.conf

Edit the newly created ramboot.conf file and type-in exactly the following 2 lines of code:

add_drivers+=" zram "
add_dracutmodules+=" ramboot "

Press Ctrl + O to save ramboot.conf file.
Press Ctrl + X to exit nano editor.

Perform the following verification steps for your ramboot.conf file:

a. Verify that your ramboot.conf file has the exact correct contents with a file hash checksum…

sudo sha256sum /etc/dracut.conf.d/ramboot.conf

…should show output of:

60d69ee8f27f68a5ff66399f63a10900c0ea9854ea2ff7a77c68b2a422df4bef /etc/dracut.conf.d/ramboot.conf

If this output shows as incorrect, run the previous nano edit commands to correct the file contents and then re-verify this file again. Repeat until correct. Note that only an exact duplication, character by character, of the same file contents from these instructions will produce a successful file hash match, where even extra blank spaces or extra blank lines will cause a changed file hash mismatch. Being very precise counts!

b. Verify that your ramboot.conf file is owned by root…

sudo ls -l /etc/dracut.conf.d/ramboot.conf

…should show output of: “root root” for /etc/dracut.conf.d/ramboot.conf.

If this output shows as incorrect, run the following command and then re-verify this file again…

sudo chown root:root /etc/dracut.conf.d/ramboot.conf

+ Step 9. Regenerate with New Dracut Automation Module:

sudo dracut --verbose --force

This command may take a couple minutes to complete. When done, the last line will read something like: "dracut: *** Creating initramfs image file '/boot/initramfs-6.1.56-1.qubes.fc37.x86_64.img' done ***".

+ Step 10. Shutdown, Restart, Test Qubes Stateless:

Shutdown & Restart your computer. You are ready to test Qubes Stateless!

After normally decrypting your Qubes boot drive, you should be prompted with:

Boot to RAM? (y/n):

You can enter “y” (without quotes) to boot & use your Qubes Dom0 to Stateless mode in RAM. Your pre-existing persistent TemplateQubes, AppQubes, Data Files, etc, will also be accessible to you while operating in Stateless mode. Note that it can take a few minutes to load Qubes Dom0 into RAM once you enter “y”.
You can enter “n” (without quotes) to boot & use your Qubes Dom0 normally to Persistent mode from your storage drive. Here you can set and save any permanent state changes you’d like for the next time you boot to Stateless mode. For example, use this normal Persistent mode for: installing Qubes OS updates, installing/configuring TemplateQubes, creating new persistent AppQubes, ServiceQubes, etc, where your next boot into Stateless mode can then take advantage of such persistent changes, improvements, updates, etc.

Notes on Using Qubes Stateless:

Booting into Persistent mode is where you can make any permanent changes/settings and updates to your Dom0/TemplateQubes, etc, which will then be available in your next Stateless mode session.

You’ll most likely want to keep your Qubes OS Xen, Dom0, and TemplateQubes up to date by semi-regularly restarting into Persistent mode and installing updates there.

Booting into Stateless mode is where you can operate a stateless Dom0 environment in RAM (which resets its state, upon shutdown, to your last saved Persistent state) and where you also have simultaneous Persistent read/write access to your boot storage drive and secondary storage drives.

When operating in Stateless mode, you can still access and save data to your pre-existing user qubes that were created in Persistent mode or on a secondary storage drive, as well as create and use new user qubes and set their storage volumes (root, private, volatile) to operate fully stateless in Dom0 RAM space using the “varlibqubes” storage pool.

You can create new user qubes while in Stateless mode, however, they may or may not persist and be accessible after shutdown (Dom0 metadata is wiped). In the “Create new qube” GUI tool, you can use the “Advanced” tab to specify a “Storage pool”. The “vm-pool” pool is located on your boot storage drive by default and the “varlibqubes” pool is located in your Dom0 filesystem at /var/lib/qubes as traditional file-backed volumes (runs in Dom0 RAM space during Stateless mode). However, if you choose “varlibqubes”, depending on some various conditions and settings, all data volumes of the qube may not be stored in RAM and require extra special configuration for running fully stateless user qubes.

As a precaution to losing important data, assume all of your new changes and new data may be lost by default after your stateless session ends, unless you send the data to a pre-existing persistently created “save” qube (see Step #0) or otherwise really know what you are doing to save your data effectively.

WARNING: If you don’t understand what you’re doing with an advanced RAM-based stateless configuration of Qubes OS, then it can be very easy to lose data. Ensure that you understand and get familiar with (by testing) the different modes and data saving capabilities of this RAM-based system, and take extra precaution to setup and test reliable methods for saving your important data beyond a Stateless session.

WARNING: For those relying upon Qubes Stateless for anti-forensic use of Qubes OS (which is possible), there are some major caveats to learn about. By default, Qubes Stateless only runs the Dom0 environment as stateless in RAM (take care to disable Dom0 Swap). Other user qubes (such as AppQubes, ServiceQubes, TemplateQubes, etc) might retain some state in their storage volumes if not specifically setup to operate fully stateless in Dom0 RAM space. Other user qubes have multiple storage volumes (root, private, volatile) which can each operate either as Persistent or Stateless depending on various conditions and settings.

Qubes Stateless is an “advanced hybrid state system” where you have both Stateless and Persistent spaces to work with in the same OS environment, and you can optionally control where individual qube volumes operate from (Stateless storage or Persistent storage). The various storage drives, storage pools, and storage volumes involved can be confusing but very relevant to the degree of statelessness you need or desire for each individual qube.

Some useful commands for managing your statelessness:

qubes-prefs

qvm-pool

qvm-volume

qvm-prefs

lsblk

ls -lsa /var/lib/qubes

Qubes Stateless: Conclusion...

A massive THANK YOU to the ultimate legend @xuy who brought Qubes in tmpfs/zram to life for us!

Everyone feel free to enjoy Qubes Stateless and post any feedback, improvements, tweaks, critiques, issues, ideas, etc.

Enjoy being stateless.

Edit Updates:

#1. 2023-11-11

Corrected malformed curly/angled-quotes to now be plain straight-quote characters ("), by removing improper HTML code tags that did not work well with Qubes Forum software (Discourse), which was preventing the stateless boot software from running properly if users had copied the malformed quote characters verbatim.
Added Qubes 4.1 GRUB commands.
Added user backup warning/recommendation.
Minor edits.

#2. 2023-11-12

Added Disable Dom0 Swap step.
Added file verification sub-steps.
Minor edits.

#3. 2023-11-12

Added links to cited resources.
Minor edits.

#4. 2023-11-12

Fixed error in tmpfs.sh that caused stateless boot prompt to be skipped.

#5. 2023-11-12

Corrected number of lines listed for tmpfs.sh file.

#6. 2023-12-08

Added permanent disable of Dom0 Swap.

#7. 2023-12-08

Improved several explanations throughout post.
Minor edits.

qubesuser1234 · November 7, 2023, 6:55pm

Thanks for the update ! But when i try to run “sudo dracut --verbose --force” i have the following error “dracut Cannot install a hook (”/usr/lib/dracut/modules.d/01ramboot/pass.sh") that does not exist.

pass.sh is installed inside the 01ramboot folder. I tried to delete every files and folder and restart again but i have the same error 3 times…

qstateless · November 7, 2023, 9:34pm

Hi @qubesuser1234

Next time you get this error, before deleting or altering the files, can you then run and post the output of the following two commands, so we can potentially help you fix…

sudo ls -lsa /usr/lib/dracut/modules.d/01ramboot

sudo sha1sum /usr/lib/dracut/modules.d/01ramboot/*

qubesuser1234 · November 7, 2023, 10:21pm

Thanks for your reply ! So with your first command sudo ls -lsa /usr/lib/dracut/modules.d/01ramboot I got this :

total 20
4 drwxr-xr-x 2 root root 4096 Nov 7 23:02 .
4 drwxr-xr-x 64 root root 4096 Nov 7 23:01 …
4 -rwxr-xr-x 1 yukz yukz 172 Nov 7 22:59 module-setup.sh
4 -rwxr-xr-x 1 yukz yukz 386 Nov 7 22:59 pass.sh
4 -rwxr-xr-x 1 yukz yukz 292 Nov 7 22:59 tmpfs.sh

and with your last command i got this : a7eb5499b7d45c33fa49a268947c4fc9093e5e55 /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
bef4f0747b4a9b43886e48bd7466018e53f1204b /usr/lib/dracut/modules.d/01ramboot/pass.sh
900621278b582f42dedf6354b3530ebef8d8dbb7 /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

I forget to say i’ve reinstalled Qubes 4.1.2 i’m not using anymore 4.2-rc4. Do you think the issue is this ?

qstateless · November 8, 2023, 2:07am

@qubesuser1234

The only thing I believe that’s changed here between Qubes R4.1 and R4.2 is in Step #3…

If still using Qubes 4.1.X, you’ll also want to change Step #3 to this:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg

However, I do not think that issue caused your specific error that you described when running Step #9 “sudo dracut --verbose --force”.

First thing I notice is that your script files in the 01ramboot directory are owned by a user account named “yukz” instead of “root”. This is probably caused by not typing the commands exactly and not placing “sudo” in the commands for creating and editing these script files.

You could correct this by either deleting and re-doing the steps exactly with “sudo”, or just run the following command to change ownership to root for the existing script files you’ve already created:

sudo chown root:root /usr/lib/dracut/modules.d/01ramboot/*

Re-running sudo ls -lsa /usr/lib/dracut/modules.d/01ramboot should then show all the script files as owned by “root root” instead of “yukz yukz”.

Another issue I notice is that your sha1sum file hashes are different than mine for all 3 .sh script files. This means that the exact contents put into the script files is different than the instructions precisely demonstrate. Even less or more blank spaces or extra blank lines can change the hashes, so it may or may not have an impact in causing the error, depending upon what exact differences you put into the script files, even if just minor typo mistakes.

You could double check the exact character by character contents of the script files, since your posted hashes mean that something is different in each of the 3 .sh script files from my exact posted instructions.

Then you can try re-running Step #9 again to see if the error is resolved.

sudo dracut --verbose --force

If errors still persist, then re-check those script files until there is absolutely no mistakes or differences from the posted instructions.

I will probably post file hashes soon to help further aide users in confirming that they typed in the exactly correct file contents that matches the instructions character for character.

Note to self: We need an easy way to copy these texts or files into Dom0, without precise script file typing needed by users. Anyone know of a good way to do this?

Interested to hear if you get Qubes Stateless working and how you enjoy it!

gonzalo-bulnes · November 8, 2023, 9:02am

Depending on what your circumstances are, packaging them as an RPM package may have merits.

Quoting myself (source):

[…] copying files from less-trusted to more-trusted qubes should be avoided. Because dom0 is the most trusted qube and the most critical qube, copying files to dom0 from a work qube on a regular basis doesn’t seem reasonable to me (your circumstances may be different!)

There are, however, mechanisms by which dom0 can be updated. In particular, the dom0 secure updates mechanism provides better security than copying files between qubes. In order to take advantage of the secure updates mechanism, we need to package our [files] as RPM packages, and use the security features of the RPM workflow to allow dom0 to verify them. This guide explains how to do that.

My personal approach is to make it easier for folks to package themselves the files I provide, so they only have to trust themselves, but you could as well distribute RPM packages if that’s your preference.

That writing and code originated from the following topic on this forum (centered on copying Salt files, but applies to any kind of file):

qubesuser1234 · November 8, 2023, 12:54pm

Thanks for help ! i’ve listen what you said and it’s working running “sudo dracut --verbose --force” give me no error now. I wrote the script myself instead of copy and paste !

But… there is a little issue the script don’t work my system is booting without telling me “Boot to ram ?”

I don’t know if it’s normal or not but when i edit grub to increase dom0_mem=max:10240M for me i have to do “sudo grub2-mkconfig -o /boot/efi/EFI/qubes/grub.cfg” instead of sudo grub2-mkconfig -o /boot/grub2/grub.cfg sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg

Perhaps i have to delete the folder “/boot/efi/EFI/qubes/grub.cfg” ? I’m wondering if this a good idea…

qstateless · November 9, 2023, 10:25pm

Thanks @gonzalo-bulnes - Yes, packaging as RPM would likely be the ultimate best, if I get time to learn in the future. Thanks for the resources on that!

Using this command seems to be the right quick and dirty method (source):

qvm-run -p QUBE_NAME 'cat NAME_OF_FILE ' > NAME_IN_DOM0

@qubesuser1234 - I also have a 4.1.X installation, and the grub.cfg is located specifically at “/boot/efi/EFI/grub.cfg”. You could try running the command for all three locations. But I don’t think this step should fundamentally prevent the stateless system from booting either, rather, should just boot with lower maximum Dom0 RAM available.

sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo grub2-mkconfig -o /boot/efi/EFI/qubes/grub.cfg
sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg

It looks like the Qubes Forum software maybe changed the quote characters in my posted script files (module-setup.sh, pass.sh, tmpfs.sh) from straight-quotes (") to curly/angled-quotes (”), because I used a different code tag than I probably should have. These malformed quote characters may have been some or all of the source of confusion and non-working file corruption.

All quotes typed into script & config files should strictly be plain straight quote characters (").

I will plan to soon repost with corrected formatting of quote characters.

In the meantime, here are the believed correct file hashes I have:

sha1sum

105190ac916968e49f85911fd3ca2e29909ae327  /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
8ea26fb215217c67c6806b40d760b57f24025047  /usr/lib/dracut/modules.d/01ramboot/pass.sh
f9155e309af3a0a776b3a5b0f8ec2951f1178ff1  /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
1ed767101974b65db6fb30346e96030a78d89567  /etc/dracut.conf.d/ramboot.conf

sha256sum

cb3e802e9604dc9b681c844d6e8d72a02f2850909ede9feb7587e7f3c2f11b8a  /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
a2750fa31c216badf58d71abbc5b92097e8be21da23bbae5779d9830e2fdd144  /usr/lib/dracut/modules.d/01ramboot/pass.sh
37cafbead5124b57a98bb9427e078cecf7d3c04003267d9fc16e34188aaffe93  /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
60d69ee8f27f68a5ff66399f63a10900c0ea9854ea2ff7a77c68b2a422df4bef  /etc/dracut.conf.d/ramboot.conf

sha512sum

8a1551a0d9fdb6fe543ef302a89f085de6c3e4dfb1648795fa73f6096277e5d9e2e108fcb4756549f4ab1544c6950a86c76f05701156b229325e592447972bae  /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
3a890a4055bd8cad1b790ad2092c3f481ed512d7082d491951f2b474f519124c086cc38b8a3055914938cab223e496a1c101cc79d415cb5cf57010d6e6ef2fb6  /usr/lib/dracut/modules.d/01ramboot/pass.sh
401f4f3be222b07c4d36705410e5525c62ec6c025f86eae1eca5dc7538739eff0c845a9aa978dbb2b2985ee9c885911fbbbc721d3f7128314dac8f0e53eef0bb  /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
86d4ff45c86c4cee0ac5e92f8ed52183fa2e8445154f963bfe3cd91ced390a488a926e3627c55b9c6fd598e8105088638c0b90a838a403c4acaefe6519758a05  /etc/dracut.conf.d/ramboot.conf

gonzalo-bulnes · November 9, 2023, 10:36pm

This seems indeed to be the case. When I edit your post, I see some HTML tags (e.g. <code>, <br>). Did you input these by hand?

If you did, that was a good guess, but clearly the Discourse editor is not quite happy with them, the code is formatted as code, but the usual replacement characters for non-code were applied. (For example, the curly quotes.)

Try using fenced code blocks instead. You can open and close them with three backticks (```) on their own line before and after the block. As a bonus, the newlines will be reapected, so no need for <br> or any other HTML.

qstateless · November 9, 2023, 10:55pm

Got it. Yup, you’re right. For some reason, Discourse editing/formatting buttons aren’t visually showing up in my Tor Browser, so it was partial guessing.

Thanks for the helpful tips @gonzalo-bulnes!

Apparently I don’t seem to have an edit capability on past posts , so I may have to re-post new step-by-step instructions without fixing the previously posted code that contain error producing curly-quotes in it.

gonzalo-bulnes · November 10, 2023, 12:46am

That’s surprising. Any thoughts @deeplow ?

deeplow · November 10, 2023, 7:20am

As peoppe use the forum they gain more capabilities. @qstateless was pretty much in Trust Level 2 (missing only giving one like). I’ve upgraded you know.

So you should now be able to edit posts up to 30 days I hope that helps