Qubes in tmpfs 🤫

Skipping the 2 and 3 step, then rebuild new initramfs with this module, then add new grub menu would be good too. So we can have 1 persistent + 1 tmpfs, the persistent aka normal boot would be only use for upgrading dom0, or configuring something that need to be persistent.

I’ve not tested how it would be good for others, since it need a lot of ram, I’ll try to upgrade my laptop to 64gb this week and see if my daily use is fine.

I still searching a good way to boot it as live os, at least we need a kickstart config and livecd-tools.

Have you tried it with less ram or not at all? If so, how much ram and how did it go?

Also, I’m assuming this to be compatible with a detached encrypted boot. Do you have any info?

Yes its compatible, The detached boot is happen in the earlier stage, but my approach is not like in the script above, “it would do same” but not identical.

Actually even with 8gb ram it would run but not much you can do, and for just default install I think 16-32gb would be fine. Need to test first.

1 Like

Btw tmpfs can be swapped to disk.

ramfs cannot.

1 Like

With ramfs there is no dynamic size limit. The ramfs size is severely cut off from RAM at the ramfs starts. So ramfs is not optimal memory costs. With 𝚝𝚖𝚙𝚏𝚜 an empty filesystem size doesn’t take RAM space.

And that is true, tmpfs can be swapped to disk. Therefore the swap desirable to be disabled. If you leave the swap enabled, some 𝚝𝚖𝚙𝚏𝚜 (root fs) data may be saved on swap. This data may contain a names of your AppVMs and other 𝚌𝚘𝚗𝚏𝚒𝚍𝚎𝚗𝚝𝚒𝚊𝚕 𝚒𝚗𝚏𝚘𝚛𝚖𝚊𝚝𝚒𝚘𝚗.

The simplest solution:

sudo swapoff -a

No swap, no problem.

_
Or if swap is necessary for a some reason, you need a separate physical SSD storage for swap.

  • We can encrypt the standard QubesOS swap [LVM Logical] volume /dev/qubes_dom0/swap, but that will be a double encryption, because the /dev/qubes_dom0/swap is already encrypted with the underlying [LVM Physical] qubes_dom0 volume. That’s why we don’t do that.

Instead, we’ll encrypt the swap SSD with detached header, so no one knows it’s a 𝚜𝚠𝚊𝚙 𝚂𝚂𝙳 (moreover, it is an encrypted drive at all, just an unformatted disk with 𝚛𝚊𝚗𝚍𝚘𝚖 𝚍𝚊𝚝𝚊). The swap header file can be removed immediately, as well as the swap drive data is temporary and are not valuable.

1. Prepate a swap drive, let’s say 𝚜𝚍𝚇:

DRIVE=/dev/sdX

2. Fill the 𝚜𝚍𝚇 with uniform layer of random data:

dd if=/dev/urandom of=$DRIVE bs=4096 status=progress

3. Encrypt drive with detached header:

Generage keyfile (instead of a password):

cd /dev/shm
sudo dd bs=512 count=4 if=/dev/urandom of=swapkey.luks iflag=fullblock
sudo chmod 600 swapkey.luks

Format and Open the drive:

yes | sudo  cryptsetup luksFormat $DRIVE --key-file swapkey.luks --header swap-header.luks
sudo cryptsetup luksOpen --header swap-header.luks --key-file swapkey.luks $DRIVE swap

4. Create and mount a new swap:

sudo mkswap /dev/mapper/swap
sudo swapon /dev/mapper/swap

The swap-header.luks is the drive header file.

More detailed, look here:
https://wiki.archlinux.org/title/Dm-crypt/Device_encryption

You don’t if you can spare a couple more gigs

Have tested root to tmpfs yesterday, this is my notes:

  • After rd.break, we need to mount everything manually which I don’t like so I did change the base dracut script instead of using rd.break which would boot same as the normal does.

  • The difference in performance is not too significant (I install template and run it from varlibqubes), but still good since there’s no additional log from now on.

After researching for a few days I got many new thing that I don’t know, for the qubes os live project, I’ve seen joanna post about the disadvantage of using qubes live, and one of the reason is

  1. We had to solve the problem of Qubes too easily triggering Out Of Memory
    condition in Dom0 when running as Live OS.

This last problem has been a result of Qubes using the copy-on-write backing for
the VM’s root filesystems, which is used to implement our cool Template-based
scheme [2]. Normally these are backed by regular files on disk – even though
these files are discardable upon VM reboots, they must be preserved during the
VM’s life span, and they can easily grow to a few tens of MBs per VM, sometimes
even more. Also, each of the VM’s private image, which essentially holds just
the user home directory, typically starts with a few tens of MBs for an “empty
VM”. Now, while these represent rather insignificant numbers on a disk-basked
system, in case of a LiveUSB scenarios all these files must be stored in RAM,
which is always a scare resource on any OS, and especially Qubes.

I will continue my report in next week about the progress.

3 Likes

Did you find time to continue this?

haven’t try again, maybe in the next time.

Also, because Qubes manager will not show a in-RAM created VMs after PC reboot.
Therefore, change the default pool parameters to save new VMs as files (instead as LVM volumes) will be more user-friendly. This is an example parameters for opt00 file-pool:

qvm-pool add opt00 file -o dir_path=/opt
qubes-prefs --set default_pool opt00
qubes-prefs --set default_pool_private opt00
qubes-prefs --set default_pool_root opt00
qubes-prefs --set default_pool_volatile opt00

If you need to remove the wrongly created VMs as LVM Volumes you can use lvdisplay, lvremove commands. It’s probably a way to clone (copy) a VM instead of removal. Anyway, it’s best to avoid a compromising names of your virtual machines.

I have tried this and after step 3 and reboot the system boots just as it did before.

On step 1 do I only need to add rd.break at the end of the line and leave the rest of the line as it is?

Try cat /proc/cmdline to find out booted kernel command line parameters.

Try sudo find /boot -name "*.cfg" to search the grub config location.

stant, you should edit the line as shown in the example in first post

I have been told 40GB ram is not enough for running qubes in ram.

1 Like

The way the “Qubes OS in tmpfs” implies that AppVMs are simply stored on an encrypted disk (SSD). The main trick is the SSD is encrypted with a detached header. Without the detached header file, your disk will be the same as empty disk. The file is stored somewhere else from the SSD.

We can make the task easier and don’t boot Qubes in RAM to make traceless/stateless system:

You can just mount the SSD`s filesystem containing your secret AppVMs files above the default file system containing public AppVMs files. So you will get two versions of the same AppVMs: secret and public.

When you’ve done your secret daily work, you should stop all you secret AppVMs and unmount the disk (SSD filesystem). Then the public (non-secret) AppVMs becomes available on the lower (layer) file system directory. So you’re not leaving any trace of your work outside the encrypted SSD. You just need to hide your SSD`s detached header file.

If human right violators will force you to decrypt the system, you will boot to your Qubes OS, show you AppVMs and say that (secret;) SSD doesn’t contain any data. You can store your detached header file (16MB size) in a cloud file hosting service or somewhere else.

In general terms you will need to:

  • Encrypt you SSD with a detached header [cryptsetup]
  • Create a pool to store an AppVMs files (choose a storage directory) [qmv-pool]
  • Mount the encrypted SSD to the AppVMs files storage directory [mount]
  • Create an AppVMs in this pool [qvm-create -P]

All needed terminal commands are available on this thread.
This method doesn’t change the OS system requirements. You just need to use non-compromising names for your AppVMs. And disable the swap, I guess.

I must warn you, there may be a slight difficulty in changing the parameters of both (secret and public) machines from Qubes Manager, private storage size, for example.

3 Likes

For RAM saving, you can use a compressed RAM-based block device - zram kernel module instead of tmpfs. There is approximately 45% compression ratio for root directory (/) :

Demonstration:

user@dom0 ~ % zramctl

NAME       ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo-rle        10G   7G  3.1G  3.2G       8 /

user@dom0 ~ % zramctl -h
...
DATA  uncompressed size of stored data
COMPR  compressed size of stored data
...

Setup actions:

  1. Add + lines, remove - line in our (mentioned earlier) file /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh :
-               mount -t tmpfs -o size=100% none /sysroot
+               modprobe zram
+               echo 10G > /sys/block/zram0/disksize
+               /mnt/usr/sbin/mkfs.ext2 /dev/zram0
+               mount /dev/zram0 /sysroot
  1. Add line to our file /etc/dracut.conf.d/ramboot.conf :
add_drivers+=" zram "
  1. Regenerate the /boot/initramfs… and reboot.
user@dom0 ~ % sudo dracut --verbose --force
2 Likes

Hello ! I’ve tried to use the script of @xuy in the last version of Qubes OS 4.2.0-rc4 and nothing appear. When i start my system i don’t have the option “Boot to ram” perhaps it’s because i didn’t setup a disk password in the installation of qubes ?

Do i need to reinstall Qubes OS 4.2.0-rc4 with disk password encryption for the script running correctly ?

News… Qubes Stateless (RAM-based Qubes in tmpfs/zram, like Tails OS) is working in Qubes 4.2.

Easy and full Step-by-Step instructions provided here in this post to implement a basic Qubes Stateless mode in 4.1 or 4.2. Super credit to @xuy! :100:


Qubes Stateless: Overview...

Qubes Stateless runs your entire Qubes Dom0 environment as Disposable / Amnesic / Ephemeral / Anti-Forensic in RAM, where, unless you purposefully save things in specific alternative ways, your Qubes environment may reset your changes upon restart. This is similar to how Tails OS works, but with the more powerful security & capabilities of Qubes OS.

At each boot, you have the option to run your Dom0 session in Stateless mode or Persistent (normal) mode. Qubes Stateless runs Dom0 from RAM, and you can optionally create and run other user qubes in RAM using the “varlibqubes” storage pool. Your persistent boot storage drive and other attached persistent storage drives will also be available and writable to you during a Stateless session, including any pre-existing user qubes stored on these drives.

Qubes Stateless is an “advanced hybrid state system” where you have both Stateless and Persistent spaces to work with in the same OS environment, and you can optionally control where individual qube volumes operate from (Stateless storage or Persistent storage). This gives us great power and flexibility for controlling the level of statelessness of our qubes.

For those wondering how much hardware system RAM is required to run Qubes Stateless, I’d say 8GB minimum to work and 16GB for basic comfort. Here are my general feelings on amounts of RAM:

  • 8GB RAM = Minimal Running
  • 16GB RAM = Basic Comfort - A basic Qubes Stateless system can run comfortably on 16GB RAM if number and intensity of AppQubes remains towards lower end.
  • 32GB RAM = Extra Comfort - Only needed if your number or intensity of AppQubes demands more.
  • 64GB RAM = Luxury Comfort - Only needed if your number or intensity of AppQubes demands more.

Below are the exact step-by-step commands & instructions for a basic RAM-based Qubes Stateless, primarily derived from @xuy’s posts…

These steps implement the following aspects & components:




Qubes Stateless: Step-by-Step…


WARNING: Although these Qubes Stateless instructions are made easy enough for almost any Qubes user to follow, Qubes Stateless does involve the modification of your Dom0 system files. Therefore, as a precaution, it is highly recommended that you first perform a full backup of your qubes and data before proceeding, in case your system accidentally gets corrupted or won't boot. No warranty is provided.

In Dom0 of Qubes OS 4.1 or 4.2, open a command line Dom0 Terminal, and perform the following commands & instructions:



+ Step 0. Make a Save Qube:

Optional (Recommended):

qvm-create -l blue --prop=netvm= save

Further explanation:

To give yourself a convenient option for permanently saving data files while operating in RAM-based Stateless mode, I suggest you create a persistent vault AppQube named “save”, so that you can easily copy/move files that you want to save, to the “save” qube, before your computer shuts down, restarts, crashes, etc, and you lose important files while working in the RAM-based Stateless mode.

If you create a “save” qube while in the normal persistent boot mode, then this “save” qube will be available and permanently retain any files sent to it, on your Qubes storage drive, even before, during, and after you are in the RAM-based Stateless mode.

WARNING: If you don’t understand what you’re doing with an advanced RAM-based stateless configuration of Qubes OS, then it can be very easy to lose data. Ensure that you understand and get familiar with (by testing) the different modes and data saving capabilities of this RAM-based system, and take extra precaution to setup and test reliable methods for saving your important data beyond a stateless session.


+ Step 1. Disable Dom0 Swap:
sudo nano /etc/fstab

Using nano, type a # character in front of the line containing “/dev/mapper/qubes_dom0-swap”, to comment it out and disable it, like this:

#/dev/mapper/qubes_dom0-swap ...

Press Ctrl + O to save fstab file.
Press Ctrl + X to exit nano editor.

That will permanently disable your Dom0 Swap after restart, which is likely best for operating Qubes Dom0 in Stateless mode.

If you still want to frequently use Qubes in Persistent mode and have Dom0 Swap enabled, then you can alternatively skip this permanent disable step, and rather temporarily disable Dom0 Swap when you startup Qubes Stateless by remembering (each and every time) to manually run the following Dom0 command…

sudo swapoff -a

+ Step 2. Edit GRUB to Increase Dom0's Max RAM:

sudo nano /etc/default/grub

Within the GRUB_CMDLINE_XEN_DEFAULT line, make a partial modification of this line, from dom0_mem=max:4096M to now be dom0_mem=max:10240M → Just change the text “4096M” to “10240M” (without quotes).

Press Ctrl + O to save grub file.
Press Ctrl + X to exit nano editor.

Note: If you have something like 32GB of RAM or greater in your system, and you want to run several fully stateless user qubes within Dom0 RAM space, then you may want to consider increasing this “10240M” Dom0 RAM maximum value, in order to allow for more Dom0 RAM space where you can store more user qubes as fully stateless. For example, with 32GB of system RAM, you may choose to allocate “20480M” (20GB) or even greater if you wish.


+ Step 3. Regenerate with New GRUB Configuration:

If you are using Qubes 4.2, you should run this single command:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

If you are using Qubes 4.1, you should instead run these two commands:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg


+ Step 4. Make New Directory for Dracut Automation Module:

sudo mkdir /usr/lib/dracut/modules.d/01ramboot


+ Step 5. Make New Dracut Script File module-setup.sh:

sudo touch /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
sudo nano /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

Edit the newly created module-setup.sh file and type-in exactly the following 11 lines of code:

#!/usr/bin/bash
check() {
return 0
}
depends() {
return 0
}
install() {
inst_simple "$moddir/tmpfs.sh" "/usr/bin/tmpfs"
inst_hook cleanup 00 "$moddir/pass.sh"
}

Press Ctrl + O to save module-setup.sh file.
Press Ctrl + X to exit nano editor.

Perform the following verification steps for your module-setup.sh file:

a. Verify that your module-setup.sh file has the exact correct contents with a file hash checksum…

sudo sha256sum /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

…should show output of:

cb3e802e9604dc9b681c844d6e8d72a02f2850909ede9feb7587e7f3c2f11b8a /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

If this output shows as incorrect, run the previous nano edit commands to correct the file contents and then re-verify this file again. Repeat until correct. Note that only an exact duplication, character by character, of the same file contents from these instructions will produce a successful file hash match, where even extra blank spaces or extra blank lines will cause a changed file hash mismatch. Being very precise counts!

b. Verify that your module-setup.sh file is owned by root…

sudo ls -l /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

…should show output of: “root root” for /usr/lib/dracut/modules.d/01ramboot/module-setup.sh.

If this output shows as incorrect, run the following command and then re-verify this file again…

sudo chown root:root /usr/lib/dracut/modules.d/01ramboot/module-setup.sh


+ Step 6. Make New Dracut Script File pass.sh:

sudo touch /usr/lib/dracut/modules.d/01ramboot/pass.sh
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/pass.sh
sudo nano /usr/lib/dracut/modules.d/01ramboot/pass.sh

Edit the newly created pass.sh file and type-in exactly the following 6 lines of code:

#!/usr/bin/bash
command -v ask_for_password >/dev/null || . /lib/dracut-crypt-lib.sh
PROMPT="Boot to RAM? (y/n)"
CMD="/usr/bin/tmpfs"
TRY="3"
ask_for_password --cmd "$CMD" --prompt "$PROMPT" --tries "$TRY" --ply-cmd "$CMD" --ply-prompt "$PROMPT" --ply-tries "$TRY" --tty-cmd "$CMD" --tty-prompt "$PROMPT" --tty-tries "$TRY" --tty-echo-off

Press Ctrl + O to save pass.sh file.
Press Ctrl + X to exit nano editor.

Perform the following verification steps for your pass.sh file:

a. Verify that your pass.sh file has the exact correct contents with a file hash checksum…

sudo sha256sum /usr/lib/dracut/modules.d/01ramboot/pass.sh

…should show output of:

a2750fa31c216badf58d71abbc5b92097e8be21da23bbae5779d9830e2fdd144 /usr/lib/dracut/modules.d/01ramboot/pass.sh

If this output shows as incorrect, run the previous nano edit commands to correct the file contents and then re-verify this file again. Repeat until correct. Note that only an exact duplication, character by character, of the same file contents from these instructions will produce a successful file hash match, where even extra blank spaces or extra blank lines will cause a changed file hash mismatch. Being very precise counts!

b. Verify that your pass.sh file is owned by root and is executable…

sudo ls -l /usr/lib/dracut/modules.d/01ramboot/pass.sh

…should show output of: “-rwxr-xr-x” and “root root” for /usr/lib/dracut/modules.d/01ramboot/pass.sh.

If this output shows as incorrect, run the following commands and then re-verify this file again…

sudo chown root:root /usr/lib/dracut/modules.d/01ramboot/pass.sh
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/pass.sh


+ Step 7. Make New Dracut Script File tmpfs.sh:

sudo touch /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
sudo nano /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

Edit the newly created tmpfs.sh file and type-in exactly the following 21 lines of code:

#!/usr/bin/bash
read line
case "${line:-Nn}" in
[Yy]* )
mkdir /mnt
umount /sysroot
mount /dev/mapper/qubes_dom0-root /mnt
modprobe zram
echo 10G > /sys/block/zram0/disksize
/mnt/usr/sbin/mkfs.ext2 /dev/zram0
mount /dev/zram0 /sysroot
cp -a /mnt/* /sysroot
exit 0
;;
[Nn]* )
exit 0
;;
* )
exit 1
;;
esac

Press Ctrl + O to save tmpfs.sh file.
Press Ctrl + X to exit nano editor.

Perform the following verification steps for your tmpfs.sh file:

a. Verify that your tmpfs.sh file has the exact correct contents with a file hash checksum…

sudo sha256sum /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

…should show output of:

d9e85c06c3478cc0cf65a4e017af1a4f9f9dd4ad87c71375e8d4604399f5217d /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

If this output shows as incorrect, run the previous nano edit commands to correct the file contents and then re-verify this file again. Repeat until correct. Note that only an exact duplication, character by character, of the same file contents from these instructions will produce a successful file hash match, where even extra blank spaces or extra blank lines will cause a changed file hash mismatch. Being very precise counts!

b. Verify that your tmpfs.sh file is owned by root and is executable…

sudo ls -l /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

…should show output of: “-rwxr-xr-x” and “root root” for /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh.

If this output shows as incorrect, run the following commands and then re-verify this file again…

sudo chown root:root /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh


+ Step 8. Make New Dracut Config File ramboot.conf:

sudo touch /etc/dracut.conf.d/ramboot.conf
sudo nano /etc/dracut.conf.d/ramboot.conf

Edit the newly created ramboot.conf file and type-in exactly the following 2 lines of code:

add_drivers+=" zram "
add_dracutmodules+=" ramboot "

Press Ctrl + O to save ramboot.conf file.
Press Ctrl + X to exit nano editor.

Perform the following verification steps for your ramboot.conf file:

a. Verify that your ramboot.conf file has the exact correct contents with a file hash checksum…

sudo sha256sum /etc/dracut.conf.d/ramboot.conf

…should show output of:

60d69ee8f27f68a5ff66399f63a10900c0ea9854ea2ff7a77c68b2a422df4bef /etc/dracut.conf.d/ramboot.conf

If this output shows as incorrect, run the previous nano edit commands to correct the file contents and then re-verify this file again. Repeat until correct. Note that only an exact duplication, character by character, of the same file contents from these instructions will produce a successful file hash match, where even extra blank spaces or extra blank lines will cause a changed file hash mismatch. Being very precise counts!

b. Verify that your ramboot.conf file is owned by root…

sudo ls -l /etc/dracut.conf.d/ramboot.conf

…should show output of: “root root” for /etc/dracut.conf.d/ramboot.conf.

If this output shows as incorrect, run the following command and then re-verify this file again…

sudo chown root:root /etc/dracut.conf.d/ramboot.conf


+ Step 9. Regenerate with New Dracut Automation Module:

sudo dracut --verbose --force

This command may take a couple minutes to complete. When done, the last line will read something like: "dracut: *** Creating initramfs image file '/boot/initramfs-6.1.56-1.qubes.fc37.x86_64.img' done ***".


+ Step 10. Shutdown, Restart, Test Qubes Stateless:

Shutdown & Restart your computer. You are ready to test Qubes Stateless! :smiley:

After normally decrypting your Qubes boot drive, you should be prompted with:

Boot to RAM? (y/n):

  • You can enter “y” (without quotes) to boot & use your Qubes Dom0 to Stateless mode in RAM. Your pre-existing persistent TemplateQubes, AppQubes, Data Files, etc, will also be accessible to you while operating in Stateless mode. Note that it can take a few minutes to load Qubes Dom0 into RAM once you enter “y”.

  • You can enter “n” (without quotes) to boot & use your Qubes Dom0 normally to Persistent mode from your storage drive. Here you can set and save any permanent state changes you’d like for the next time you boot to Stateless mode. For example, use this normal Persistent mode for: installing Qubes OS updates, installing/configuring TemplateQubes, creating new persistent AppQubes, ServiceQubes, etc, where your next boot into Stateless mode can then take advantage of such persistent changes, improvements, updates, etc.


Notes on Using Qubes Stateless:

Booting into Persistent mode is where you can make any permanent changes/settings and updates to your Dom0/TemplateQubes, etc, which will then be available in your next Stateless mode session.

You’ll most likely want to keep your Qubes OS Xen, Dom0, and TemplateQubes up to date by semi-regularly restarting into Persistent mode and installing updates there.

Booting into Stateless mode is where you can operate a stateless Dom0 environment in RAM (which resets its state, upon shutdown, to your last saved Persistent state) and where you also have simultaneous Persistent read/write access to your boot storage drive and secondary storage drives.

When operating in Stateless mode, you can still access and save data to your pre-existing user qubes that were created in Persistent mode or on a secondary storage drive, as well as create and use new user qubes and set their storage volumes (root, private, volatile) to operate fully stateless in Dom0 RAM space using the “varlibqubes” storage pool.

You can create new user qubes while in Stateless mode, however, they may or may not persist and be accessible after shutdown (Dom0 metadata is wiped). In the “Create new qube” GUI tool, you can use the “Advanced” tab to specify a “Storage pool”. The “vm-pool” pool is located on your boot storage drive by default and the “varlibqubes” pool is located in your Dom0 filesystem at /var/lib/qubes as traditional file-backed volumes (runs in Dom0 RAM space during Stateless mode). However, if you choose “varlibqubes”, depending on some various conditions and settings, all data volumes of the qube may not be stored in RAM and require extra special configuration for running fully stateless user qubes.

As a precaution to losing important data, assume all of your new changes and new data may be lost by default after your stateless session ends, unless you send the data to a pre-existing persistently created “save” qube (see Step #0) or otherwise really know what you are doing to save your data effectively.

WARNING: If you don’t understand what you’re doing with an advanced RAM-based stateless configuration of Qubes OS, then it can be very easy to lose data. Ensure that you understand and get familiar with (by testing) the different modes and data saving capabilities of this RAM-based system, and take extra precaution to setup and test reliable methods for saving your important data beyond a Stateless session.

WARNING: For those relying upon Qubes Stateless for anti-forensic use of Qubes OS (which is possible), there are some major caveats to learn about. By default, Qubes Stateless only runs the Dom0 environment as stateless in RAM (take care to disable Dom0 Swap). Other user qubes (such as AppQubes, ServiceQubes, TemplateQubes, etc) might retain some state in their storage volumes if not specifically setup to operate fully stateless in Dom0 RAM space. Other user qubes have multiple storage volumes (root, private, volatile) which can each operate either as Persistent or Stateless depending on various conditions and settings.

Qubes Stateless is an “advanced hybrid state system” where you have both Stateless and Persistent spaces to work with in the same OS environment, and you can optionally control where individual qube volumes operate from (Stateless storage or Persistent storage). The various storage drives, storage pools, and storage volumes involved can be confusing but very relevant to the degree of statelessness you need or desire for each individual qube.

Some useful commands for managing your statelessness:

  • qubes-prefs
  • qvm-pool
  • qvm-volume
  • qvm-prefs
  • lsblk
  • ls -lsa /var/lib/qubes

Qubes Stateless: Conclusion...

A massive THANK YOU to the ultimate legend @xuy who brought Qubes in tmpfs/zram to life for us! :star_struck:

Everyone feel free to enjoy Qubes Stateless and post any feedback, improvements, tweaks, critiques, issues, ideas, etc.

Enjoy being stateless. :slight_smile:




Edit Updates:

#1. 2023-11-11

  • Corrected malformed curly/angled-quotes to now be plain straight-quote characters ("), by removing improper HTML code tags that did not work well with Qubes Forum software (Discourse), which was preventing the stateless boot software from running properly if users had copied the malformed quote characters verbatim.
  • Added Qubes 4.1 GRUB commands.
  • Added user backup warning/recommendation.
  • Minor edits.

#2. 2023-11-12

  • Added Disable Dom0 Swap step.
  • Added file verification sub-steps.
  • Minor edits.

#3. 2023-11-12

  • Added links to cited resources.
  • Minor edits.

#4. 2023-11-12

  • Fixed error in tmpfs.sh that caused stateless boot prompt to be skipped.

#5. 2023-11-12

  • Corrected number of lines listed for tmpfs.sh file.

#6. 2023-12-08

  • Added permanent disable of Dom0 Swap.

#7. 2023-12-08

  • Improved several explanations throughout post.
  • Minor edits.
1 Like

Thanks for the update ! But when i try to run “sudo dracut --verbose --force” i have the following error “dracut Cannot install a hook (”/usr/lib/dracut/modules.d/01ramboot/pass.sh") that does not exist.

pass.sh is installed inside the 01ramboot folder. I tried to delete every files and folder and restart again but i have the same error 3 times…

1 Like

Hi @qubesuser1234 :slight_smile:

Next time you get this error, before deleting or altering the files, can you then run and post the output of the following two commands, so we can potentially help you fix…

sudo ls -lsa /usr/lib/dracut/modules.d/01ramboot

sudo sha1sum /usr/lib/dracut/modules.d/01ramboot/*

Thanks for your reply ! So with your first command sudo ls -lsa /usr/lib/dracut/modules.d/01ramboot I got this :

total 20
4 drwxr-xr-x 2 root root 4096 Nov 7 23:02 .
4 drwxr-xr-x 64 root root 4096 Nov 7 23:01 …
4 -rwxr-xr-x 1 yukz yukz 172 Nov 7 22:59 module-setup.sh
4 -rwxr-xr-x 1 yukz yukz 386 Nov 7 22:59 pass.sh
4 -rwxr-xr-x 1 yukz yukz 292 Nov 7 22:59 tmpfs.sh

and with your last command i got this : a7eb5499b7d45c33fa49a268947c4fc9093e5e55 /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
bef4f0747b4a9b43886e48bd7466018e53f1204b /usr/lib/dracut/modules.d/01ramboot/pass.sh
900621278b582f42dedf6354b3530ebef8d8dbb7 /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

I forget to say i’ve reinstalled Qubes 4.1.2 i’m not using anymore 4.2-rc4. Do you think the issue is this ?