Qubes in tmpfs 🤫

Hi @qubesuser1234 :slight_smile:

Next time you get this error, before deleting or altering the files, can you then run and post the output of the following two commands, so we can potentially help you fix…

sudo ls -lsa /usr/lib/dracut/modules.d/01ramboot

sudo sha1sum /usr/lib/dracut/modules.d/01ramboot/*

Thanks for your reply ! So with your first command sudo ls -lsa /usr/lib/dracut/modules.d/01ramboot I got this :

total 20
4 drwxr-xr-x 2 root root 4096 Nov 7 23:02 .
4 drwxr-xr-x 64 root root 4096 Nov 7 23:01 …
4 -rwxr-xr-x 1 yukz yukz 172 Nov 7 22:59 module-setup.sh
4 -rwxr-xr-x 1 yukz yukz 386 Nov 7 22:59 pass.sh
4 -rwxr-xr-x 1 yukz yukz 292 Nov 7 22:59 tmpfs.sh

and with your last command i got this : a7eb5499b7d45c33fa49a268947c4fc9093e5e55 /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
bef4f0747b4a9b43886e48bd7466018e53f1204b /usr/lib/dracut/modules.d/01ramboot/pass.sh
900621278b582f42dedf6354b3530ebef8d8dbb7 /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

I forget to say i’ve reinstalled Qubes 4.1.2 i’m not using anymore 4.2-rc4. Do you think the issue is this ?

@qubesuser1234

The only thing I believe that’s changed here between Qubes R4.1 and R4.2 is in Step #3

If still using Qubes 4.1.X, you’ll also want to change Step #3 to this:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg

However, I do not think that issue caused your specific error that you described when running Step #9sudo dracut --verbose --force”.

First thing I notice is that your script files in the 01ramboot directory are owned by a user account named “yukz” instead of “root”. This is probably caused by not typing the commands exactly and not placing “sudo” in the commands for creating and editing these script files.

You could correct this by either deleting and re-doing the steps exactly with “sudo”, or just run the following command to change ownership to root for the existing script files you’ve already created:

sudo chown root:root /usr/lib/dracut/modules.d/01ramboot/*

Re-running sudo ls -lsa /usr/lib/dracut/modules.d/01ramboot should then show all the script files as owned by “root root” instead of “yukz yukz”.

Another issue I notice is that your sha1sum file hashes are different than mine for all 3 .sh script files. This means that the exact contents put into the script files is different than the instructions precisely demonstrate. Even less or more blank spaces or extra blank lines can change the hashes, so it may or may not have an impact in causing the error, depending upon what exact differences you put into the script files, even if just minor typo mistakes.

You could double check the exact character by character contents of the script files, since your posted hashes mean that something is different in each of the 3 .sh script files from my exact posted instructions.

Then you can try re-running Step #9 again to see if the error is resolved.

sudo dracut --verbose --force

If errors still persist, then re-check those script files until there is absolutely no mistakes or differences from the posted instructions.

I will probably post file hashes soon to help further aide users in confirming that they typed in the exactly correct file contents that matches the instructions character for character.

  • Note to self: We need an easy way to copy these texts or files into Dom0, without precise script file typing needed by users. Anyone know of a good way to do this?

Interested to hear if you get Qubes Stateless working and how you enjoy it! :slight_smile:

Depending on what your circumstances are, packaging them as an RPM package may have merits.

Quoting myself (source):

[…] copying files from less-trusted to more-trusted qubes should be avoided. Because dom0 is the most trusted qube and the most critical qube, copying files to dom0 from a work qube on a regular basis doesn’t seem reasonable to me (your circumstances may be different!)

There are, however, mechanisms by which dom0 can be updated. In particular, the dom0 secure updates mechanism provides better security than copying files between qubes. In order to take advantage of the secure updates mechanism, we need to package our [files] as RPM packages, and use the security features of the RPM workflow to allow dom0 to verify them. This guide explains how to do that.

My personal approach is to make it easier for folks to package themselves the files I provide, so they only have to trust themselves, but you could as well distribute RPM packages if that’s your preference.

That writing and code originated from the following topic on this forum (centered on copying Salt files, but applies to any kind of file):

1 Like

Thanks for help ! i’ve listen what you said and it’s working running “sudo dracut --verbose --force” give me no error now. I wrote the script myself instead of copy and paste !

But… there is a little issue the script don’t work :sweat_smile: my system is booting without telling me “Boot to ram ?”

I don’t know if it’s normal or not but when i edit grub to increase dom0_mem=max:10240M for me i have to do “sudo grub2-mkconfig -o /boot/efi/EFI/qubes/grub.cfg” instead of sudo grub2-mkconfig -o /boot/grub2/grub.cfg sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg

Perhaps i have to delete the folder “/boot/efi/EFI/qubes/grub.cfg” ? I’m wondering if this a good idea…

Thanks @gonzalo-bulnes - Yes, packaging as RPM would likely be the ultimate best, if I get time to learn in the future. Thanks for the resources on that!

Using this command seems to be the right quick and dirty method (source):

qvm-run -p QUBE_NAME 'cat NAME_OF_FILE ' > NAME_IN_DOM0

@qubesuser1234 - I also have a 4.1.X installation, and the grub.cfg is located specifically at “/boot/efi/EFI/grub.cfg”. You could try running the command for all three locations. But I don’t think this step should fundamentally prevent the stateless system from booting either, rather, should just boot with lower maximum Dom0 RAM available.

sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo grub2-mkconfig -o /boot/efi/EFI/qubes/grub.cfg
sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg

It looks like the Qubes Forum software maybe changed the quote characters in my posted script files (module-setup.sh, pass.sh, tmpfs.sh) from straight-quotes (") to curly/angled-quotes (), because I used a different code tag than I probably should have. These malformed quote characters may have been some or all of the source of confusion and non-working file corruption.

All quotes typed into script & config files should strictly be plain straight quote characters (").

I will plan to soon repost with corrected formatting of quote characters.

In the meantime, here are the believed correct file hashes I have:

sha1sum

105190ac916968e49f85911fd3ca2e29909ae327  /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
8ea26fb215217c67c6806b40d760b57f24025047  /usr/lib/dracut/modules.d/01ramboot/pass.sh
f9155e309af3a0a776b3a5b0f8ec2951f1178ff1  /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
1ed767101974b65db6fb30346e96030a78d89567  /etc/dracut.conf.d/ramboot.conf

sha256sum

cb3e802e9604dc9b681c844d6e8d72a02f2850909ede9feb7587e7f3c2f11b8a  /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
a2750fa31c216badf58d71abbc5b92097e8be21da23bbae5779d9830e2fdd144  /usr/lib/dracut/modules.d/01ramboot/pass.sh
37cafbead5124b57a98bb9427e078cecf7d3c04003267d9fc16e34188aaffe93  /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
60d69ee8f27f68a5ff66399f63a10900c0ea9854ea2ff7a77c68b2a422df4bef  /etc/dracut.conf.d/ramboot.conf

sha512sum

8a1551a0d9fdb6fe543ef302a89f085de6c3e4dfb1648795fa73f6096277e5d9e2e108fcb4756549f4ab1544c6950a86c76f05701156b229325e592447972bae  /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
3a890a4055bd8cad1b790ad2092c3f481ed512d7082d491951f2b474f519124c086cc38b8a3055914938cab223e496a1c101cc79d415cb5cf57010d6e6ef2fb6  /usr/lib/dracut/modules.d/01ramboot/pass.sh
401f4f3be222b07c4d36705410e5525c62ec6c025f86eae1eca5dc7538739eff0c845a9aa978dbb2b2985ee9c885911fbbbc721d3f7128314dac8f0e53eef0bb  /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
86d4ff45c86c4cee0ac5e92f8ed52183fa2e8445154f963bfe3cd91ced390a488a926e3627c55b9c6fd598e8105088638c0b90a838a403c4acaefe6519758a05  /etc/dracut.conf.d/ramboot.conf

This seems indeed to be the case. When I edit your post, I see some HTML tags (e.g. <code>, <br>). Did you input these by hand?

If you did, that was a good guess, but clearly the Discourse editor is not quite happy with them, the code is formatted as code, but the usual replacement characters for non-code were applied. (For example, the curly quotes.)

Try using fenced code blocks instead. You can open and close them with three backticks (```) on their own line before and after the block. As a bonus, the newlines will be reapected, so no need for <br> or any other HTML. :slightly_smiling_face:

Got it. Yup, you’re right. For some reason, Discourse editing/formatting buttons aren’t visually showing up in my Tor Browser, so it was partial guessing.

Thanks for the helpful tips @gonzalo-bulnes! :slightly_smiling_face:

Apparently I don’t seem to have an edit capability on past posts :frowning: , so I may have to re-post new step-by-step instructions without fixing the previously posted code that contain error producing curly-quotes in it.

That’s surprising. Any thoughts @deeplow ?

As peoppe use the forum they gain more capabilities. @qstateless was pretty much in Trust Level 2 (missing only giving one like). I’ve upgraded you know.

So you should now be able to edit posts up to 30 days :slight_smile: I hope that helps

2 Likes

@qstateless I think i have to delete the folder and files inside efi/qubes and then try to run “sudo grub2-mkconfig -o /boot/grub2/grub.cfg” and “sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg”

But i’m wondering if it will break my system ? Can someone tell me what will happen if i try to do this ?

Thanks @deeplow and @gonzalo-bulnes! It looks like I can edit that past post now. That helps! :slightly_smiling_face:

I will plan on updating the step-by-step instructions post with proper straight-quote (") characters, as well as some file hash verification steps.

@qubesuser1234 - DO NOT delete your EFI or EFI/qubes folders. There are other Xen files in there, and your system might not boot if you did that. No to deleting folders here.

The matter just comes down to what path your EFI grub.cfg file should be located at. As mentiond…

My EFI grub.cfg in Qubes 4.1.X is specifically located at:

/boot/efi/EFI/grub.cfg

Buy you said that your EFI grub.cfg in your Qubes 4.1.X is specifically located at:

/boot/efi/EFI/qubes/grub.cfg

I wonder if you also have another grub.cfg at /boot/efi/EFI/grub.cfg ?

To check for these grub.cfg files, run the following commands and see if any grub.cfg files are present in one or both of these locations:

sudo ls /boot/efi/EFI
sudo ls /boot/efi/EFI/qubes

I believe the grub2-mkconfig command generates the grub.cfg file and places it at whatever path one specifies in the -o parameter.

I would not think that having an extra grub.cfg file within both of the /boot/efi/EFI and /boot/efi/EFI/qubes directories would interfere with the boot, as the system would probably just only use one of them.

After updating your /etc/default/grub file in Step #2, you could probably generate a grub.cfg in all three locations, like this:

sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo grub2-mkconfig -o /boot/efi/EFI/qubes/grub.cfg
sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg

In my step-by-step instructions, the only thing that’s changed in GRUB is the Max Dom0 memory, which should not prevent stateless booting itself.

Since Dom0 system files are being altered, it is highly advisable that one first performs a full backup all of their qubes and data, in case the worst case scenario happens and your system won’t boot.

Running sudo ls command give me this :

yukz@dom0 ~> sudo ls /boot/efi/EFI
BOOT fedora grub.cfg qubes
yukz@dom0 ~> sudo ls /boot/efi/EFI/qubes
fonts grub.cfg grubx64.efi initramfs-6.1.12-1.qubes.fc32.x86_64.img xen-4.14.6.efi xen.efi

Yes but why my grub.cfg is inside a qubes folder by default ? I do not understand why it’s not like you :sweat_smile: but like i said i think this is the problem your script can’t work properly on my system because of this

FYI: I just updated the step-by-step instructions to correct the malformed quote characters. Will plan to still add some additional verification steps in another update.

So you do have 2 separate grub.cfg files, one in both of those different directories.

Not sure why. Are you sure it was in there by default? Could you have possibly accidentally mistyped a command or file path and caused an extra one to be put into the qubes sub-directory?

xuy’s original instructions from last year, during the Qubes 4.1.X era, used the same path that my own system’s EFI grub.cfg file is in (/boot/efi/EFI/grub.cfg).

A clean install of 4.1.X could tell you what the default location of the grub.cfg files are for certain, if you are determined to find that out.

In Qubes 4.2, the multiple grub.cfg files situation for EFI and legacy boot was unified into one single grub.cfg file location, so it is a bit simpler in Qubes 4.2.

I don’t think this would be the case, as explained in my previous replies.

Have you re-checked the hashes of your script files since I posted the hashes?

Run:

sudo sha1sum /usr/lib/dracut/modules.d/01ramboot/*
sudo sha1sum /etc/dracut.conf.d/ramboot.conf

Should match these hashes:

yukz@dom0 ~> sudo sha1sum /usr/lib/dracut/modules.d/01ramboot/*
105190ac916968e49f85911fd3ca2e29909ae327 /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
8ea26fb215217c67c6806b40d760b57f24025047 /usr/lib/dracut/modules.d/01ramboot/pass.sh
f9155e309af3a0a776b3a5b0f8ec2951f1178ff1 /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
yukz@dom0 ~> sudo sha1sum /etc/dracut.conf.d/ramboot.conf
1ed767101974b65db6fb30346e96030a78d89567 /etc/dracut.conf.d/ramboot.conf

As you can see we have the same hashes but it’s doesn’t work :confused: i will try to reinstall Qubes 4.1.2 and see if it’s working.

Just for be sure i will reinstall the system and give you some feedback i will see if it was there by default

I just updated the Qubes Stateless step-by-step instructions. Enjoy! :slightly_smiling_face:

This update includes the addition of a Disable Dom0 Swap step and file verification sub-steps.

Edit Updates thus far:

#1. 2023-11-11

  • Corrected malformed curly/angled-quotes to now be plain straight-quote characters ("), by removing improper HTML code tags that did not work well with Qubes Forum software (Discourse), which was preventing the stateless boot software from running properly if users had copied the malformed quote characters verbatim.
  • Added Qubes 4.1 GRUB commands.
  • Added user backup warning/recommendation.
  • Minor Edits

#2. 2023-11-12

  • Added Disable Dom0 Swap step
  • Added file verification sub-steps
  • Minor Edits
1 Like

Thanks a lot @qstateless and @xuy . You came up with a big breakthrough for Qubes IMHO. I am a big noob and I am concerned that I might mess up or misunderstand something, so as a good noob I went to a language model and asked to make a script based on the instructions of @qstateless .

#!/bin/bash

# Check if the script is running as root
if [ "$(id -u)" != "0" ]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

# Step 0: Create a Save Qube (optional)
echo "Creating a Save Qube..."
qvm-create -l blue --prop=netvm= save

# Step 1: Disable Dom0 Swap
echo "Disabling Dom0 Swap..."
sudo swapoff -a

# Step 2: Edit GRUB to Increase Dom0's Max RAM
echo "Editing GRUB Configuration..."
sed -i 's/dom0_mem=max:4096M/dom0_mem=max:10240M/' /etc/default/grub

# Step 3: Regenerate with New GRUB Configuration
echo "Updating GRUB Configuration..."
if [ -d /boot/efi/EFI/grub.cfg ]; then
    sudo grub2-mkconfig -o /boot/grub2/grub.cfg
    sudo grub2-mkconfig -o /boot/efi/EFI/grub.cfg
else
    sudo grub2-mkconfig -o /boot/grub2/grub.cfg
fi

# Step 4: Make New Directory for Dracut Automation Module
echo "Creating Dracut Automation Module directory..."
sudo mkdir -p /usr/lib/dracut/modules.d/01ramboot

# Step 5: Create Dracut Script File module-setup.sh
echo "Creating module-setup.sh..."
cat <<EOF > /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
#!/usr/bin/bash
check() {
    return 0
}
depends() {
    return 0
}
install() {
    inst_simple "\$moddir/tmpfs.sh" "/usr/bin/tmpfs"
    inst_hook cleanup 00 "\$moddir/pass.sh"
}
EOF
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/module-setup.sh

# Step 6: Create Dracut Script File pass.sh
echo "Creating pass.sh..."
cat <<EOF > /usr/lib/dracut/modules.d/01ramboot/pass.sh
#!/usr/bin/bash
command -v ask_for_password >/dev/null || . /lib/dracut-crypt-lib.sh
PROMPT="Boot to RAM? (y/n)"
CMD="/usr/bin/tmpfs"
TRY="3"
ask_for_password --cmd "\$CMD" --prompt "\$PROMPT" --tries "\$TRY" --ply-cmd "\$CMD" --ply-prompt "\$PROMPT" --ply-tries "\$TRY" --tty-cmd "\$CMD" --tty-prompt "\$PROMPT" --tty-tries "\$TRY" --tty-echo-off
EOF
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/pass.sh

# Step 7: Create Dracut Script File tmpfs.sh
echo "Creating tmpfs.sh..."
cat <<EOF > /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
#!/usr/bin/bash
read line
case "\${line:-Nn}" in
[Yy]* )
    mkdir /mnt
    umount /sysroot
    mount /dev/mapper/qubes_dom0-root /mnt
    modprobe zram
    echo 10G > /sys/block/zram0/disksize
    /mnt/usr/sbin/mkfs.ext2 /dev/zram0
    mount /dev/zram0 /sysroot
    cp -a /mnt/* /sysroot
    exit 0
    ;;
[Nn]* )
    exit 0
    ;;
esac
EOF
sudo chmod 755 /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh

# Step 8: Make New Dracut Config File ramboot.conf
echo "Creating ramboot.conf..."
cat <<EOF > /etc/dracut.conf.d/ramboot.conf
add_drivers+=" zram "
add_dracutmodules+=" ramboot "
EOF

# Step 9: Regenerate with New Dracut Automation Module
echo "Regenerating Dracut..."
sudo dracut --verbose --force

echo "Setup completed. Please manually shut down and restart your system to test Qubes Stateless."

Is this any good? Would it work just moving it to dom0 and executing it as root?

I have tried to follow all of the instructions as well, I have checked that all of the hashes match and they do. But it is not working for me :frowning:.

The system boots again but without asking if I want to boot in RAM or not.

Major Update: I just updated the Qubes Stateless step-by-step instructions.

CODE ERROR DETECTED AND CORRECTED! :speak_no_evil:

I found the slight code error that is the culprit preventing your Qubes Stateless from running on boot.

There indeed was a code error preventing a successful run within the tmpfs.sh file.

The case logic in my posted tmpfs.sh file ended like this:

[Nn]* )
exit 0
;;
esac

However, it should have ended like this:

[Nn]* )
exit 0
;;
* )
exit 1
;;
esac

It was missing this slight portion of code near the end:

* )
exit 1
;;

This was causing the stateless “Boot to RAM? (y/n)” prompt not to show and be automatically skipped.

Somehow, between my development computer and my test computer, a code regression occurred, where my test computer received the good working code but my development computer was showing the bad non-working code making me misbelieve it was good. :male_detective:

Error is fixed now. Should be fully working. My apologies for the frustration. :slightly_smiling_face:

@qubesuser1234 and @Bob3 and everyone else… Please feel free to retry the Qubes Stateless step-by-step instructions again and confirm that Qubes Stateless is now successfully working for you.



Also, here are the new believed correct file hashes I have (which I matched in the updated instructions):

sha1sum

105190ac916968e49f85911fd3ca2e29909ae327  /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
8ea26fb215217c67c6806b40d760b57f24025047  /usr/lib/dracut/modules.d/01ramboot/pass.sh
9c8309dc808204d4eccee0da55d592a6cc320834  /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
1ed767101974b65db6fb30346e96030a78d89567  /etc/dracut.conf.d/ramboot.conf

sha256sum

cb3e802e9604dc9b681c844d6e8d72a02f2850909ede9feb7587e7f3c2f11b8a  /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
a2750fa31c216badf58d71abbc5b92097e8be21da23bbae5779d9830e2fdd144  /usr/lib/dracut/modules.d/01ramboot/pass.sh
d9e85c06c3478cc0cf65a4e017af1a4f9f9dd4ad87c71375e8d4604399f5217d  /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
60d69ee8f27f68a5ff66399f63a10900c0ea9854ea2ff7a77c68b2a422df4bef  /etc/dracut.conf.d/ramboot.conf

sha512sum

8a1551a0d9fdb6fe543ef302a89f085de6c3e4dfb1648795fa73f6096277e5d9e2e108fcb4756549f4ab1544c6950a86c76f05701156b229325e592447972bae  /usr/lib/dracut/modules.d/01ramboot/module-setup.sh
3a890a4055bd8cad1b790ad2092c3f481ed512d7082d491951f2b474f519124c086cc38b8a3055914938cab223e496a1c101cc79d415cb5cf57010d6e6ef2fb6  /usr/lib/dracut/modules.d/01ramboot/pass.sh
e95af846c48d7cf6e0fd659af5296aa532cbd1fb1d337d05c2a69251429e17b47471d4476ad31fe93f96b17703bf38e1eb20306a3875aef973592103c40f20e8  /usr/lib/dracut/modules.d/01ramboot/tmpfs.sh
86d4ff45c86c4cee0ac5e92f8ed52183fa2e8445154f963bfe3cd91ced390a488a926e3627c55b9c6fd598e8105088638c0b90a838a403c4acaefe6519758a05  /etc/dracut.conf.d/ramboot.conf
2 Likes

Yeah, big thanks to @xuy for researching and delivering the innovation to us. I’ve been using it since last year and it is a game changer. I hope my step-by-step instructions can now make it accessible for more people in the Qubes community.

Couldn’t be sure without testing. A lot of it looks ok at a glance, but not sure.

I may be thinking of developing an automated Qubes Stateless installer script in the upcoming future to make this system even more easy. :wink:

Yes! My fault. Error found and corrected. See my prior post for explanation.

Qubes Stateless step-by-step instructions have been updated and should be fully working now.

Enjoy being stateless! :slightly_smiling_face: