Qubes in tmpfs 🤫

[qstateless]

To those interested in fully stateless (anti-forensic) use of Qubes Stateless, this is an important update.

@Bob3 et al…

I did some further digging into and testing of Qubes Stateless operations.

It seems there are some lingering issues & caveats with achieving fully stateless (anti-forensic) use of Qubes Stateless.

1. Dom0 Still Swaps to Storage Drive

More info here on Wikipedia for those not familiar with what swap is.

It appears that Dom0 Swap is still enabled with each system restart, and that @xuy’s recommended command sudo swapoff -a in post #18 for turning off Dom0 Swap is only temporary for any given Persistent mode session or Stateless mode session, but doesn’t last beyond restart (or maybe even a logoff?).

@xuy

And that is true, tmpfs can be swapped to disk. Therefore the swap desirable to be disabled. If you leave the swap enabled, some 𝚝𝚖𝚙𝚏𝚜 (root fs) data may be saved on swap. This data may contain a names of your AppVMs and other 𝚌𝚘𝚗𝚏𝚒𝚍𝚎𝚗𝚝𝚒𝚊𝚕 𝚒𝚗𝚏𝚘𝚛𝚖𝚊𝚝𝚒𝚘𝚗.

The simplest solution:

sudo swapoff -a

No swap, no problem.

We need to come up with a persistent way to disable Dom0 Swap. I haven’t researched this yet.

In the meantime, you should run the temporary command in the Dom0 Terminal at the beginning of every new Qubes Stateless session…

sudo swapoff -a

2. AppQubes Typically Created in ‘varlibqubes’ Still Partially Using Storage Drive

While testing in Qubes 4.2-RC4, I discovered that creating an AppQube that is based on a TemplateQube and setting the storage pool to ‘varlibqubes’ stores everything in Dom0’s /var/lib/qubes directory, EXCEPT for one thing I’ve found…

Running in Dom0 Terminal:

lsblk

lsblk | grep YOURQUBENAME

…reveals that your AppQube is still storing its ‘root’ volumes on your storage drive.

Here, you can clearly see volumes named ‘qubes_dom0-vm--YOURQUBENAME--root--snap’.

These ‘root’ volume can write sensitive data during your Qubes Stateless session to your persistent storage drive, which could potentially be recovered from the storage drive.

For a fully stateless workaround with AppQubes, see the approaches in “4. Workarounds for Fully Stateless AppQubes & DisposableQubes”.

3. DisposableQubes in Stateless Mode are Typically Not Fully Stateless

You should be aware that typical default configurations of DisposableQubes while in Stateless mode are not fully stateless and store their data on your storage drive.

For a fully stateless workaround with DisposableQubes, see specific approah “c” in “4. Workarounds for Fully Stateless AppQubes & DisposableQubes”.

4. Workarounds for Fully Stateless AppQubes & DisposableQubes

Thankfully, there seems to be a fully stateless workaround for the issues I described in “2. AppQubes Typically Created in ‘varlibqubes’ Still Partially Using Storage Drive” and “3. DisposableQubes in Stateless Mode are Typically Not Fully Stateless”, although quite costly in some RAM space…

You can take a few different approaches to resolve this fully stateless issue now:

• a. Instead of using AppQubes based on TemplateQubes, you could alternatively create StandaloneQubes in the ‘varlibqubes’ storage pool, which appear to store ALL data in traditional image files within Dom0’s ‘/var/lib/qubes’ directory. This is very costly in Dom0 RAM, as it copies your entire TemplateQube’s OS into Dom0 RAM space for each StandaloneQube you make, whether it is actively running or not, which is usually multiple extra GBs per qube, in addition to the RAM it takes to store any user files and the RAM it takes to run and operate the qube’s OS & apps.
  
  
• b. You could create a new TemplateQube from the previous TemplateQube you want to use, but store that new TemplateQube in the ‘varlibqubes’ storage pool. Then create a new AppQubes based on this new TemplateQube, and store this new AppQube in the ‘varlibqubes’ storage pool too. Now, when you use this new AppQube, it appears to store ALL data in traditional image files within Dom0’s ‘/var/lib/qubes’ directory. This is as costly as the other method for the first AppQube, but you do not have to copy & store the entire TemplateQube OS root filesystem for every AppQube you want to make with it, so this saves a lot of RAM space for using more than one qube.
  
  
• c. Like “b”, you could create a new TemplateQube from the previous TemplateQube you want to use, but store that new TemplateQube in the ‘varlibqubes’ storage pool. Then create a Disposable Template by creating new AppQubes based on this new TemplateQube, and store this new AppQube in the ‘varlibqubes’ storage pool too. After creation, in the settings of this AppQube, under the “Advanced” tab, you can check to turn on “Disposable template” and after applying also select “Default disposable template” to either be “(none)” or that very same AppQube itself. Now, you can use both this TemplateQube and Disposable Template AppQube to create new AppQubes and DisposableQubes fully within the ‘varlibqubes’ storage pool. This is likely to generally be the most desirable approach for most people.

Here is an example implementation of approach “c”:

Let’s say you want to base some of your fully stateless qubes on the ‘debian-12-xfce’ persistent template.

• 1. In Persistent mode: Create & Configure a new TemplateQube named ‘debian-12-xfce-stateless’ based on ‘debian-12-xfce’ and choose to store it in storage pool ‘varlibqubes’ (Advanced tab).
     
     
• 2. In Persistent mode: Create & Configure a new AppQube named ‘debian-12-xfce-stateless-dvm’ based on ‘debian-12-xfce-stateless’ and choose to store it in storage pool ‘varlibqubes’ (Advanced tab).
     
     
• 3. In Persistent mode: After creation, for the AppQube ‘debian-12-xfce-stateless-dvm’, change the ‘Advanced’ tab setting ‘Disposable template’ to be checked as turned on (click Apply), then the ‘Default disposable template’ to either be ‘(none)’ or ‘debian-12-xfce-stateless-dvm’ itself.
     
     
• 4. In Persistent mode: Create & Configure any new AppQubes based on ‘debian-12-xfce-stateless’ that you want to exist across multiple stateless boot sessions and choose to store them in storage pool ‘varlibqubes’ (Advanced tab).
     
     
• 5. In Stateless mode: You are free to now use any AppQubes based on ‘debian-12-xfce-stateless’ and DisposableQubes based on ‘debian-12-xfce-stateless-dvm’, which appear to remain fully stateless by storing ALL data in traditional image files within Dom0’s ‘/var/lib/qubes’ directory (that directory gets wiped and reset back to match the state of your last persistent session once your stateless session is powered down).

One may need to think about re-creating more or all of their system’s various types of qubes to be fully stateless like this, if needing such levels of statelessness. It should be possible to make every single qube on one’s system be fully stateless in ‘varlibqubes’ storage pool.

With higher fully stateless RAM space demands, you may need a computer with higher amounts of hardware RAM and an increase to the ’ dom0_mem=max:10240M’ setting in my Qubes Stateless step-by-step instructions to be set meaningfully higher than ‘10240M’.

5. The Pull Out Method

One approach I experimented with last year was to boot from a USB drive, and once Qubes Stateless was at the login screen, I just pulled out the USB drive, and the system seemed to continue working fine.

I haven’t done further testing on this approach yet, but it could be a powerful hardware enforced method for ensuring you are fully stateless.

@qstateless

When booting the drive with write access, I was able to pull out the USB drive once Dom0 booted up to the user login screen, seemingly without encountering any unexpected errors after that.

Qubes Stateless is an advanced hybrid state system, so controlling your level of statelessness can be a complex thing to manage, if desiring to be fully stateless.

Feel free to ask questions and provide further thoughts.