Qubes in tmpfs 🤫

[qstateless]

Thanks @gonzalo-bulnes! Edit ability for the post solves, so a “wiki” works I guess.

As far as making the post its own independent topic, I’m not sure about that yet. This version of instructions are primarily meant to be a step-by-step implementation of @xuy’s how-to posts within this thread. I may decide to fundamentally upgrade and re-implement the whole Qubes Stateless system in the future, so that would certainly deserve its own topic.

For now, let’s just keep this post in-place, and I will let you know if that should change.

Thank you!

[qstateless]

Today, 2023-12-08, I have updated the Step-by-Step Instructions for Qubes Stateless.

This update includes an instruction to permanently disable Dom0 Swap.

This update also includes many re-written explanations to better explain what Qubes Stateless is and how it works. For those making use of Qubes Stateless, it may be worth re-reading.

Feel free to ask any questions or make any comments. Enjoy!

@qstateless

Edit Updates:

#6. 2023-12-08

• Added permanent disable of Dom0 Swap.

#7. 2023-12-08

• Improved several explanations throughout post.
• Minor edits.

[nealr]

I read this post yesterday and read it closer today.

I can see where this would be useful for a live USB setup, but I think it would be better as a live DVD, a sort of "take your hard drive free laptop to DEFCON’ kinda thing, where you can leave it sitting somewhere and nobody can do anything that would survive a restart.

I am not sure what value there is to having dom0 running from ram on a system that does have a disk. If something in a VM can reach out and compromise dom0, it’s already compromised Xen, and it can fiddle with VMs on disk. I guess dom0 mods for persistence, something that kinks one or more VMs each time the system started, would be a potential problem?

But the “first, let’s assume dom0 is compromised” seems like an enormous stretch. A quick Google shows there have been four ‘game over’ Xen breakouts during the life of this project. Has there every been a proof of concept demonstrated?

Isn’t there a trusted boot process that would render persistence impossible?

[qstateless]

Hi @nealr

Good thoughts…

@nealr

If something in a VM can reach out and compromise dom0, it’s already compromised Xen, and it can fiddle with VMs on disk. I guess dom0 mods for persistence, something that kinks one or more VMs each time the system started, would be a potential problem?

From a hardcore security perspective, you are generally right that a traditional boot drive is still accessible and writable, even with Dom0 running from RAM. So a breakdown of Qubes OS security via a Xen VM breakout exploit could still compromise one’s system across restarts, if a persistent drive is used.

@nealr

Isn’t there a trusted boot process that would render persistence impossible?

Detectable at least, with a TPM. I’ve read about things like UEFI Secure Boot, Heads, Anti Evil Maid, TrenchBoot (currently being integrated for Qubes AEM by 3mdeb), etc.

@nealr

I can see where this would be useful for a live USB setup, but I think it would be better as a live DVD, a sort of "take your hard drive free laptop to DEFCON’ kinda thing, where you can leave it sitting somewhere and nobody can do anything that would survive a restart.

Yes. Using write-protected USB or read-only DVD boot media would certainly increase one’s ability to eliminate the possibility of persistent exploits that could otherwise survive a system restart.

@nealr

I am not sure what value there is to having dom0 running from ram on a system that does have a disk.

I see a few general reasons for making a Qubes Dom0 boot session be disposable in RAM…

• System Administration
• Anti-Borking
• Anti-Forensics
• Anti-Fingerprinting
• System Security

System Administration:

• Having to account for and manage less state in less places can generally help ease sysadmin burdens. I personally take advantage of this benefit across a small fleet of multiple computers I administer. I’m planning to go further and use this to eliminate the use of local drives via RAM-based Qubes OS over PXE Network Boot.

Anti-Borking:

• If one wants to run scripts, tests, etc that might bork or mess up one’s Dom0, then running from RAM allows for a quick reset to clean system state. I personally have a Qubes-based development test computer setup specifically for this.

Anti-Forensics:

• On a normally-functioning and non-compromised system, one can achieve a greater level of anti-forensic / disposable computing for the entire Qubes OS system by having no Dom0 logs/state saved.

Anti-Fingerprinting:

• If someone needed their Qubes OS environment to have the same fingerprint between multiple restarts, then a RAM-based Dom0 would help with achieving that.

System Security:

• With a persistent boot drive, while running Dom0 from RAM, a Dom0 exploit would have to take the extra step of reaching out to compromise the drive or drive contents, beyond compromising the running Dom0 environment within RAM. So there would probably be some unaware exploits that do not assume Dom0 is even in RAM, where one’s system could be further protected against such exploits from persistent compromise beyond a restart. Not absolute security, but one more step further.

Of course, one could potentially use write-protected USB or read-only DVD boot media too. I experimented with RAM-based Dom0 and write-protected USB last year using Qubes 4.1, which I believe I wrote about some in this thread.

Layering various system attributes can potentially increase the fitness of a RAM-based Dom0 for various use cases and use conditions. A RAM-based Dom0 is another key attribute of choice that is now accessible to all who may find a use for it. Big thanks to @xuy for the technique.

Thoughts?

[nealr]

Yes, that clarified things for me.

My LinkedIn profile says R&D oriented CTO, and that’s pretty much why I’m in here. I talk to people who handle potentially “hot” data, or who operate in dangerous places. My big picture goal is living with Qubes daily on an HP Z420, then acquiring a stout laptop, likely a Dell Precision Xeon machine, and I expect pretty soon I’ll start getting messages that say “So … do I need to get this Qubes stuff, too?”

I’m more of an integrator than a developer, and I’d be advising people in operations.

I’ve encountered an Evil Maid in the wild and it is an ongoing issue, one that I suspect will ramp up as things in the U.S. increasingly destabilize. Ensuring the system I come home to is running the same OS I shut down when I left is important to me, and to others.

Write protected USB? Does such a thing actually exist? I recall having write protected SD adapters for microSD, this was just a simple plastic slide that put one pin out of service. I need to do some research here.

Having dom0 in ram looks like it’s good for development, testing, etc, but the added capital cost for groups of irregulars who struggle to get just A laptop that’ll run Qubes … my systems have the capability to do this, but as a rule I do not think that will be the case.

I moved at the start of the month and now I’ve got an environment where I can push off, do a half turn while rolling, and be in front of my second Z420, instead of having to run up and down stairs. I have a couple SATA SSDs, a couple small PCIe NVMe, and an external USB to SATA drive carrier. I’m working through the production scenarios - a person in the field with a single laptop, a single external drive, how do they install/run/backup? How do they handle seizure or theft of a machine?

So … I can see a dozen things I need to do prior to ever considering mucking around in dom0 internals and I’m likely the only one in my environment who would use this tmpfs solution. That could maybe change if a true read only USB device were available, but I suspect there are other keyring U2F devices that would accomplish more.

[qstateless]

@nealr

@nealr

Yes, that clarified things for me.

Glad it helped!

@nealr

I’ve encountered an Evil Maid in the wild and it is an ongoing issue, one that I suspect will ramp up as things in the U.S. increasingly destabilize. Ensuring the system I come home to is running the same OS I shut down when I left is important to me, and to others.

Yes. Having a securely measured and verifiable boot process, remote backup, and battery-redundant remote video/sensor physical monitoring of one’s spaces & devices would likely be important to crafting an Evil Maid solution. The Haven app was made to help with evil maid attacks, I believe.

@nealr

Write protected USB? Does such a thing actually exist? I recall having write protected SD adapters for microSD, this was just a simple plastic slide that put one pin out of service. I need to do some research here.

Yes. Write protected USB devices do exist. I’ve used USB thumbdrives with a WP switch, and SATA to USB adapters with a WP switch, and USB proxies that had a WP feature. Not sure how their circuitry is implemented. Maybe usually a simple one pin disable slide, like you mentioned.

@nealr

That could maybe change if a true read only USB device were available

Without checking, I’ve also never heard of true RO USB (read-only), as in being like ROM (read-only memory). I’m just familiar with WP USB (write-protected). Since one probably has to be able to write/rewrite data onto the USB storage device to be generally useful, any read-only/write-protection features would probably need to be switchable or added in-line as a secondary device. A write-once/non-rewriteable optical disk (DVD, etc) via a USB connected reader is the only thing I could think of for true RO USB.

@nealr

I moved at the start of the month and now I’ve got an environment where I can push off, do a half turn while rolling, and be in front of my second Z420, instead of having to run up and down stairs.

Nice. I know that feeling well. I have had to do the repetitive up and down stairs sysadmin bit many times.

@nealr

I talk to people who handle potentially “hot” data, or who operate in dangerous places.

@nealr

I’m working through the production scenarios - a person in the field with a single laptop, a single external drive, how do they install/run/backup? How do they handle seizure or theft of a machine?

If a network could be utilized without major issues (and that’s potentially a big IF in hostile scenarios), then, for local data purposes at least, I’d think it would be ideal to simply not store such data locally, and just access/load such things remotely instead.

@nealr

So … I can see a dozen things I need to do prior to ever considering mucking around in dom0 internals and I’m likely the only one in my environment who would use this tmpfs solution.

@nealr

My big picture goal is living with Qubes daily on an HP Z420, then acquiring a stout laptop, likely a Dell Precision Xeon machine, and I expect pretty soon I’ll start getting messages that say “So … do I need to get this Qubes stuff, too?”

Best of luck with your pursuits! Qubes is really a fun and powerful OS for us security-minded geeks. With how easy other monolithic OSes can be penetrated, like a hot knife through butter, I think something like Qubes is a necessity to those wanting a fighting chance at having their digital stuff remain their own.

I’ve implemented Qubes into some non-technical people’s lives too, however, it can take consistent IT assistance. For the extra effort, either be paid very well, love the people, or love the mission!

Cheers!

[ddevz]

@qstateless

From a hardcore security perspective, you are generally right that a traditional boot drive is still accessible and writable, even with Dom0 running from RAM. So a breakdown of Qubes OS security via a Xen VM breakout exploit could still compromise one’s system across restarts, if a persistent drive is used.

@nealr

Isn’t there a trusted boot process that would render persistence impossible?

Detectable at least, with a TPM. I’ve read about things like UEFI Secure Boot, Heads, Anti Evil Maid, TrenchBoot (currently being integrated for Qubes AEM by 3mdeb), etc.

I’ve been looking at flashable firmware on PCI devices. I wanted to mention that if you wanted a truely “persistence impossible” situation, you would need to address flashable firmware in your PCI devices, as i do not believe that gets addressed in “secure boot/measured boot/trusted boot”.

Here is a link of my attempt at addressing the situation: How to avoid flashable firmware compromises via pass-through PCI devices?