Fully immutable qubes with whitelist of persistent folders

User Support
1

Hello,

Qubes OS provides write protection for the root filesystem via templates. To also clean up home, disposables can be used.

What I now really would like to do:

  • create a fully immutable app qube with volatile root and home
  • keep one (or certain) folders inside home persistent

Qubes template docs already list some of its potential advantages:

It is possible that malware, especially malware that could be specifically written to target Qubes, could install its hooks inside the user home directory files only. Examples of obvious places for such hooks could be: .bashrc , the Firefox profile directory which contains the extensions, or some PDF or DOC documents that are expected to be opened by the user frequently (assuming the malware found an exploitable bug in the PDF or DOC reader), and surely many others places, all in the user’s home directory.

[…] the problem of finding malware hooks in general is hard

Also note that the user filesystem’s metadata might got maliciously modified by malware in order to exploit a hypothetical bug in the app qube kernel whenever it mounts the malformed filesystem.

An immutable- by-default filesystem in combination with minimal templates would reduce attack vectors, provide clean state on restart and give an easy mental model, where to look for potential malware.

Example case for me is a specific self-contained AppImage (*1) jailed in a sandbox. This app relies on one persistent data folder, which gets mutated during usage. Otherwise no write access should be permitted to other folders in home.

My first idea has been to use disposables in combination with bind-dirs (*2), but according to #3704 (*3), this does not seem to be possible yet.

I also found a second issue “Feature suggestion: optionally immutable /home #3258” (*4). There has been done some work (*5) by @tasket . User 3hhh mentioned (*6), a folder can be mounted somehow with qvm-block inside a disposable as a kind of persistent storage. Based on my little experience with Qubes OS, I cannot fully understand and judge both solutions yet.

So I am asking: Has there been some progess on this area or is something planned? What would be the easiest, up-to-date, non-hacky solution for disposables with whitelisted persistent folders?

Thank you for this great project!

Note:
I could only post max. two links (new user restriction). So I cut the remaining here:

*1: _appimage.org
*2: _www.qubes-os.org/doc/bind-dirs/
*3: _github.com/QubesOS/qubes-issues/issues/3704#issuecomment-538323722
*4: _github.com/QubesOS/qubes-issues/issues/3258
*5: _github.com/QubesOS/qubes-issues/issues/3258#issuecomment-340436310
*6: _github.com/QubesOS/qubes-issues/issues/3258#issuecomment-725516370

2

After a bit more investigation:

A shared folder mechanism or mount folder to an external storage VM might actually do more harm than good, security-wise. It is also not supported officially yet.

Hence I did the following, which is (imo) a good compromise between persistent app qube and disposabe: delete home folder before shutdown in app qube, whitelisting app-specific folders.

Example script:

#!/bin/sh

# delete all (incl. hidden) files in home, but keep following exceptions:
# ~/.config/app
# ~/app
# ~/.local/share/applications
# ~/.config/systemd/user
BASE=/home/user
find $BASE -type d \
  \( -path $BASE/.config/app -o -path $BASE/app -o -path $BASE/.local/share/applications -o -path $BASE/.config/systemd/user \) \
  -prune -false \
  -o \( -type l -o -type f -o -type p -o -type l -o -type s \) \
  -exec rm -rf -- {} +
 
# delete all empty folders
find $BASE -empty -type d -delete

systemd shutdown service:

# /home/user/.config/systemd/user/cleanup.service
[Unit]
Description=Cleanup home after shutdown

[Service]
Type=oneshot
RemainAfterExit=true
StandardOutput=journal
ExecStop=/home/user/app/cleanup.sh

[Install]
WantedBy=default.target

Now create ~/evil.sh and reference it in .bashrc. Both will be cleaned up, when the qube shuts down.

1 Like
3

This is what you’re looking for:

2 Likes
4

@deeplow Thanks, this actually has been my first idea.

It would be great, if bind-dirs could support folders inside /home for disposables.

1 Like
5

If you want anything to be persistent you don’t want a disposable. What you probably would want in that case it so change the bind-dirs of a template to exclude /home/user/ but include specific folders inside home, for example /home/user/Pictures. Never tried this but I think it would be possible.

1 Like
6

@deeplow That is an interesting idea.

I did look up, what is stored in bind-dirs, but could not find /home/user to exclude. The home dirs seem to be mounted in a different way than with bind-dirs (if I understand you correctly).

Following the general idea, I managed to do the same thing with pure mount --bind, /home/user/Pictures being whitelisted as example.

whitelist-home.sh:

# create volatile home in /tmp
mkdir -p /home/user/Pictures 
mkdir -p /tmp/home/user/Pictures 
chown -R user:user /tmp/home/user

# remap private.img ("real" home) to different folder
mkdir -p /run/mount/xvdb
mount /dev/xvdb /run/mount/xvdb

# Overlay home with the volatile one from /tmp, Pictures whitelisted
mount --bind /tmp/home/user /home/user
mount --bind /tmp/home/user/Pictures /run/mount/xvdb/home/user/Pictures

Systemd service, invoked on early boot:

[Unit]
Description=mount bind, reset home with whitelist folders
Before=basic.target
After=local-fs.target sysinit.target
DefaultDependencies=no

[Service]
Type=oneshot
ExecStart=/usr/local.templ/whitelist-home.sh

[Install]
WantedBy=basic.target

The service + script is put in the template, so it cannot be overwritten in the app qube.

Test:

touch /home/user/evil.sh # lands in volatile home (/tmp), cleared
touch /home/user/Pictures/mypic.jpg # persisted

Is this a “reasonable” approach? I think, something like this also would be an interesting automated Qubes OS-feature to have in app qubes.

7

By excluding /home/user/ I assumed it would just be removing it from bind-dirs but as you state, apparently it is not this way. Then I don’t know what could be the solution.

My hunch is that there is a better way. But I don’t know too much about this, I’m afraid. Hoping someone else can chip in.

8

No worries, thanks anyway. I’ll stick with the erase home on shutdown alternative, as a bit easier to maintain.

Also found mentioning of this method in this blog post:

By having “cleaning” scripts in the root filesystem (so they could execute before the private image gets mounted/used) which, when activated, would sanitize, clean and/or remove most (all?) of the scripts that are in the home directory. Assuming such cleaning script is effective (i.e. does not misses any place in the home directory which would be used to persist attacker’s code), the user might just 1) upgrade the vulnerable software in the template, 2) restart the AppVM. It’s worth stressing that, as the upgrading of the vulnerable software takes place in a TemplateVM (instead of the compromised AppVM), the attacker who exploited the vulnerability previously, has no way of blocking the upgrade.

And:

The third method has not been officially merged into Qubes yet, and it is unclear how effective (complete) it could be in practice, but some discussions about it, accompanied by an early implementation, can be found on the mailing list.

So no clear solution on this.