Multi-user Qubes: Using sys-gui to make non-adminstrative user logins

Notes:

  • This is multi-user that can actually work for some limited use cases (see “use case” below). It is not secure multi-user yet.
    While it’s useful for running tests and developing multi-user functionality, and it might be useful for the cooperative use cases I mention below, do not assume that it is currently appropriate for giving accounts on your qubes system to your enemies.
    Some specific security issues (that give good reasons not to use it at this time beyond cooperative use cases) are at the end of this document.
  • This is a procedure that I came up with and am posting as a user. I’m not part of the qubes team, and the qubes team has not seen this procedure before anyone else is going to see it, so dont think it has any kind of “seal of approval” from the qubes team.

Use case:

Qubes can have a steep learning curve. This means that when you are setting up a system for someone else to use, that explaining qubes in full to them can be difficult.

Here we try to create restricted sys-guis, where they only need to see the parts that pertain to them. Basically a administrative gui login (dom0), and a restricted user sys-gui login.

The restricted user gui login serves the following:

  • Cut down the user interface to just the necessary interface to tasks that they need to do

    • example: Scrolling through a list of 63 AppVMs to find the correct one is a bit cumbersome for me, and could be confusing to new users.
      While the ‘find’ option is there, that would require explaining, and for them to know the name of the appvm they want and/or the command they want.
      (I.E. they cannot use process of elimination of the available options to figure out which one they want)
    • example: They may not need to see that sys-firewall type qubes connect to sys-net. Instead some users may be better off just knowing about a sys-internet (renamed from sys-firewall for them), and sys-vpn-to-work, and may be better off not even knowing about sys-net on day one.
  • Remove as many possible accidents from them as we can

    • example: removing the possibility to accidently use sys-net instead of sys-firewall
  • Put certain administrative abilities behind a seperate gui login/password to force a discussion on the security of doing whatever changes they were hoping to acccomplish before implementing it.

Please note that this procedure has not gone through any security review

Step 1: set up the basic sys-gui:

You can set up a sys-gui qube by doing:

qvm-prefs default-mgmt-dvm template fedora-38

(Note: fedora-40-xfce is another option, but sometimes gives a “no such module” salt.{something}.six} error### (also, dont use debian)

sudo qubesctl top.enable qvm.sys-gui
sudo qubesctl top.enable qvm.sys-gui pillar=True
sudo qubesctl --all state.highstate

if everything worked, you can then:

sudo qubesctl top.disable qvm.sys-gui

Then finally create a qube called work-sys-gui and then do:

qvm-create work-sys-gui label=blue
qvm-prefs  work-sys-gui guivm sys-gui

Then you can logout, then change the upper right corner option on the login screen to “sys-gui” then login again.

You should now be logged into sys-gui! (sys-gui seems to have a blue hamster as the desktop background nowdays, so if you see that you are definately looking at sys-gui). Great. Now quick, log out and log back into dom0 before the screen lock triggers!

(Should we give some “normal” pointers to documentation on using sys-gui here?)

Should we give something here at this point to give people some sys-gui stuff to explore before moving on to a restricted user? Give me your thoughts.

Step 2: From basic sys-gui to a restricted user

Give yourself a way to unlock the screen when logged into sys-gui:

Before anything else, lets create a way for you to unlock the screen when you are running sys-gui:

sudo /bin/sh -c "sudo grep '^user:' /etc/shadow | sed 's/^user:\([^:]*\):.*/sys-gui:\1/' >  /etc/qubes-sys-gui-user-passwords"
sudo chmod 600 /etc/qubes-sys-gui-user-passwords

now create a file in dom0 called /usr/local/bin/fix-guivm-password.sh, that contains:

NEW_PASSWORD_HASH=`sudo grep "^$1:" /etc/qubes-sys-gui-user-passwords | sed 's/^[^:]*://'`

ESCAPED_PASSWORD_HASH="${NEW_PASSWORD_HASH//\//\\\/}"

qvm-run --pass-io --no-gui --user=root $1 "sed -i 's/^user:[^:]*:/user:$ESCAPED_PASSWORD_HASH:/' /etc/shadow"

then do:

chmod a+x /usr/local/fix-guivm-password.sh

Then, make a backup of /usr/bin/qubes-guivm-session , and replace it with:

#!/usr/bin/bash -e

USER=`whoami`
if [ $USER = 'user' ]; then
    if [ $1 = 'startxfce4' ]; then
	startxfce4
	exit;
    else
	VM='sys-gui';
    fi
else
    VM="sys-gui-$USER";
fi

#echo "Should be running $VM as our sys-gui domain... but actually running $1 as our sys-gui domain instead"
echo "Throwing away pramater $1 and running $VM as our sys-gui domain instead"

print_usage() {
cat >&2 <<USAGE
Usage: $0 vmname
Starts given VM and runs its associated GUI daemon. Used as X session for the
GUI domain.
USAGE
}

if [ $# -lt 1 ] ; then
    print_usage
    exit 1
fi

# Start VM, gui-daemon and audio
qvm-start --skip-if-running "$VM"
qvm-start-daemon --watch "$VM" &

#exec qvm-run --pass-io --no-gui $1 sed -i "^user:/user:`sudo grep '^1' /etc/qubes-sys-gui-user-passwords | sed \"s/^[^:]*://\"`" /etc/shadow

date --iso=minute >> /tmp/qubes-guivm-session-whoami-test-`whoami`
whoami >> /tmp/qubes-guivm-session-whoami-test-`whoami`


#/home/user/qubes-scripts/sys-gui-user/fix-guivm-password.sh $VM
/usr/local/bin/fix-guivm-password.sh $VM

# Run the inner session (Xephyr) and wait until it exits
exec qvm-run -p --no-gui --service "$VM" qubes.GuiVMSession

(WARNING: /usr/bin/qubes-guivm-session will get overwritten when you upgrade dom0. You can instead call it /usr/bin/qubes-guivm-session.multi and then change /usr/share/xsessions/sys-gui.desktop to call qubes-guivm-session.multi instead of qubes-guivm-session)

Now you should be able to logout of dom0, change the upper right corner option on the login screen to “sys-gui” then login again.

This time when the screen locks due to the idle timer, it should be the same password as “user” in dom0 to get back in (hopefully)

add a second sys-gui:

Create a file called “add-new-sys-gui-user.sh” that contains:

sudo qubes-dom0-update mkpasswd

NEWUSER=$1
NEWSYSGUI=sys-gui-$NEWUSER

sudo adduser $NEWUSER --groups qubes,tty

echo enter the password for the new $NEWUSER user
NEW_PASSWORD_HASH=`mkpasswd`
ESCAPED_PASSWORD_HASH="${NEW_PASSWORD_HASH//\//\\\/}"

###todo: if a password for that sys-gui-user vm  is already in there it will create 2.  add a check for that
sudo /bin/sh -c "echo '$NEWSYSGUI:$NEW_PASSWORD_HASH' >> /etc/qubes-sys-gui-user-passwords"
sudo sed -i "s/^$NEWUSER:[^:]*:/$NEWUSER:$ESCAPED_PASSWORD_HASH:/" /etc/shadow
echo password added

sudo /bin/sh -c "sed 's/user/$NEWUSER/' /etc/sudoers.d/qubes-input-trigger > /etc/sudoers.d/qubes-input-trigger-$NEWUSER"

##create the new sys-gui qube
sudo qvm-clone sys-gui $NEWSYSGUI

sudo /bin/sh -c "sed 's/sys-gui/$NEWSYSGUI/g' /etc/qubes/policy.d/50-gui-sys-gui.policy > /etc/qubes/policy.d/50-gui-$NEWSYSGUI.policy"

echo "now create the qubes that this user should have access to and do 'qvm-prefs guivm {new qube} $NEWSYSGUI' for each one"

then do:

chmod a+x add-new-sys-gui-user.sh;

Then doing:

./add-new-sys-gui-user.sh bob

will ask for a password for bob, then add a new dom0 user login called “bob”, create a sys-gui-bob, and set up some basic policies for it. Once that is done, then just like before, pick a color and do something like:

qvm-create bob-work --label blue 
qvm-prefs bob-work guivm sys-gui-bob
qvm-create bob-personal --label blue
qvm-prefs bob-personal guivm sys-gui-bob

in order to give bob some VM’s. Then log out of dom0, select “bob” as the login person, select “sys-gui” from the upper right session list, then type bobs password and you should see a new sys-gui
the blue “qubes domains” cube icon in the system tray should acknowledge the existence of the qubes you set sys-gui-bob as the guivm for.
(It’s fine if at this point, the bue “qubes domains” cube icon and the other qubes widgets (including the pull down meny) dont work properly in the sys-gui-bob yet, at this point we are happy just to have the second sys-gui for a differnt user. (We will address how to fix broken widgets in step 3))

You can confirm you are in bobs sys-gui-bob by right clicking on the desktop, selecting terminal and seeing the hostname of sys-gui-bob.

You could also go back to dom0 at this time and do:

./add-new-sys-gui-user.sh pete
qvm-create pete-work --label orange 
qvm-prefs pete-work guivm sys-gui-pete
qvm-create pete-personal --label orange 
qvm-prefs pete-personal guivm sys-gui-pete

and now you’ll have a pete account on dom0 with his own sys-gui.

  • Currently, all dom0 users can log into a xfce session in dom0. To remedy this, we can:

Create a file /usr/share/xsessions/admin-xfce.desktop that contains:

[Desktop Entry]
Name=Administrative xfce4 login
Exec=qubes-guivm-session startxfce4
Type=Application

Log out then try to log into the new “administrative xfce” xsession as user “user”. It should look exactly the same as your normal xfce dom0 desktop.

If that works, then log out then try to log into sys-gui xsession as user “user”. It should work.

If that works, then log out and try to log into the new “administrative xfce” xsession as user “bob”. It should just log bob into sys-gui-bob as if he had selected “sys-gui”.

If you are sure that everything is working, you can remove /usr/share/xsessions/xfce.desktop with

sudo mv /usr/share/xsessions/xfce.desktop  ~/backup-xfce.desktop

(Note: if you totally break your ability to log into X you can hit ctrl-alt-F3 and login there)

Why keep sys-gui when we can have as many sys-gui-bob, sys-gui-pete, … as we want? In the next section we are going to try to actually do something with sys-gui-bob, and the sys-gui generation salt script has been tested with the username “user”. This means when things dont work for you in sys-gui-bob, I want you be able to go try doing it in sys-gui when logged in as “user” and see if it works there in the more tested environment.

Step 3: The actual configuring of sys-gui-bob (I.E. opening up the policies)

This is where the real work starts. The majority of sys-gui’s configuration is through qubes policies.

Here we need to make a choice:

  1. If it’s your first user qube, then you probably want to give full write permissions, so you can make sure that sys-gui-bob works just like sys-gui. You can do that by:
sudo /bin/sh -c "grep 'sys-gui[^-]' /etc/qubes/policy.d/include/admin-local-rwx  | sed 's/sys-gui\([^-]\)/sys-gui-bob\1/g' >> /etc/qubes/policy.d/include/admin-local-rwx"
sudo /bin/sh -c "grep 'sys-gui[^-]' /etc/qubes/policy.d/include/admin-global-rwx  | sed 's/sys-gui\([^-]\)/sys-gui-bob\1/g' >> /etc/qubes/policy.d/include/admin-global-rwx"

If you are ready to start figureing out the policies, you have the option to enable “read only” permissions like this:

sudo /bin/sh -c "grep 'sys-gui[^-]' /etc/qubes/policy.d/include/admin-local-rwx  | sed 's/sys-gui\([^-]\)/sys-gui-bob\1/g' >> /etc/qubes/policy.d/include/admin-local-ro"
sudo /bin/sh -c "grep 'sys-gui[^-]' /etc/qubes/policy.d/include/admin-global-rwx  | sed 's/sys-gui\([^-]\)/sys-gui-bob\1/g' >> /etc/qubes/policy.d/include/admin-global-ro"

Alternatively, you can try leaving all those permissions off, and start exploring. Realize that if you do this and you run sys-gui-bob then try to load the settings for the work-bob qube, you probably wont be able to get to the settings screen (example: try clicking on the “qubes domains” blue cube in the upper right tray, then hovering over “work-bob” and see if the settings shows up. it probably wont).

Why can’t we load the settings? Policies! The qubes policy system on dom0 is denying tongs of sys-gui-bob qrexec requests, but we can’t see the notifications, cause the “policy denied request” notifications are on dom0!

How do you use policies, and how do we get the “policy denied request” notifications? Well luckily I just wrote that down in a seperate guide:

Our solution will be at this part: How to use the Qubes Admin Policies/API despite the lack of documentation - WIP
(where we log into dom0, then install sys-gui-vnc, then remote into sys-gui-vnc while logged into dom0, and try to create the same problem we are having in sys-gui-bob in sys-gui-vnc, but this time we get the notifications, cause we are logged into dom0 xfce, not a sys-gui!)
But you should probably read through the stuff before that to make sure you are in sync with the whole policies thing.

qvm-prefs {some vm} guivm sys-gui-vnc to give it access to a qube

Now you’ll start actually getting the policy errors! like maybe this one:
admin.vm.feature.List from sys-gui-vnc to debian-12

This is because debian-12 is the template of the qube that i did the “qvm-prefs {some vm} guivm sys-gui-vnc” command with, and sys-gui-vnc doesn’t have access to debian-12

Now you can start poking around, watch what error messages show up, edit the policies in the /etc/qubes/policy.d/50-gui-sys-gui-vnc.policy file, (possibly reboot sys-gui-vnc so it doesnt use cached values), try poking it again, repeat. Then after you get it working, copy those policies to /etc/qubes/policy.d/50-gui-sys-gui-bob.policy

Interesting places to poke around:

While the above script copied the sys-gui settings in /etc/qubes/policy.d/50-gui-sys-gui.policy into /etc/qubes/policy.d/50-gui-sys-gui-bob.policy, there are settings in /etc/qubes/policy.d/include/ that I did not copy since i wasnt sure what they did yet. We can see those policies with:

grep sys-gui /etc/qubes/policy.d/include/*

which should display something like:

admin-global-rwx:sys-gui @adminvm allow target=dom0
admin-global-rwx:sys-gui @tag:guivm-sys-gui allow target=dom0
admin-global-rwx:sys-gui sys-gui allow target=dom0
admin-global-rwx:sys-gui-vnc @adminvm allow target=dom0
admin-global-rwx:sys-gui-vnc @tag:guivm-sys-gui-vnc allow target=dom0
admin-global-rwx:sys-gui-vnc sys-gui-vnc allow target=dom0
admin-local-rwx:sys-gui @tag:guivm-sys-gui allow target=dom0
admin-local-rwx:sys-gui sys-gui allow target=dom0
admin-local-rwx:sys-gui-vnc @tag:guivm-sys-gui-vnc allow target=dom0
admin-local-rwx:sys-gui-vnc sys-gui-vnc allow target=dom0

Note that the “@tag:guivm-sys-gui” from this line:

admin-global-rwx:sys-gui @tag:guivm-sys-gui allow target=dom0

@tag:guivm-sys-gui” seems to be a autogenerated tag that documents what the qubes-pref “guivm” is set to for the vm. This means there is a autogenerated “@tag:guivm-sys-gui-bob” for work-bob. Given this, adding the line:

admin-global-rwx:sys-gui-bob @tag:guivm-sys-gui-bob allow target=dom0

gives sys-gui-bob permissions read,write,execute permissions for all VMs that have sys-gui-bob set as its guivm. Sounds pretty reasonable, so we probably want that one most of the time (depending on your use case). (it’s likely that something like that is needed for the “big Q” menu item to work in sys-gui)

also:

admin-global-rwx:sys-gui sys-gui allow target=dom0

says that sys-gui is allowed to control everything about sys-gui!

another option might be to give it a read-only version.

We could do something like:
sudo /bin/sh -c “grep ‘sys-gui[^-]’ /etc/qubes/policy.d/include/admin-local-rwx | sed ‘s/sys-gui([^-])/$NEWSYSGUI\1/g’ >> /etc/qubes/policy.d/include/admin-local-ro”
sudo /bin/sh -c “grep ‘sys-gui[^-]’ /etc/qubes/policy.d/include/admin-global-rwx | sed ‘s/sys-gui([^-])/$NEWSYSGUI\1/g’ >> /etc/qubes/policy.d/include/admin-global-ro”

Now it’s just the read-only part of those policies.
then remove those policies that we dont want.

But the important thing is that you now have the ability to set the policies to make sys-gui-vnc ultra powerful, and set the policies to make sys-gui-vnc really tight to the point that it doesn’t work, then compare the working and non-working sys-gui using the technique in How to use the Qubes Admin Policies/API despite the lack of documentation - WIP in order to figure out the minimum permissions that will allow whatever feature you are interested in to work. (then copy those permissions to the actual {sys-gui-user} that you care about.)

Remember to post below what you discover/ what configurations you end up using (and what your use case that the configuration is for)

Notes:

Note that you can run things in work-bob with access to your default network. You can also change the network from sys-firewall (or whatever your default network was) to “none”, and network access will no longer be available. perfect.

However, you are not allowed to change network access back from “none” to sys-firewall (or whatever your default network was).

also check the policy files to make sure no unwanted admin permissions have been given to this qube. You can do this with:

grep -ir sys-gui-bob /etc/qubes/policy.d/ | less

(Note there are more sys-gui policies in /etc/qubes/policy.d/50-gui-sys-gui.policy/include that should to be assessed too)

TODO: we need to come up with some naming conventions for everyone to default to. (example: call the qube “work-bob” or “bob-work”?)

Troubleshooting:

Here are some tips on troubleshooting.
Lets suppose that when you log as “user” into sys-gui things are fine, but when you try to log in as bob, the screen flashes black, then bounces you back to a new lightdm login screen.

The following procedure can be used to get info to debug the process:

hit ctrl-alt-F3 and login to dom0 console as “user”

do:

  auditctl -a exit,always -F arch=x86_64 -S execve

(this will turn on logging for all exec calls, meaning you will be able to figure out what programs are running when after this moment)
(note: this logging will stay on until you reboot the machine. Once you reboot the machine it will be gone)

Then hit ctrl-alt-F1

then produce the error again (I.E. log into sys-gui as user and succeed,
then try to login to sys-gui-bob as bob and fail again)

Hit ctrl-alt-F3

Now try to find the first timestamp of the events related to your loging in as bob by doing:

sudo journalctl --since=today | grep USER_

Lets say the first event of your session looks like its from “oct 21 14:32:01”

Then you can do:

sudo journalctl --since="2024-10-21 14:31:50" | tee /tmp/bob-login.log

(starting a few seconds early to make sure we catch the very start of the event)

from that output you should be able to see what programs ran

Now do the same thing but for sys-gui, find the the first time associated with the login events of user “user”, and

sudo journalctl --since={the time you found minus a few seconds} | tee /tmp/user-login.log

you can now compare the 2.

Since the timestamps will never match, you could get rid of them by:

cut -d ' ' -f 4- /tmp/bob-login.log > /tmp/bob-login.log.trimmed
cut -d ' ' -f 4- /tmp/user-login.log > /tmp/user-login.log.trimmed

then diff /tmp/bob-login.log.trimmed and /tmp/user-login.log.trimmed

(Note: you’ll probably want to use something like git diff’s “–word-diff=color” parameter, and not just use normal “diff”)

There is a much larger topic to address here about the untapped potential of doing general debugging with auditd. I’ll need to come back when i get more time to talk about it.

Important note on journalctl!:

When systemd itself tells you that the command that you should type is "journalctl -xe" don’t believe it! That command almost always either shows you nothing at all or a bunch of unrelated stuff to what you want.
The command you want is "sudo journalctl"
(journalctl will not give you a error message if you forget to use sudo)

Possible future extenstions

Standarization of simple things

Setting a naming convention for default qubes could eliminate code

This would need to be negotiated with the qubes team.
If we can negotiate the naming convention with the qubes team, then the second password file (/etc/qubes-sys-gui-user-passwords) will become unnecessary
By naming convention I mean:

how the login name should relate to the qube names

I used the naming convention {username}-work for work qubes and sys-gui-{username} for guivm qubes.

If qubes was to add multiple users in the future, would the qubes team want by default a users qubes to be named bob-work and bob-personal, or work-bob and personal-bob, or something else entirely?
Same for sys-gui-bob or bob-sys-gui.

A advantage of puting the username first for all associated qubes, is that alphabetical listings then also automatically become sorted by user

However, does it matter from a security perspective? Do we need to worry about the dom0 user name being something like this?:
sys-gui-bob" "rm -rf /";

Which would make his sys-gui be named: sys-gui-bob" "rm -rf /";-sys-gui
meaning that a script calling
qvm-run "$VM_NAME" "ls /home/user"
might suddenly be calling:
qvm-run "sys-gui-bob" "rm -rf /";-sys-gui" "ls /home/user"
which is the same as calling:
qvm-run "sys-gui-bob" "rm -rf /"

man, that guy must really hate bob. :slight_smile:

Also, if qubes was to split the administrator from the user in the future, would they rename the default dom0 xfce user (user 1000) to “d0root” or “dom0admin” or “admin” or something like that, and a new user (user 1001) called “user” ?

Security Inssues/Enhancements:

Note that the dom0 user “bob” is all powerfull in the same way that user “user” is.
If bob can ever get to a dom0 shell, bob effectively becomes a administrator.
While this may be usable for the cooperative use cases listed at the begining of this document, dont give “bob” a account on your qubes system if you dont trust bob completely.

I currently see 2 possible ways to handle it, either remove permissions from user “bob” prevent bob from ever getting a shell in dom0.

Removing bob permissions: bob is part of the qubes dom0 group

Note this runs in a script as “bob” during login:

qvm-run --pass-io --no-gui --user=root $1 "sed -i 's/^user:[^:]*:/user:$ESCAPED_PASSWORD_HASH:/' /etc/shadow"

One option would be to take away bobs qubes group and give him passwordless sudo to user user, but just for the command: sudo -u user qvm-run --pass-io --no-gui --user=root sys-gui-bob *?

(need to test if a user (bob) can still use a sys-gui, if a differnt user started that sys-gui (user))

similarly we could do that for other qubes commands that are part of the login sequence?

however, we would need to block bobs ability to get to root too.

Alternatively we can try to just make sure that bob is never able to get to a shell in dom0.

Preventing shell access: “Restricted” users can bypass the xsession by hitting ctrl-alt-F3 and logging in

This gets bob to a shell on dom0.
Deny them the ability to log into console? (probably a pam thing)

Preventing shell access: can lightdm ever get to a shell?

What happens if a user types their password into lightdm, hits enter, then really quickly starts spamming Control-C . Is there any way they could get to a shell?

This procedure should be security reviewed before trusting it, and not just rely on the list the security things I thought of while writing this document. :slight_smile:

3 Likes