Split veracrypt

General Discussion
1

INTRODUCTION

I spent my second month with Qubes working on a better way to support veracrypt volumes. (The first, was with minimal qubes.) I didn’t let the fact that I was Qubes n00b stop me from poking around a lot.

I managed to accomplish this, but since I am a n00b it’s entirely possible that out of ignorance, I did some very clumsy things that could be done more simply another way.

Wen I started using Qubes, I had a VM with sole access to the veracrypt volumes, and my initial plan was to use it to mount the volumes, then work on them there–which of course would have entailed installing all sorts of stuff on that qube. That’s exactly how I worked it on my previous oracle virtualbox machine.

Then, realizing that it’s actually pretty easy to move files from one qube to another, I decided I would use copy-to-vm to move files to the qube where I would work on them. That lets me use one of Qubes’ strengths: compartmentalization.

But it could be problematic if the files are large. So that is a push in the other direction–towards having a bunch of qubes, each specialized with one application, access the place where the volumes are kept.

But I already liked compartmentalization…so I didn’t want a bunch of AppVM accessing those veracrypt volumes…especially when they are on a network and I don’t want those AppVMs accessing any network.

My physical system is connected to WiFi (to get to the internet) and a (hopefully) disconnected-from-the-internet ethernet. Qubes access one or the other, not both. And this is reflected in my color scheme (which doesn’t correlate perfectly with “trust”). Red, orange and yellow qubes connect to Wifi, and purple qubes connect to ethernet. Blue and green are offline linux and windows, respectively. And the veracrypt volumes are out on the Ethernet.

So what I really wanted was “split veracrypt” where the system that uses the volume is connected to it without seeing where the encrypted volume resides or otherwise accessing the location it’s stored.

Well eventually I did get such a thing. And I then worked on making it more sophisticated–and thereby easier to use.

How I’m going to organize this: First I’ll present a verbal outline of what you have to do to get this to work.

Then, I’m going to go through the key command lines. With these, you should be able to open the different VMs that are involved in the process, issue the commands, and eventually have a decrypted volume mounted on the “target” VM.

The next level of sophistication is to use qrexec so you can run all of the commands from one VM, or even create a combined script to run them for you. I’ll talk a little bit about this.

Finally I created a couple of GUIs, very similar to each other. The most important one is one where I can simply select my volume off a list of volumes, press a button for either read-only or read-write access, then wait for a password prompt, and then voila, the volume is decrypted and mounted on the VM on which the GUI was running. (Alternatively, have it run on some other machine. Which brings up the question as to whether it’s more secure to run this from dom0 than it is from the target machine. I welcome security-centered comments on this!)

THE OUTLINE

There is one object: The veracrypt volume. Which for this description, will be a file named volume.vrc. It can be either “locked,” with the contents inaccessible because it’s not decrypted, or “unlocked.” Notice that some of the actors I’m about to describe below may see it as locked, while others see it as unlocked.

[Incidentally, when one starts doing scripts and especialy GUI front ends, it’s handy to have a naming convention of some kind to be able to identify veracrypt volumes, which would otherwise just look like files with noise in them or more accurately, files you don’t know how to read. My convention is the .vrc at the end of the name. If your threat model is such that you don’t want people to be able to see which files are veracrypt volumes, then obviously, don’t do this just because I suggested it; think about it first!]

There are multiple actors here. There is the user, i.e., you or some other carbon unit. But more importantly there are the following qubes, which I will give notional names to. (I don’t actually have qubes with these names; my names are more in keeping with the all-lower-case, use-dashes-for-spaces convention Qubes developers tend to use. You and I will have to replace those names with real names, in command lines I show below.)

Store has direct access to the actual files which are veracrypt volumes.

Decryptor actually decrypts the volumes.

Receiver is the qube that will actually access what’s on the now-decrypted volume; he’s the one that wants to work on something in the volume.

Keys (optional) a qube which manages the veracrypt decryption keys, letting Decryptor have access when the user approves.

Also optionally, you can have a Manager, which issues the commands to the other qubes. That comes into play when you start writing remote scripts. If you’re just doing things at the command prompt, then you’re likely opening a terminal on every qube that’s involved and banging on your keyboard. YOU are the manager in that case. One you start writing scripts, you will be deciding, even if implicitly, which qube or qubes will take on the management job for you. It may still be split among individual qubes, or there could be another dedicated qube to do it…or it could be the Receiver, basically “pulling” veracrypt volumes towards itself. In my case I do both. Linux Receivers do their own management; Windows qubes (or rather, my one windows qube) have need a different VM, a Linux VM, to act as a Manager since Windows won’t run bash scripts.

A question for the security gurus: Is having the Receiver do the management insecure for some reason? If so, would dom0 make a good manager, or would a dedicated Manager qube be sufficient?

OK, we’ve identified the actors. What’s the sequence of events?

  1. (Implicit, unless on a gui front end, in which case it will be explicit) selecting volume.vrc, making sure it’s where Store can see it.

  2. Store needs to mount volume.vrc to a loop; so it becomes /dev/loop1. This is called “Checkout.” I’ll talk about loops a bit in the Local Command line section. But the main point is that once the loop is mounted, not only can Store see it, dom0 can see it. (But, importanly, dom0 hasn’t mounted it.)

  3. The loop needs to be made available to Decryptor–as near as I can tell this must be done by dom0. The Decryptor will see a /dev/xvdi device. This is called “Mounting”

  4. (part a) Decryptor does the heavy lifting. It actually runs veracrypt. It needs to prompt for password or keyfiles and decrypt the loop it has been given. This is called “Unlocking.” Now for the first time files on volume.vrc are accessible, but right at this moment, it’s actually mounted as /media/veracrypt1 on the decryptor (note that the name doesn’t resemble the name of the volume file as seen by Store).
    (part b) Decryptor needs to mount the decrypted volume to another loop, and this entails dismounting it from /media/veracrypt1. Doing this again makes it the new loop visible to dom0.

  5. The loop must be made available to Receiver (Attaching). Again it looks like this must be done by dom0.

  6. Receiver mounts the loop. Receiver can then access what’s on the decrypted volume, as if it was part of Receiver’s file system, but it is not limited to the file system space that Reciver has set for it in settings, rather it’s limited by the size of the volume.vrc file. (Another benefit of this is you are not doing read-writes to the SSD you’ve installed QubesOS on.)

The process of relinquishing access works in the opposite direction (and we will count the steps downward): 5-relinquish (done by Reciever), 4-detaching (done by dom0), 3-relock (done by Decrytor), 2-dismount (done by dom0), and finally 1-checkin, done by Store). You will see that there’s one substep of step 3 done in getting the volume onto Receiver, that has no corrsponding undo in step 3 of the reverse process; otherwise it’s pretty logical, the command names will obviously correspond.

With that, it’s time to get down to brass tacks.

LOCAL COMMAND LINE

1 CHECKOUT, done by Store

Summary

I’m going to assume that the Store qube has access to the actual file that is the veracrypt volume, and that it’s in /home/user/Volumes. So the full path is /home/user/Volumes/volume.vrc. Now it could be on a thumb drive (which you’ll have to mount) or a NAS (which [wait for it…] you’ll have to mount) or a separate SSD (which is completely different; in this case you’ll have to mount it), or it could just be stored in the Store VM. Which ever one it is, I’m assuming you’ve already taken care of that, and volume.vrc resides in /home/user/Volumes.

[Obviously if your names are different, you make substitutions.]

The Store qube handles all veracrypt volumes. There could be five or ten distinct VMs running that might want to be receivers, with perhaps some of them having two or more decrypted volumes mounted. It makes no difference, there’s one Store and he doesn’t even have to know the name of the Receiver. Or Receivers. [In my case Store does know the name, but only so he can display status in my GUI front end, as to who’s connected to which veracrypt volume.] The only restriction, which seems to be enforced by veracrypt itself, is that ONE user can access a particular decrypted volume at a time.

Essentially, this step is a matter of taking volume.vrc and making a block device out of it. But instead of being set up as a disk or partition (e.g., /dev/xvdi), it will be mounted as a loop. In fact, loops are typically how you turn a file (as opposed to an ssd, thumbdrive or hard drive) into a block device.

Unlike thumbdrives, that automatically get set up as a disk device and just have to be mounted, a file must be explicitly “deviceified.” (Once you do that, IF the file has the right layout, it can then be mounted as a file system. However, a veracrypt volume that is not unlocked is just random-looking bits, so we can’t do that here.) To make the volume.vrc file into a loop device, you use the losetup command.

So to check volume.vrc out, you can type the following:

sudo losetup loop1 /home/user/Volumes/volume.vrc

If you want Receiver to have read only access to the volume, do the following instead:

sudo losetup loop1 /home/user/Volumes/volume.vrc -r

Unless you’ve got things set up a lot differently than I do, you will need the sudo on the command; without it my Store qube claims to have no idea what losetup is.

(If you have multiple veracrypt volumes checked out, then you’ll have loop2, loop3 and so forth; please substitute appropriately. It appears that losetup wants to use a number, though so keep that in mind.)

Once you do this, you will see loop1 on an lsblk listing (with no indication of what it is, but the full listing will give you the size). The lsblk command shows you all block devices (including loops, but also including disks and partitions). Your loop will also appear on losetup’s listing mode, “sudo losetup -l”, where only loops are shown. There, you will see it’s /dev/loop1 and you will see the file that got mounted (/home/user/Volumes/volume.vrc). In both cases (lsblk and losetup -l) you will see an indication that the loop is read only if that’s how you did it.

Also, dom0 can see this loop. The point of this step, in fact, is to ensure that dom0 can see it. More on that in the next step.

If you are undoing this whole process, going in the reverse direction, the last step of that process is:

losetup -d loop1

That dismounts the volume from the loop device, and now it’s just a file, volume.vrc, sitting on Store.

2 MOUNT, done by dom0

Summary

Now dom0 can see the loop1 device, that is actually volume.vrc.

And you as the user can see it in two different ways, on dom0.

The first is to issue the qvm-block command in a dom0 terminal, without any parameters.

The command will show a bunch of devices (such as USB devices) but you should see Store:loop1 listed. And it will actually give the path on Store, in this case /home/user/Volumes/volumes.vrc. There is also a “used by” command but this should be blank, for now.

The second way is to look at the qui-devices menu on your desktop. This is the one that has a little picture of an SD card and a usb thumb drive (at least I think that’s what they’re supposed to be), when you bring that menu up it typically lists your thumb drives and so forth. Well, when you do it now, it will list your new loop device in bold, near the top. Store:loop1 - /home/user/Volumes/volume.vrc

IMPORTANT: At this point Decryptor must be up and running. So I have to discuss that, as it could be trivial or not, depending on how you choose to set up Decryptor.

Decryptor could be an ordinary AppVM, or it could be a disposable VM.

You could just have a regular AppVM as decryptor, in which case it can get multiple volumes attached and you will see xvdj, xvdk, and so on. In fact you’ll probably want to do this at first as you learn the process (then later if you go with a DVM, delete that AppVM). If that’s what you’re doing, it might have been running before you even checked out volume.vrc; otherwise you just have to start it at now. Either way, you’re ready to do the actual mount.

But I chose to use a disposable VM (and I think you will want to, as well, once you know what you are doing), so that each of these could handle its own volume, which simplifies the script (it can assume /dev/xvdi). Another advantage is the disposable VM must, however briefly, know your decryption key, and a disposable VM will forget it when shut down–an AppVM won’t. If you’re doing that, then part of dom0’s job at this step is to fire up the disposable VM. I’m going to assume a named disposable VM, i.e., one where you specify the name, rather than one of the random ones (e.g., disp1248) If you do follow the Disposable VM route, you’ll need a disposable VM template. I’m going to assume you know how to set that up (especially since we have GUIs that can do it for you), and assume you’ve named the template “Decryptor-Template”. The command to fire it up is:

qvm-create --class DispVM --template=Decryptor-Template --label purple Decryptor

(Pick whatever color you like, obviously. I went with purple because Store is purple on my system; Store is purple because Store is on the Ethernet.)

That command will start the Decryptor qube, and you’re ready to do the mount.

[Note: if using DVMs you will need different names for different Decryptors, perhaps you could name this DVM as “Decryptor-volume” since it’s decrypting volume.vrc.)

If you’re totally manual you can just use the qui-devices menu to attach the volume to Decryptor (Decryptor has to be running): Just hover over that line on the menu, move right, and a flyout menu listing all running qubes shows up. Select Decryptor and you’ve done Step 2. (In fact, I’d do it this way the very first time since that very first time, you’re trying to prove the concept.)

But, there’s a command line to do this, and if you ever want to automate with scripts or a gui front end, you need to know it. So here it is: In a dom0 terminal:

qvm-block attach Decryptor Store:loop1

Which simply means attach the block device named Store:loop1 to Decryptor.

It’s not necessary, in this step, to specify read-only if you want read only.

Now decryptor will see the thing in /dev/xvdi (or if it’s got multiple things connected to it, it could be /dev/xvdj, etc.). We’re now ready for the next step.

To undo this step, on Dom0:

qvm-block detach Decryptor Store:loop1

And, if Decryptor is a disposable VM, shut it down.

3 UNLOCK, done by Decryptor

Summary

Decryptor must have veracrypt command line version installed if you want to automate the process. If one were willing to go with a more manual mode requiring you to open a terminal on Decryptor, then you might be able to run the GUI version of veracrypt on Decryptor. I never tried it. However one key command must still be issued at the command line (because it is not a command to Veracrypt). I don’t know how the GUI version would respond to the results of this command.

I’m going to assume here that there’s one veracrypt volume per Decryptor VM. If you’re not doing it that way, then please allow for the possibility of multiple disk devices being attached to Decryptor at one time.

This step starts with a device mounted to /dev/xvdi. This is the still-encrypted veracrypt volume.vrc file, but the Decryptor has no idea where the thing actually resides; it’s just an object that needs to be mounted.

In this step, veracrypt will decrypt this, then mount it, thereby unlocking it. Then we partially dismount it and make a loop available to dom0–except this loop is the decrypted volume.vrc

The Decryptor must remain up while the decrypted volume is in use; its role is to decrypt/encrypt things on the disk behind the scenes.

Veracrypt’s command line version will decrypt the volume with the following command:

veracrypt ... -k "keyfile" --pim=nnn --non-interactive --stdin --slot=1 /dev/xvdi /media/veracrypt1

OR

veracrypt ... -k "keyfile" --pim=nnn --password="yourpassword" --slot=1 /dev/xvdi /media/veracrypt1

Where I show quotes, you actually need to type the quotes.

The … should be replaced by “--mount” for read-write, and “-m ro” for readonly. Note that if you originally checked out the volume as read only, it’s really going to remain readonly, even if you were to specify read-write here. Receiver might even think it can write to the decrypted volume, but no changes made will stick; if you relock and unlock the volume, they’ll be gone. It’s best to match them, though, so the user doesn’t accidentally make changes that will vanish. If Receiver sees a readonly system, it won’t let the user do anything at all to the volume.

If there is no keyfile, then type -k "" (i.e., explcitly type the empty pair of quotes.)

If you are not using a pim, then use zero, i.e., “--pim=0”

The first form of the command will read your password out of standard input, which means it should be part of a command something like this:

cat passwordfile.txt | [that whole command] or echo $PASSWORD | [that whole command]

The second form of the command seems to require you to type the password on the command line, which is a very bad idea in a script (it’s almost as bad to put it on the command line if typing manually). Be aware that your command line might end up in a journalctl file somewhere. But it is possible to use that form of the command without actually typing the password; say it’s stored in a file: (I’m assuming, for this example, read-write, no PIM, and no key file, to keep things simple.)

veracrypt --mount -k "" --pim=0 --password="cat password.txt" --slot=1 /dev/xvdi /media/veracrypt1

The cat password.txt part, surrounded by backward ticks (upper leftmost key of a US keyboard) causes the system to execute that command and put its output there, so your password ends up between the quotes.

Now someone might object that storing your passwords in a file is a bad idea, and normally they’d be right–the file would usually be stored in the DVM template somewhere (or worse the TemplateVM that the DVM template is based on). But if the file is being brought in from somewhere else, it’s not so bad.

You have the flexibility to determine how to keep that password (and keyfile, and PIM number, as appropriate) secure. I actually created a rather complex scheme where yet another VM actor, called Keys, prompts the user for the password; Keys then encrypts it and puts it on a RAM disk on Decryptor, which then decrypts it and feeds it to the veracrypt command. The password is thus never saved to disk. That’s a subject worthy of its own post, possibly.

The result of a successful decrypt is:

There is a new device, /dev/mapper/veracrypt1. This will show in lsblk. The ‘1’ matches the slot number in the veracrypt command. If you are doing multiple decrypts on one VM, you’ll need to number other ones with slot 2, and so forth.

Veracrypt created this and will maintain it. But it’s not usable as a drive, so veracrypt automatically mounts it. And if you look, you will see that…

There is also a drive mounted to /media/veracrypt1. This is your veracrypt volume, decrypted. You COULD access it here if you wanted to. Note that this name was given on the command line; you could conceivably mount it anywhwere. However, we do NOT plan to leave things this way for long–so I used the Veracrypt GUI’s default here because I’m used to it. Nonetheless, it’s probably not a bad idea (if doing multiple decrypts on one VM) to make the number match the slot number just to be able to keep things straight.

We do NOT want this decrypted container mounted on this Decryptor VM, we want to mount it on Receiver!

So dismount it.

This is the hack that is key to the whole process. Veracrypt will normally mount decrypted drives wherever it is run, but it’s possible with this hack to mount them on a different VM! (This is not, so far as I know, documented by Veracrypt; I was simply goofing around and said “gee I wonder what happens if I do this?”)

Simply issue:

sudo umount /media/veracrypt1

and then make it into a loop:

losetup loop1 /dev/mapper/veracrypt1

Or, for read only, you should do

losetup -r loop1 /dev/mapper/veracrypt1

Again I recommend replacing the ones with some other number if you are dealing with multiple containers in one VM (I don’t because I use the DVM method). This is for consistency so you can tell which things are associated with which container.

[There is one potential exception to the statement that the losetup should be done with a -r for a volume that started out from Store as read only, and that is if it is to be attached to a Windows VM. Windows VM won’t mount it as a drive if it thinks it’s a read-only drive. If you want a read only drive on windows, do NOT specify it here. Windows will mount it, and it will behave like a read-write drive, but no change you make will actually “stick”–once the drive is detached and relocked, the changes will be gone. It’s potentially confusing if you’re doing things on a Windows box and forget that D: is really only read only, so you might just prefer to never mount a read only drive to Windows.]

Now you have a loop that dom0 can see. This should seem eerily familiar! But the key difference is, this loop is decrypted!

To RELOCK, when you are done, first remove the loop:

losetup -d /dev/loop1

The decrypted drive is already dismounted and we like it that way. Veracrypt command line will not fail if it’s not mounted. So:

veracrypt --dismount --slot=1

gets rid of /dev/mapper/veracrypt1.

The volume is now locked again. If you have done this in a disposable VM, you can shut it down now.

4 ATTACH, done by dom0

Summary

This is the step where the decrypted volume is given to Receiver. Receiver will simply see /dev/xvdi (or some later letter; the Receiver can have multiple veracrypt volumes attached to it). He’ll have no idea it’s a decrypted volume and that the volume “lives” on Store.

Just like Step 2 you can use qui-devices to do the attach, or the command line in dom0:

qvm-block attach Receiver Decryptor:loop1

There is again, no need to distingush read-only from read write. Because the loop was created at the end of step 3 as read only or read write, the /dev/xvdi block device will be marked the same way, and that will carry through when Receiver actually does the mount.

Dismounting works the opposite way, like before:

qvm-block detach Reciver Decryptor:loop1

5 MOUNT the decrypted drive, done by Receiver

Summary

If Receiver is a Windows box, nothing need be done. The loop, once assigned to the machine by dom0 with the qvm-block command, automatically gets mounted as D: drive (or E: drive, etc).

Linux qubes, on the other hand, will see a new entry in lsblk, /dev/xvdi, /dev/xvdj, and so on.

Linux Receivers have to figure out which one of these is the new drive, then mount them.

Of course it’s your choice where to mount them; this is the part that in my scripts, is the most customizable.

There is also one more wrinkle you should be aware of. The mount command will differ according to what file system is on the decrypted drive.

You can determine the file system with:

lsblk -f /dev/xvdi -o FSTYPE -n

(where, obviously you use xvdj or k or whatever, as appropriate)

If the result of this commnd is either vfat, exfat, or ntfs, you must mount like this:

sudo mount -o uid=1000,gid=1000 /dev/xvdi /home/user/your-chosen-mountpoint

Otherwise the disk will mount read-only no matter what (it will look like root is the owner and you, as “user” have no write privilege, and you won’t be able to change that ownership, even with sudo. And even if you can, it’s a pain in the ass to do so).

If it’s any other file system then just do:

sudo mount /dev/xvdi /home/user/your-chosen-mountpoint

If you try to use the -o uid/gid stuff, it will give you a rather nasty error message and refuse to comply.

To undo, i.e., to relinquish the volume, just do

sudo umount /home/user/your-chosen-mountpoint

Though you can actually just rip the drive away by having dom0 do a qvm-block dismount, it’s more graceful to do this first, then do the qvm-block dismount.

COMMAND LINE Conclusions

If you’re still with me you should be able to walk through the process, rather tediously, and mount a veracrypt volume visible on one qube, on a very different qube that can’t see the volume file at all. Once you’ve done that, you will positively want to write scripts, even ones that run on the actors, and perhaps even automate some stuff that has to be done manually–for example, the decryptor script could determine whether /dev/xvdi is readonly from the lsblk command and decide on its own whether to do read-only.

REMOTE SERVICE CALLS

You may want to run these steps from another machine other than the one they take place on. In fact, if you want to use a manager (or to have Receiver take on that role), you must be able to do this.

One way (the only way I know of) is to use qrexec-client-vm. But this requires a fair bit of setup. In fact, I’ve already myself forgotten a lot about how to do it, and I’ll be refreshing my memory as I write this.

Client Side of a Service Call

The general command line structure of qrexec-client-vm is like this. I’m using Store as an example “doer” of the command, and will be discussing the check out.

qrexec-client-vm Store vera.store-checkout client.sh arg1 arg2 arg3...

Store, is of course the VM you want to perform the command…in this case it will be to check out, or check in, a container. The servicename (here vera.store-checkout) specifies this. client.sh is a script on the calling machine (i.e., one where you issue the command). And then there are the command arguments.

Let me discuss the client.sh first, it’s simplest and provides some context for the rest.

During a qrexec-client-vm call, the client.sh script is invoked; it writes things to standard output. The doer (Store) reads things from standard input. Store then executes the command and can write things to its standard output, which client.sh will see on its standard input. Store then returns a status, and client.sh exits. So note: in qrexec-client-vm, all communication between the client and service (other than the return code) are done on standard input and standard output.

I’ve fiddled with this a bit and have never been able to get it to stay open for more interaction between the two systems; no doubt it’s possible some way I haven’t tried. (I am a bit of a n00b after all.) In fact, since for calling checkout, I’m just telling someone else to check out the file, I really don’t need anything back from Store other than its return code; a way to indicate it failed somehow (e.g., maybe I fatfingered and gave it the name of a non-existent veracrypt volume).

I ended up writing so many different client.shs with names specific to the services that did nothing but pass command line parameters to the service, that I eventually just wrote five generic versions of it and was done; each version took a different number of command line parameters.

So my client that takes 3 parameters, I named vera-3-param-client. Where to put it? Well, do you want this stuff in your template, or in your AppVM? You’ll probably figure out where best it goes as you move towards more and more automation. (I settled on /usr/lib/qubes in a templateVM qube, for things installed on templates; for things installed into AppVMs I went with /home/usr/.local/vera-client. Most of my Receivers have things installed in their templates.

This is one case where I will give source code, because this file is so simple:

#!/bin/bash

ONE=$1
TWO=$2
THREE=$3

echo $ONE
echo $TWO
echo $THREE
exec cat >&$SAVED_FD_1   # print result here, not there.

When run by qrexec-client-vm it simply copies three command line arguments to standard output (so that the service on the other VM sees them) and awaits anything on std output (it does pass standard output back, so you could have your service return error messages and the like if you want). Any return code from the service will be returned by this script.

Note that this script doesn’t need to know the name of the service it’s communicating with; that’s handled, apparently, by qrexec-client-vm.

It should be obvious what to do for vera-0-param-client, vera-1-param-client, and so on.

[32000 character limit…this continues in the first comment.]

7 Likes
2

This is a continuation of the main post.

Setting up a service

So, now for this BIG thing…how do you set up a service. What do you do on the server side of the service to get it going?

In this example, Store does the checkins and checkouts, so Store is the server for the service. All of this will be done on Store.

By convention, service names have dots in them, for example, in this case I named the service, on Store, to do the checkout “vera.store-checkout”. I created a separate service to do read-only checkouts, “vera.store-checkout-readonly”.

(I named all of these services vera.something, and in general the thing right after the dot is the actor name or something close to it (vera.decrypt-something, etc.). It will be important someday to be able to find anything having to do with this, so I made sure “vera” was in the name of all scripts and services.)

Things must be done on the service provider, in this case Store, as well as on dom0. Dom0 is the hidden mediary behind all calls (it can also, itself, provide services).

On Store, you need a script that will actually do the work. In this case I created a script named vera-store-checkout-svc. This will simply read its command line parameters (these are the same parameters sent by vera-N-param-client), validate them (one thing to do is make sure the volume exists) and then do the losetup command I mentioned above. It will return an error code if it fails; that will be seen on the client machine as the return code from the qrexec-client-vm call.

Now, you’ll note that I mentioned two distinct services, but there is only ONE script to do the work! That’s because the service is really a front end for the script.

Now a service must be defined in /etc/qubes-rpc…in this case it’s on Store. In here, you create your services, vera.store-checkout and vera.store-checkout-readonly. You can actually have these be scripts. I chose to have both of these be links to /usr/bin/vera-store-checkout-svc.

vera-store-checkout-svc checks its zeroth command line parameter, $0…which is the name it was invoked under. if $0 is vera.store-checkout-readonly, it does a readonly losetup, otherwise, it does a regular (read-write) one.

You don’t have to do it this way, you can have two scripts in /etc/qubes-rpc, or two in /usr/bin with two different links to them, or whatever. I like doing it the way I did it, because there’s one place to make changes to a script if I decide to change things.

You’ll want another service set up to do checkins (no read-only variant needed here), and you may want one that sends back a list of available veracrypt volumes (handy if you want a GUI front end for the whole thing). (That would be a case of the client qube needing to handle standard input from the service. But as it happens, I’m able to use vera-0-param-client for this too, because it returns stdout from the service.)

Connecting Service Client to Server

So that just leaves one piece of the puzzle. You still have to connect the client to the service, and that goes through dom0.

And dom0 will want to validate that your client has permission to call the service!

This is controlled, in the /etc/qubes/policy.d directory on dom0.

If you go here and do an ls, you’ll see a number of files with names starting with numbers. Essentially, when someone invokes a service somewhere, dom0 starts looking through these files until it finds a relevant line that will help it determine if the caller has permission to invoke the service on the server.

The numbers seem to cluster around the 20s and 30s, and around the 80s and 90s. You could insert the new service stuff into one of these files, but if you read about these things in other topics, where someone wants to change the permissions on some service, they will always recommend you copy the original file to one with a lower number, and make the changes there, that way you can revert to the original settings by just deleting your file.

In this case, we’re defining new services, so I just created a totally new file, 60-vera.policy. By the time you’re done doing this there will be a number of services you’ve defined.

Here is part of this file. Note the name of the service in the first column, a star in the second column, a tag in the third column. The fourth column is the name of the Store qube (in my case I went with vera-store). Finally the word “allow”.

So rendering the first line as an English sentence, it means, “allow any qube with the tag vera-store-client, to invoke the vera.store-checkout on qube vera-store.”

vera.store-checkout          *   @tag:vera-store-client    vera-store                allow
vera.store-checkout-readonly *   @tag:vera-store-client    vera-store                allow
vera.store-checkin           *   @tag:vera-store-client    vera-store                allow

I use tags to validate requestors; I could instead have hardcoded the name of a qube. In any case this is the qube you are invoking the service from, the client.

To set a tag, just do “qvm-tags QUBENAME add vera-store-client” on dom0 (this is permanent until undone, like a qvm-feature). To see a list of tags for a qube, just do “qvm-tages QUBENAME” with no tag name at the end. To remove a tag, issue the first command again, only use “del” instead of “add”.

You’re now set up! On some other qube, you can cause Store to check out a volume!

Even with a service written and available, you will probably want to write a script on the service’s client qube to invoke it, because qrexec-client-vm commands can be a pain in the butt to type.

In fact, you might even end up writing a big, omnibus script that invokes the checkout service, then the mount service on dom0, and the unlock service and the attach service. I even created a service on Recipient to notify it that a volume had been given to it and it should go ahead and mount the volume (if I am actually ON Recipient rather than on a different Manager qube, that script gets called directly, of course). And that’s what I eventually did. That way I could issue ONE command (On Recipient, or a Manager qube) and have the whole process happen. Error handling at that point gets to be interesting, let me tell you, because if some early step fails you want to exit AND know where you exited…and if you write another script for releasing the volume, you’ll want to make it smart enough to know where the other script had to leave off because of the error. That’s all interesting stuff, but it is “just” bash script writing even if it is complicated, and I’ll leave you to go off and learn that if you don’t know it already.

Once you have these scripts, if you have the skill you can write a Gtk gui front end that runs on the Recipient or Manager. If Store has an LS service, the front end can invoke it and then, using the output, can display a list of available volumes; you can select one and the Gui front end invokes your omnibus script. Easy peasy.

You can do all sorts of stuff; what I’ve given you here is info that someone who knows how to write scripts will need to get started.

2 Likes
3

I’m still going thru this before starting the implementation. Just a quick q. Were you able to script the attaching, mounting, & LUKS level decryption of an external HD or NAS that would hold the VC vaults in the Store qube?

The problem I had was the device ID changes depending on the USBs and the state of Wifi & Mic.

I could not find a static identifier for the external drive. As a result it drove my bonkers, attaching, decrypting manually. It just wasn’t viable for my workfow. I can imagine it would work great if my internal HD could handle it…but for the data I deal with, that’s just not doable. I need to make it work with an external drive or NAS, and it needs to be a streamlined, friction-free, the attaching and mounting process.

4

Does this expose the Veracrypt files directly to dom0? And if so, is it possible a malicious file could leak to dom0?

5

In which direction do you imagine things going wrong?

If you’re worried about dom0 accessing the drive behind your back: As far as I know, dom0 would have to actually mount it as a drive, which it doesn’t. (It shows up as a device in the lsblk listing, akin to /dev/xvdi before you mount it somewhere.) I can’t, however, exclude the possibility that some clever piece of software could somehow read it even so–I don’t know enough about Linux at that level.

Unfortunately, there doesn’t seem to be any way to avoid dom0 involvement at this level. USB devices are “visible” to it as devices even with sys-usb so I would infer that the risk is deemed minimal. But I can’t speak for the security gurus.

I’m much more confident that nothing on that loop can “leap out” and infect dom0, not without the loop being mounted. So malware on the drive can’t infect dom0 through this path.

To summarize, the only way something could get off a mounted loop onto dom0 is if dom0 pulled it from there, somehow…(and I’m not even sure that’s possible). If dom0 is doing that, it already has malware on it.

6

I’m still going thru this before starting the implementation. Just a quick q. Were you able to script the attaching, mounting, & LUKS level decryption of an external HD or NAS that would hold the VC vaults in the Store qube?

I never tried to decrypt the drive that store looks at to see the veracrypt volumes. That’s actually unencrypted. (Since it contains nothing but veracrypt volumes and infrastructure to back them up regularly…it seemed pointless to even try…the containers are encrypted, so why encrypt them again?)

I did create a gui popup to prompt for the share password and then mount it, but that works because mounting a share is a command line. I don’t know what-all LUKS is capable of. If you can come up with a command line that will unlock your LUKS-encrypted drive, one that doesn’t interactively prompt, then you could probably do something similar to what I did. As far as I know, though, LUKS will prompt for a password.

If you can get a “password included on the command line” LUKS decryption command, then one can certainly set up a script to prompt for the password, then pass it to that command line, without actually hard coding it in the script (eek!). One of the things I wrote in GTK was a little popup to prompt for a password; I use slightly different versions of it to mount my share, and to get the veracrypt decryption key. (There are very slight differences between those two programs; I think I use #ifdef macros to switch between them.)

I didn’t really discuss any of that, because gaining access to your decrypted volumes is a sort of “prerequisite” to decrypting them, and your case and mine differ. (I didn’t even realize you had them on a LUKS encrypted drive until now,)

7

Hard to say. I would like to hear of anyone else can imagine how malware could leak thru the loop device.

I’m wondering though if you could close an potential hole by replacing dom0 with a manager qube or having the decryptor itself expose the decrypted loop to the receiver? Is there anyway for the qube to expose the loop device to another qube?

8

This has been the sticking point. And boy did I try to make it work. If anyone comes up with a script that will auto attach, mount, and decrypt a Luks device, and work regardless of the USB attached devices or wifi/mic state, please let me know how you did it.

9

Believe me I did try to do a more direct transfer. I found out I could NOT call qvm-block directly from another qube. Though even that would have been the qube directly calling it on dom0, so that’d have been unsatisfactory. I had to write a service on dom0 just to turn around and issue the qvm-block command.

I’m not hugely concerned about this because dom0 isn’t mounting the loop, and I can’t imagine an unmounted loop harming dom0 unless dom0 is already compromised and malware goes "fishing’ on the loop device (somehow) to find more bad stuff

10

Balancing security with resources:

When setting up the vaults, compartmentalization would give you the most security.
But, setting up 1 large Veracrypt vault, and then have folders for each qube in the main vault would be faster and a better use of resource.

The folders could be exposed as loop devices right?

If so the loop devices should keep the compartmentalization. No files should be able to leak outside the folder right?

This would mean only 1 decrypt operation. And the qubes could share all the free space in the vault.

This seems a better use of resources. And if files can’t leak between the loop devices, it should not lose any security.

Do you see any problem with this?

11

Also, placing the scripting inside a fast opening dom0 vault, then encrypting the password files, would allow you to pass the decrypt password to the password file (not the actual password), as an arg to the script in the qube.

Dom0 has the password to the password file. The qube has the encrypted password file. Dom0 calls the Decryptor script, and passes the password to the password file when calling the script.

This would allow you to remove all password prompts except for an initial password prompt when opening the top level dom0 vault, that held all the scripts. And you could have separate 128char password strings for each vault.

The only human readable password would be for the dom0 top level scripts vault.

This requires Veracrypt in Dom0. Not sure how much of a security hole that is. But having your scripts exposed is likely a bigger hole.

I’m also not sure where the arg passed to the script ends up in log files.

12

OK, first let me makes sure I understand what you’re saying here. You’re talking about opening a vercrypt vault somewhere–possibly on a VM that can see the veracrypt volume itself, rather than on some other system–and then using loops to pass around folders on that unlocked volume. So basically, adopting my “actor” names, Owner would be the same as Decryptor, but then Owner would pass a loop of a (decrypted) folder from within the unlocked volume.

Or perhaps you do want the structure I described, but with more added…Recipient would be as disconnected from Store as before, but rather than being the final user, it would pass loops for individual folders to other places.

I’m honestly not sure you can make a loop from a folder. The problem is, as I understand it, a loop must be a contiguous block of storage, or at least a FILE, and the files in a folder could be anywhere, physically, on the file system. (Remember, a folder is itself really a file, it just contains pointers to other files.) If your loop contains pointers outside of the file it was made from, what happens? But I could well be wrong about this! I need to try an experiment. Another thing is, even if a folder is a loop I’m pretty sure its size is fixed at that point, and you won’t be able to add things to it.

You could get that effect, however, by nesting veracrypt volumes and hard-coding the passwords for the contained ones…that would basically make them as secure as a folder (i.e., not at all, not by itself anyway), whilst ensuring what’s in the “folder” is contiguous. The downside is that you have to worry about filling your “folder” since its maximum content size is the size of the volume. (And of course you do NOT use the same password for your nested volumes, as for the outer one!) Or, even better, you can just create a big file and turn it into a loop [I am not completely sure as to how to do it], without encrypting it at all…but again, its maximum size is fixed.

Certainly a scheme like this can work if you don’t mind everything being controlled by one password, but now you have to worry about filling up the big volume, and (severally) each of the little ones, whereas only having little ones means you only have the latter worry. (Of course either way you have to worry about the physical capacity of whatever-it-is you’re storing things on; but that’s simply part of life no matter what.) Also, you end up backing up the entire big volume every time you do a backup, unless you create a second, third, etc. big volume to back up the little ones into. (I have a backup scheme for my multiple volumes that functions on the NAS, but it’s a totally different story–this schema of mine may have helped it work a little better as a side effect. Long story.)

It should be possible to use the command lines I’ve given you as a starting point for building something like this; you’ve got info on how to unlock a veracrypt volume; you’ve got info for how to pass loops around. It’s just a matter of putting the pieces together, in a different way than I did.

1 Like
13

I guess I’ve got to elaborate on my password handling a bit. I pretty much elided that in my description above, assuming that the password “somehow” got to Decryptor which had a choice of either echoing into standard input and having veracrypt read that, or having it insert it into the command line. I actually do the former (though I’ve commented out a version that does the latter).

At first, I had Decryptor simply prompt for a password, which meant the script couldn’t run unattended. Then I created Keys, and there’s a popup in Keys to get the password and pass it back to Decryptor. But I encrypt the password. I have Decryptor generate a public/private key pair, and it passes the public one to Keys. Keys encrypts the password, and passes that back to Decryptor. Decryptor actually puts that file on a ramdisk (easy to create), decrypts it there, then echoes it, piping the output to the veracrypt command line. Nothing actually gets written to disk in the clear.

The password popup on Keys is actually a security feature, otherwise ANY qube, including mallet-qube could simply generate a key pair and invoke the service on Keys to get the password. The popup gives the name of Recipient, and the name of the volume. (The logic here is similar to being forced to select, in a popup, the name of the VM you’re doing a qvm-copy to. There’s no reason they couldn’t let you specify this on the command line (as you can do on dom0 only); they just chose not to so that you as a user would have to positively affirm the move. Otherwise a qube could push malware to other qubes without your knowledge.)

Now Keys doesn’t HAVE to ask for a password; it COULD just open a file with passwords in it, once, and pass them back on request–I think this is pretty close to what you want, and it actually doesn’t break anything I described; the password is getting to the Decryptor “somehow” but “somehow” is different. It could even be different passwords for different volumes. Or the file could be full of veracrypt keyfiles (which Decryptor would just leave on its ramdisk and give veracrypt the path to); again either use the same keyfile for every volume, or different ones. Either one of those would spare you the inconvenience of typing a password every time. And I have been thinking about doing something more like this; e.g., unlocking the password vault IF a token is present and after I type a password once. Keys could be a LOT more elaborate than it is right now, and probably will be someday. (I’m not tired of typing passwords, yet, though it would be nice if the dang popup wouldn’t popup in the upper left hand corner of the screen, every time…how about under the mouse, or perhaps on top of the front end where I just hit the button that says “Read Write”? Not trivial, because it’s a completely separate program on a different VM. Hmm, I just realized part of the request could be a set of screen coordinates…)

But even if you’re totally allergic to typing in passwords, I STRONGLY recommend popping up a prompt to confirm, every time you want to unlock a vault, to guard against spoofing. Something with “who” and “what” written on it, with an OK button. (And of course you need to actually READ it.)

That aside, as you can see there are ways to make multiple independently encrypted containers almost as convenient as a one-time-password, even within my schema. It’s a matter of writing Keys the way you want it…and I didn’t say much about Keys, nor its interaction with Decryptor, because it’s largely a separate “thread.” And there are enough split-gpg instructions out there to give one a start on that.

(Another thing I’d want in place is a secure token. I often read that something like Yubikey or LibremKey “can” act as a secure token; but they never say how to actually use it that way, because they really want to spend pages and pages to tell you about how it handles gpg keys, so they just drop that comment in passing and move on.)

1 Like
14

If you get this to work please let me know how you did it. I would love to add that layer of security to VC vaults. I couldn’t get it to work so far.

15

@Emily
It wouldn’t be direct; it’d be something I’d set up Keys to require to unlock whatever it is I keep the VC vault passwords in.

16

Are you setting up any vm-pools on the NAS device for this storage qube, or are all your vm-pools on your primary drive?

If they are on your primary drive, are you able to get dom0 to recognize the additional space on your NAS?

17

All VMs live on the primary drive. I haven’t the foggiest idea how to work that otherwise.

dom0 doesn’t access the NAS, only Store (and other purple VMs) do. In a totally different area of the NAS, for instance, I keep my music library, so my music player qube of course has to access that. Theres also a qube that has the browser I use to interface with the NAS operating system (that browser can go nowhere online). And I still have the old qube around that I used to use to connect to veracrypt volumes before I did the split. It actually has the gui version of veracrypt on it, and I still use it to make new volumes and other sorts of maintenance.

So dom0 doesn’t touch the NAS, that’s what purple qubes are for.

And conversely, so far no veracrypt volumes reside anywhere on this system, either where dom0 can see them or any of the qubes. Though I might make one for Keys someday if I make it more elaborate (how better to store key files, for instance).

1 Like
18

RE: Sharing VC vault with dom0.

I want to bind all of dom0 files to mirrored files in a VC vault, so that when the bind is activated, dom0 is effectively running off the VC vault.

Will dom0 allow you to attach and mount a loop device to itself?

And if it does, will this open up a security hole that you can see?

19

Will dom0 allow you to attach and mount a loop device to itself?

I’m afraid I don’t understand the question. You want to know if dom0 can attach a loop to dom0? Or attach the loop to the loop. (What is refered to by “itself” is unclear to me.)

Assuming the first–that you want to know if dom0 can attach a loop to itself, as far as I know dom0 can do that. I’ve never tried, because I have no interest in doing so.

Security implications are, as far as I know, the same as usual (though the specific nature of loops might add more I’m unaware of); we’re routinely urged to not put anything on dom0 we don’t understand fully; and in keeping with this, moving anything to dom0 is deliberately made as difficult as possible.

(This is why, when I started doing minimal qube templates after reading what Sven did on the subject, I decided that most of the setup of the template would be done with a script copied onto the template from a different VM (vault level, install repository) right after it is cloned, rather than having to maintain stuff on dom0. Of course managing this is itself complex and I ended up with a big “over script” on dom0, so I probably didn’t gain much. My next tinkering project (after creating and installing sys-audio) will probably be to try to slim that mess down.)

But I’m a bit puzzled by the direction this conversation is leading. At first you seemed concerned that dom0 would be corrupted by the mere act of passing unmounted loops from one VM to another; now it looks like you want dom0 to be involved in the whole process. Maybe I’m just not understanding you.

[EDIT: OK, I found this https://forum.qubes-os.org/t/architecting-an-undetectable-ghost-layer/8087/57 so now at least I think I understand your ultimate goal.]

20

In the light of something I learned when playing with using ZFS as the file system within a Veracrypt Container, I want to modify what I wrote above. Under decrypt, opening the summary tab, I describe mounting the decrypted volume then unmounting the file system so that it can be passed as a loop to the receiver.

I discovered that there is a command-line parameter to avoid mounting the file system. It’s absolutely necessary if the container has a file system that veracrypt doesn’t understand; it’s also a good idea no matter what so that the decryptor has no visibility into the contents of your container (not even for a split second). This can also be done in the veraracrypt GUI by opening up the mount dialog, clicking “Options” in the lower right, then, almost at the bottom is an option to not mount the filesystem. Check that.

OK here’s the rewritten section.