I am experiencing a strange issue with a new external USB drive on Qubes 4.2.4. I have successfully setup LUKS, then a single gpt partition for the whole drive using XFS filesystem.
I attach the data block device to a qube, where I successfully run cryptsetup luksOpen. A notification shows up that a new block device is available.
Problem 1
I can’t attach that block device to the same qube. – If I try, the VM becomes unresponsive and it is difficult to kill it or detach the block device from it. The journal in dom0 shows:
dom0 libvirtd[3109]: internal error: Failed to shutdown domain '27' with libxenlight
dom0 qubesd[3906]: socket.send() raised exception.
... # Traceback...
dom0 libvirtd[3109]: internal error: libxenlight failed to detach disk 'xvdj'
qubesd[3906]: unhandled exception while calling src=b'dom0' meth=b'admin.vm.device.block.Detach' dest=b'ddd533' arg=b'ddd533+dm-1' len(untrusted_payload)=0
... # Traceback...
Problem 2
I can attach the the new block device to a different qube without that qube asking me for LUKS password. Unless I am loosing my mind, this is a serious security issue.
Problem 3
After successfully attaching the block device to the second qube:
$ df -h /dev/xvdi1
Filesystem Size Used Avail Use% Mounted on
/dev/xvdi1 13T 91G 13T 1% /media/user/14T
i.e. it says that there are 91G used on the partition that has just been formatted, i.e. blank with no files written to it.
Why is this happening and what is the way to have it work correctly?
I’m not sure why you want to do this? The volume is already available in that qube, adding it again creates a loop situation.
You have unlocked a luks volume in a qube and it has given you this new block device, this volume will also be unlocked as it is in the source qube. Why do you think this is a security issue?
Your drive is quite large, maybe it’s just xfs doing some allocation? ~90G is not really that much on a 14T drive.
I’m not sure why you want to do this? The volume is already available in that qube, adding it again creates a loop situation.
It is a partition inside a LUKS container, not the opposite. Unlocking the LUKS container does not instantly reveal the partition, so I had to run partprobe in qubeA. Only then it becomes visible and mountable.
The confusing part with this is that mounting the partition inside qubeA does not “attach” the Qubes block device associated with it to qubeA, so it remains available for attaching in other qubes.
What happens when I attach it to qubeB and mount it there too:
Creating a file on the partition in qubeA does not show up in qubeB and vice versa, i.e. the 2 mounts behave as 2 separate drives.
If I create a file in qubeA, then remount the block device in qubeB, the latter shows that file.
This situation creates the possibility for messing up data and it is not clear which qube takes priority.
You have unlocked a luks volume in a qube and it has given you this new block device, this volume will also be unlocked as it is in the source qube. Why do you think this is a security issue?
Because entering a secret in one qube should not make data available in another - at least not without a special policy permitting that (which is not the case).
Your drive is quite large, maybe it’s just xfs doing some allocation? ~90G is not really that much on a 14T drive.
You are right, and this part is not Qubes related.
*info
Hmm I do seem to vaguely remember that mounting a block device used to remove it from the list of block devices available for attaching? Might have been a while ago… Right now, according to sudo udevadm monitor mounting an XFS or ext4 filesystem doesn’t emit any udev events so there’s nothing for /usr/lib/qubes/udev-block-add-change (theoretically responsible for removing a mounted device) to latch on to.
But you said you’re attaching the new block device created by luksOpen (the virtual device-mapper device with a plaintext view of data) to another qube. That’s kind of the opposite of making encryption secrets available to the destination qube, because the destination qube is totally unaware of the encryption passphrase or master key or even the fact that encryption is involved (in the source qube) at all.
While I am using the HDD in qubeA and it is not attached to any other qube, from time to time a desktop notification says either that the block device is available or that it is no longer available.
It is not power management of the USB docking station because the drive is in active use. There is no hardware disconnection either.
Hmm I do seem to vaguely remember that mounting a block device used to remove it from the list of block devices available for attaching?
The actual device (sda, i.e. /dev/sda in sys-usb) can only be attached to a single qube. However, the fictitious device (in fact, a partition) which appears after cryptsetup luksOpen (e.g. /dev/mapper/dm-1 in the domU) shows up as available for attaching even after being mounted in the qube which unlocks the LUKS container.
Might have been a while ago… Right now, according to sudo udevadm monitor mounting an XFS or ext4 filesystem doesn’t emit any udev events so there’s nothing for /usr/lib/qubes/udev-block-add-change (theoretically responsible for removing a mounted device) to latch on to.
A bug?
But you said you’re attaching the new block device created by luksOpen (the virtual device-mapper device with a plaintext view of data) to another qube. That’s kind of the opposite of making encryption secrets available to the destination qube, because the destination qube is totally unaware of the encryption passphrase or master key or even the fact that encryption is involved (in the source qube) at all.
After luksOpen any qube can access the data on the device without even needing the secret. All that is necessary is to attach that decrypted block device to a qube.- Yes, a necessary step but way too easy, so mistakes are possible.
I think this is a security problem because:
Attaching a particular storage to a particular qube happens with the intention to use that storage only in that qube, i.e. not to share the data with others (otherwise why do we need VM isolation?)
The data is what the key protects. If access to the data is granted, the key does not matter.
I don’t know why data exfiltration should be the central concern. Destroying important data is not less (and could be far more) important. So, a compromised VM can do all kinds of nasty things, including implanting hidden malware on the storage.
Some questions about your too:
Why is your tool not a standard part of Qubes?
I read about LUKS1 in the README. I am using LUKS2. Update necessary?
What is the difference between using overwrite-everything-with-random and:
dd if=/dev/zero of=/dev/mapper/dm-1
where dm-1 is the unlocked partition?
IIUC, what split dm-crypt does is to separate the unlocking from using the data. Currently, I unlock and use the drive in an offline minimal disposable (having only the tools needed). What would be the actual benefit of using split dm-crypt instead?
I guess so. Although I don’t fully remember if/how that ever worked.
To me that kind of composability feels elegant and Unix-y. A subsystem is explicitly granted access to the plaintext layer device, and it works in a transparent fashion without any special support.
Sure it’s possible to grant this access by mistake, but that’s also true for other kinds of devices. Maybe it should be addressed on a general level if the device widget makes it too too easy by accident, e.g. by misclicking. IIRC there might already be a ticket for that.
Not sure if it’s the central concern. But encrypted storage should be opaque to anyone unauthorized who’s gained physical access to it, and Qubes OS can uphold an unusually strong version of this.
For starters, the dom0 Bash script probably needs a Python rewrite. The design could also use some tweaks, such as not passing the encryption key through dom0 memory (which wasn’t feasible before R4.0), and using a shared DisposableVM for multiple dm-crypt mappings instead of wastefully launching one per device.
Yep, I’ve finally pushed a big commit that adds LUKS2 support etc.
Not much, it’s mostly for convenience.
It aims to prevent data exfiltration from the destination side to the source side, and (theoretical) exploits from the source side to the destination side via malicious block device content like headers parsed by the kernel/udev/cryptsetup.
and Qubes OS can uphold an unusually strong version of this.
What do you mean?
overwrite-everything-with-random
It might be a good idea to have the possibility to “pause” and “resume” that writing, e.g. to continue from particular sector/byte which dd can do. Writing man TB of data takes many hours.
It aims to prevent data exfiltration from the destination side to the source side, and (theoretical) exploits from the source side to the destination side via malicious block device content like headers parsed by the kernel/udev/cryptsetup.
What do you mean by side? Which is source/destination?
The destination qube would be the only one to see the data, if you make a mistake and it is attached to another qube, this one would only be able to see encrypted data.
Like, the version of “an encrypted disk is opaque to someone who just snatched it” that does not require an asterisk saying *But what if malware has previously stashed some not so opaque data onto it…
Source: Everything from the physical disk up to and including the VM that’s exposing the encrypted block device, e.g.
an SSD + a USB adapter + a notebook’s USB controller + sys-usb; or
an HDD + a NAS box running nbd-server + sys-net + … + a VM running nbd-client
Destination: The VM mounting the decrypted block device.
