Extra Volumes

User Support
1

Is there a way to create additional volumes and (persistently) connect them to a specific qube/domain?

I have some classes of data that I would like to manage as units, without mingling them with other data in the private volumes of qubes.

I see in /var/lib/qubes/qubes.xml that there is a <volume-config> section, and it seems like I should be able to add additional volume references there, but I don’t know whether it’s safe to do that without fear of my modification being wiped, compatibility issues, inconsistency with some other backing database, etc. (Or if there’s a command I should be using to do this instead.)

Or is this whole notion just a bad idea and I should just store the data in directories in personal volumes and go with it?

2

I should mention that I’ve tried using qvm-block attach --persistent myvm dom0:dm-NN to persistently attach thin LVM volumes, but the dm-NN number changes every reboot. Is there a more persistent way I can attach these that won’t need to be reconfigured after every boot?

3

Specifically I followed this guide, but $dev changes each time.

4

I figured this out.

Specifically, I need to use /usr/lib/udev/rules.d/99-qubes-block.rules to make /dev/mapper/my--pool-myvol available, so I can use that name instead of the dm-NN name.

1 Like
5

Nope; not the solution. This just makes dom0:mapper appear in qvm-block list.

6

hi.

it might help:
https://forum.qubes-os.org/t/simple-config-for-faster-storage-for-disk-write-intensive-building-environment-via-btrfs-volumes/13616

edit: hum, maybe not, indeed, we just got dom0:dm-NN as well.

didn’t tried, but without the ENV{UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG}="1",
you could try with qvm-block attach --persistent

1 Like
7

With lvcreate, you can use:

-M|--persistent y|n
    When yes, makes the specified minor number persistent.

or

--minor Number
    Sets the minor number of an LV block device.

to get the dm-NN persistent.

As explained in the link above, make sure to use the udev rule to avoid dom0 to parse anything.

1 Like
8

Awesome, thank you, that was exactly what I’m looking for. I missed that in the lvcreate docs!

9

OK, elaborating a bit on the solution, because I want to emphasize that there are basically two parts here, and to record the full solution for posterity.

First: getting the dm-NN names to be persistent. Thanks @szz9pza for this. When creating a volume, you can add --minor NN --persistent y to force the mapped device to use a specific dm-NN ID. (Or use lvchange to change it later, bearing in mind that changing this involves a detach/reattach.) Note that if the requested minor block device ID is already in use, the device will silently not be mapped. Use higher numbers, to avoid this. I used 96 and 97 for my first two guinea pigs.)

Second: Keeping dom0 from interpreting the contents of those partitions. Qubes does this automatically for the volumes it creates, using a few naming conventions to keep udev from scanning them to create /dev/disk/by-* links. We need to add our own naming convention to exclude ours too.

Here’s where I noticed an issue: I added a /usr/lib/udev/rules.d/20-exclude-custom-lvms.rules to keep the 60-persistent-storage* rules from giving us links, but we were getting some from 13-dm-disk.rules too. Specifically /dev/disk/by-uuid/* and /dev/disk/by-label/* which (a) Qubes excludes for its internal volumes already (see 12-qubes-ignore-lvm-devices.rules), and (b) happens at #13, which would be before #20.

So I moved my file from 20-... to 12-qubes-ignore-custom-lvm-devices.rules, with these contents:

# This keeps later rules from scanning the volume for a filesystem in order to
# can create symlink device entries for the volume. To do so, dom0 would
# need to interpret the disk contents, which effectively reads untrusted data
# in dom0. So, don't do that.

ACTION!="add|change", GOTO="custom_dm_end"
SUBSYSTEM!="block", GOTO="custom_dm_end"
ENV{DM_UUID}!="LVM-*", GOTO="custom_dm_end"

# This takes uses a sentinel prefix of "custom_" to signal us to treat volumes
# this way.
ENV{DM_LV_NAME}=="custom_*", GOTO="custom_dm_ignore"

# These two volumes were created before I learned about this issue, so they
# don't have the right names, but I want to treat them the same way.
# (Update: I renamed them, but I kept this section and the GOTO above to
# keep a basic block around where I could match other patterns.)
# ENV{MINOR}=="96|97", GOTO="custom_dm_ignore"

# Make later rules ignore it.
LABEL="custom_dm_ignore"
ENV{DM_UDEV_DISABLE_DISK_RULES_FLAG}="1"
ENV{DM_UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG}="1"

# Export it.
LABEL="custom_dm_qubes_export"
ENV{QUBES_EXPORT_BLOCK_DEVICE}="1"

# Done.
LABEL="custom_dm_end"

Then I renamed my two existing guinea pig volumes to start with custom_, and gave them persistent minor device IDs, and used qvm-block attach --persistent VMNAME dom0:dm-NN with my newly-stable dm-NN.

With that file in place, now I just need to (a) name the volume custom_* to keep it from dom0 interpretation, and (b) give it a persistent minor device number, and: problem solved.

2 Likes
10

Thanks for sharing this information.

As experiment, I created a new file 12-qubes-ignore-custom.rules with just one line: ACTION!="remove", SUBSYSTEM=="block", ENV{DM_UUID}=="LVM-*", ENV{DM_LV_NAME}=="|custom2nd-*" ENV{DM_UDEV_DISABLE_DISK_RULES_FLAG}="1", ENV{UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG}="1".

And then I added two logical volumes, one with that prefix and the other with a different prefix, like lvcreate -V 2MB --thinpool vm-pool qubes_dom0 -n custom2nd-experiment.
Still from dom0, I ran mkfs.ext4 /dev/mapper/... on both volumes. Then I checked with lsblk -f, and the one with prefix specified in the udev rule had no filesystem detected, while the other volume had a file system detected (so dom0 parsed it). I didn’t yet test what happens after rebooting.

How did you decide to ignore “actions” add|change @djehuti ? I just copied /usr/lib/udev/rules.d/12-qubes-ignore-lvm-devices.rules and that said ACTION!="remove".