How to setup an appVM on an external drive?

User Support
#1

Hi all, Qubes noob here. I would like to set up an appVM to live exclusively on an external HDD (connected via USB).

I searched around and found this discussion which links to this documentation. I tried following the steps in it, but I got stumped on the very first step.

In the documentation, right at the start, it says “These steps assume you have already created a separate volume group and thin pool (not thin volume) for your HDD.”. So, I followed the links on volume group and thin pool, but could not get those to work.

On the volume group documentation, it indicates the command to create a volume group vg1 that contains physical volumes /dev/sdd1 and /dev/sde1: vgcreate vg1 /dev/sdd1 /dev/sde1. I checked the location of my external hdd /dev/sda, but when I attempt to create the volume sudo vgcreate vg1 /dev/sda on dom0 terminal I get the following error: Device /dev/sda not found.

My first thought was to run the command on sys-usb (which I have as a disposableVM) as that is the VM to which the external drive is assigned to, but I got another error sudo: vgcreate: command not found. So I went to the appVM template (fedora34) to install it and got another error: unable to find a match: vgcreate. Which is weird as the documentation on volume groups is from Red Hat.

Has anyone been able to create an appVM on an external drive? If so, any tips on what I’ve done wrong, or better yet a nice noob friendly tutorial on how to do it?

#2

When you said the very first step, which one ?

#3

Right, I should have been clearer on my post.

The goal is to create an appVM on an external drive. The first step on the documentation is to have created a volume group and thin pool. To create a volume group the documentation provides a command vgcreate vg1 /dev/sda. I have been unable to run this command successfully on both dom0 and sys-usb terminals.

To be quite sincere I am actually not sure which terminal it should be run at either. Given my understanding of how Qubes works, this should not be dom0 as the USB external drive is/should not connected to it. On the other hand, I have also been unable to run the command on sys-usb either. The template for sys-usb (fedora-34) does not have vgcreate installed and when I attempted to install it, I got an error unable to find a match: vgcreate.

My question then is: how can I create a volume group and a thin pool from an external USB drive with the goal of creating an appVM on it?

I hope I managed to clarify the issue and the step which is causing me problems.

#4

I don’t have idea how to add new pool from a VM, but i’ll explain your situation.

that means you don’t have lvm2 installed.

actually that the very first step from the guide is from

Assuming the secondary hard disk is at /dev/sdb (it will be completely erased), you can set it up for encryption by doing in a dom0 terminal (use the same passphrase as the main Qubes disk to avoid a second password prompt at boot):

follow from this step, sudo cryptsetup...........

you need to install cryptsetup if you don’t have, but it only if you want your second storage device encrypted, if not skip this step, and start from :

First create the physical volume

if you decide to use this sys-usb to handle the second storage, you somehow need to know how to pass this device to dom0 then add new pool by running :

qvm-pool --add poolhd0_qubes lvm_thin -o volume_group=qubes,thin_pool=poolhd0,revisions_to_keep=2

then you can create an vm based on this vm by running :

qvm-create -P poolhd0_qubes --label red unstrusted-hdd

I think just do some experimental first how this work before doing what you want like you explain in the #1

#5

Hi @51lieal, I did attempt the sudo cryptsetup… command on dom0 and got an error Device /dev/sdb does not exist or access denied. I did not add this to the original post (perhaps I should have) as I believe it is the same error I got when I ran sudo vgcreate vg1 /dev/sdb (which is what I added to the original post).

I will try following the documentation example after I install lvm2 on the template. If that does not work, I will pass the device to dom0 and try the tutorial there.

Thank you for responses! I will post here what happens next.

#6

don’t run the command directly, you should examine first what your actual device is.
read carefully Assuming the secondary hard disk is at /dev/sdb (it will be completely erased),

your device could be /dev/sda, sdb, sdc, sdd or /dev/nvme0n1, nvme1n1, nvme2n1.

Learn cryptsetup and lvm first is highly recommended, before you proceed with this documentation or make sure you are ready to reinstall qubes / backing up important files.

learn basic one will do.

#7

Yes, initially I was checking the location of the device via fdisk -l, but I noticed that I can see the location (minus the /dev) on Qubes Devices on bar/panel.

I am doing all of this on a clean Qubes install as I am first trying to create the setup I want before moving in and making Qubes my daily driver, so if I break anything all I have to do is plug the usb drive with Qubes installer and start again.

#8

Hi @51lieal, I’ve managed to get the sudo cryptsetup… command to work on sys-usb and the device is now encrypted with the same passphrase as the drive in which Qubes is installed.

But when I attempt to run sudo pvcreate /dev/mapper/luks-… on dom0 terminal as described in the tutorial I get an error Device /dev/mapper/luks-… not found.. I believe this might be because the device is still assigned to sys-usb (which happens automatically when I plug a usb device).

Given this I attempted to perform the steps on sys-usb, which involved adding the UUID of the device to /etc/crypttab. This seems to have partially broken qubes. Sys-usb now can’t start nor can I open sys-usb template terminal to undo the changes. Tomorrow I will re-install qubes to get a fresh start.

But I have a question, in your second answer you mentioned

Do you know of a way to pass the device from sys-usb to dom0? I don’t know how to go about doing that. I am actually not sure how to not have sys-usb handle the device in first place, as all usb devices I have connected have always been automatically assigned to sys-usb.

#9

As far as I know it’s not possible to pass single USB device to dom0, but the whole controller that the device is attached to.
Anyway, there would be no need that would made me pass any USB to dom0. Otherwise I wouldn’t use Qubes.

#10

@enmus, I agree with the sentiment. Even given my, admittedly very limited, knowledge of how Qubes works, I believe that this would be a bad idea. Which is why I tried following the whole tutorial on sys-usb and ultimately ended up breaking my installation.

I was hoping to have an appVM on a 1TB usb drive I have to keep a program with a large storage footprint as I would only need it sporadically. Looks like I will have to keep its appVM on the laptop SSD. It’s a pity as this would have been a pretty good use of the usb drive.

#11

I am not sure how it would work with Linux, but for Windows, I use dispVM to which external storage is attached, and there portable apps are stored. Private and system storages in dispVM are read only, nothing can even be executed from there, so everything happens on an external storage, I just use dispVM to access it.
Please apologize if I’m not clear or I misunderstood your goal, but the point is that hopefully no footprint can be found on a dispVM when external storage isn’t connected.
Maybe something similar is possible with flatpaks residing on an external storage in Linux?

1 Like
#12

The answer to a question why would you want to create AppVM on an external storage would probably help to get more proper suggestion from someone. In general, it’s about why and not how.

#13

@enmus, my goal is quite simple. Originally, it was to run bitcoin core on an appVM that is entirely located on a usb drive. The reason is, essentially, to save space on my laptop SSD as the current bitcoin blockchain size is about 324GB. I don’t really have any requirements for plausible deniability or even to hide it, just to save space (although I do like to implement cloak and dagger strategies on my Qubes setup just for the fun of it).

I like your idea of using a dispVM with a portable app. I’ll spend some time looking at this to see if it is possible. This would both provide me with my original goal of saving SSD space and add the fun of having a 007 setup. :smiley:

No need to apologize, if anything it should be me apologizing. I guess that my choice of words with “large storage footprint” was a bit careless and misleading, and made it seem that I would like to have some sort of setup that would be difficult to find the data forensically.

Either way thank you for the idea, I will definitely look into it. I believe the biggest issue will be finding a way to setting up bitcoin core as a portable app…

PS.: If you have any other ideas on how to make use of a usb drive to keep a large application off of the laptop SSD I’d really appreciate it.