Create custom bash command for automatic backup & restore

User Support General
, , , ,
1

I want make bash command for automatic backup
First
start backup-vm and attach block device to it from sys-usb and decrypt attached block device and mount it and make encrypted
backup apps-vm and remove apps-vm from qubesos

Second for recovery
start backup-vm and attach block device to it from sys-usb and decrypt attached block device and mount it and make encrypted
backup apps-vm and remove apps-vm from qubesos start backup-vm and attach block device to it from sys-usb and decrypt attached block device and mount it and restore encrypted
backup apps-vm and remove backup.file from block device and dis-attach it from backup-vm

on dom0 terminal

  • start app-vm
qvm-start backup-vm
  • and wait 15s
sleep 15s
  • attach usb partion from hdd to app-vm
qvm-block attach backup-app sys-usb:"block device name"
  • auto decrypt block device attached to backup-vm
???
  • create app-vm encrypted backup on backup-vm attached block device
???
  • delete vm-s after backup
qvm-remove -y app-vm

customize command

qvm-start backup-vm && sleep 15s && qvm-block attach backup-vm sys-usb:"device name"

every time I tray connect external ssd block device name have changed

2

Thanks for the writeup, @anon_chan. It’s a great idea to automate backups and your work shows some ways to do that in dom0.

You posted this in Community Guides, but you also identify an issue in your process around block devices that looks like a question. I’ll respond to that here. But you might have better luck posting in the User Support category if you want to work through this or other issues.

In this case, “device name” is probably something like /dev/sda, /dev/sda1, /dev/sdb, etc. These are the names that Linux assigns new block devices when it sees them. Is that what you see?

I’ve found that if I fail to detach the block device from the VM before I physically remove the device, then the system will assign the block device the next consecutive device name when I re-insert the device.

For example, I would plug the device in to my laptop and see /dev/sda and /dev/sda1 in the Qubes Devices menu. Then I would attach the block device to a VM, exactly like you do with qvm-block attach. Then, I would unplug the device from my laptop. Then, the next time I plugged the device into my laptop, it would show up as /dev/sdb and /dev/sdb1 in the Device menu. Is that also what you’re doing?

This is expected behavior. To avoid it, try this. Plug the device in and get a /dev/sd[x] name for it. Attach it to the VM. Before physically unplugging the device from the computer, go to the Qubes Devices menu and detach the device from the VM. Now, unplug and re-plug the device. Do you get the same name for the block device as you got before?

This will work reliably until you plug another USB SSD into a USB port before this one. The system assigns these names to devices on a first-come, first-served basis. So the first device could get /dev/sda while your backup device gets /dev/sdb.

You can not assign static block device names to USB devices that you see in the Qubes Devices menu. You might do better to skip automatic attachment entirely and attach the device manually whenever you plug it in.

However, you can get static block device names in the VM you attach the block devices to. Open a terminal in backup-vm before you attach the block device to it. Print the current static block device names with:

ls -R /dev/disk/*

These are the static names of the block devices already attached to backup-vm. Now, attach the block device from the USB SSD drive. Try this again:

ls -R /dev/disk/*

You should see new names in the output. These names are static across backup-vm restarts, USB attach/detach, and USB plug/unplug. You can use any of the new names to refer to your USB disk in backup-vm. That should help with the next step, automatically decrypting the block device in backup-vm.

2 Likes
3

I fix that.
thanks.

but now I can’t mount device on backup-vm after decrypt it
and can’t create command line for backup selective vm’s also.

this last code I tried it

qvm-start backup-vm &&  qvm-block attach backup-vm sys-usb:sdb3 && qvm-run --pass-io backup-vm -- sudo cryptsetup open /dev/xvdi bk --type luks && sudo mount /dev/mapper/bk /mnt

mount command back with error

mount: /dev/mapper/bk/mnt: can't find in /etc/fstab.

I see decrypted block device on dom:0 not on attached vm after decrypt it

4

Right. You ran mount /dev/mapper/bk /mnt in dom0. But none of the commands you ran before that created /dev/mapper/bk in dom0. Instead, the decrypted block device for /dev/xvdi is in backup-vm. That is probably where you want it.

Don’t worry about scripting backup-vm startup and attaching the block device to backup-vm right now. Focus on getting decryption to work automatically and reliably. Try your cryptsetup command in backup-vm. Open a terminal in backup-vm and type that command in the terminal window. Does it succeed? Did it create /dev/mapper/bk in backup-vm?

1 Like
5

I replace part of code

sudo mount /dev/mapper/bk /mnt

with

qvm-run --pass-io backup-vm -- sudo mount /dev/mapper/bk /mnt

it’s working now

how I can
auto decrypt block device attached to backup-vm
and
create selective vms backup encrypted with verbose

6

Great. Decryption works and the decrypted volume is available in backup-vm.

create selective vms backup encrypted with verbose

The qvm-backup utility takes VMs to backup as arguments and supports a –verbose flag. Does that do what you’re looking for?

auto decrypt block device attached to backup-vm

The first step is to automate this part of your backup script with a configuration file in backup-vm:

qvm-run --pass-io backup-vm -- sudo cryptsetup open /dev/xvdi bk --type luks

You can configure Linux to automatically decrypt block devices with the /etc/crypttab file. Since backup-vm is an AppVM, you will need to persist /etc/crypttab.

Qubes VMs don’t create or use /etc/crypttab by default. You will need to manually create the persistent /etc/crypttab file under /rw/bind-dirs, as described in the paragraph that starts with “If you want to circumvent this process, …”, here. Follow these steps in a backup-vm terminal. Do not use qvm-run --pass-io.

First, follow the Qubes persistence directions to create _ /rw/config/qubes-bind-dirs.d/50_user.conf_. Then, add this line to that file:

# /rw/config/qubes-bind-dirs.d/50_user.conf
binds+=( '/etc/crypttab' )

Next, create the persistent /etc/crypttab.

$ sudo mkdir --parents /rw/bind-dirs/etc
$ sudo touch /rw/bind-dirs/etc/crypttab

Now add a line to /rw/bind-dirs/etc/crypttab to automatically mount the encrypted block device in backup-vm.

Earlier, you used /dev/xvdi as the encrypted block device. But that name isn’t guaranteed to exist in backup-vm. As I described in my first response, think about using one of the persistent names for your encrypted block device that you can discover with:

ls -R /dev/disk/*

After you’re done editing /rw/bind-dirs/etc/crypttab, restart backup-vm. Open a terminal in backup-vm. Does /etc/crypttab exist?

If /etc/crypttab exists, you can try attaching the encrypted block device from the USB device to backup-vm to see whether it mounts automatically. But don’t be too surprised if it doesn’t work, yet. You may need to encourage systemd to re-generate some unit files when backup-vm starts up.

This might seem like a lot of trouble when qvm-run --pass-io works perfectly fine to mount the encrypted block device in backup-vm from dom0. But it’s important for at least two reasons:

It reduces script complexity

This way, you can remove a line of bash shell code. That’s less complexity and less risk of failure in the future.

It increases security

Avoid using qvm-run --pass-io in dom0. This command passes unfiltered data from domU VMs to dom0, which could process that data. It’s the same as copying data to dom0. Qubes documentation recommends against copying data to dom0.

2 Likes
7

thanks @brian
are you can help me for understanding how crypttab work?
how I can use it?

8

Sure, @anon_chan. You need at least three things to decrypt a block device:

  1. The encrypted block device
  2. A passphrase or key for the encrypted block device
  3. The decrypted block device

You give cryptsetup this information through arguments on the command line. But you have to run cryptsetup manually at exactly the right time for everything to work.

The /etc/crypttab file uses the same information and does the same thing as the cryptsetup command. But Linux reads /etc/crypttab and decrypts devices at the right time, automatically.

For example, this /etc/crypttab content would tell Linux to mount your encrypted block device during startup:

# /etc/crypttab
# decrypted-device    encrypted-device    key-file
backups    /dev/xvdi    /etc/cryptsetup-keys.d/backups.key

When Linux starts, it will read /etc/crypttab, decrypt /dev/xvdi with the passphrase in /etc/cryptsetup-keys.d/backups.key, and create a block device to the decrypted data at /dev/mapper/backups.

For this to work, you need to copy the passphrase for your encrypted USB drive into /etc/cryptsetup-keys.d/backups.key.

Now, both /etc/crypttab and /etc/cryptsetup-keys.d/backups.key must be persistent in the Qubes VM. This is a little tricky to get right the first time.

To persist these files in backup-vm, you need to use the bind-dirs method I described earlier. But you can take a shortcut just to test your /etc/crypttab configuration. Try creating a standalone VM. Open the Create Qubes VM tool, select “Standalone qube copied from a template” from the type menu, then select the template that backup-vm is based on. Call this temporary, test-only VM backup-standalone. Standalone VMs are entirely persistent. Anything you change or create in the /etc directory will persist across VM restarts.

This configuration probably won’t let you automatically decrypt the device on the first try. Don’t worry, this is just another step in the process. The most important thing now is persisting /etc/crypttab and /etc/cryptsetup-keys.d/backups.key in a VM across restarts. You can do that with bind-dirs on backup-vm, or you create a standalone backup VM. Persistence must work before you can test automatic decryption.

1 Like
9

on template of backup-vm or on backup-vm

update:
are you main I shoud create backup-vm as Hvm not as app-vm?

10

on dom0 terminal

  • start app-vm
qvm-start backup-vm --skip-if-running
  • attach usb partion from hdd to app-vm
qvm-block attach backup-vm sys-usb:"block device name"

on backup-vm (standalone)

  • auto decrypt block device attached to backup-vm
    step 1
sudo touch /etc/crypttab && sudo nano /etc/crypttab

and write information in file like this

# /etc/crypttab
# decrypted-device    encrypted-device    key-file
backups    /dev/xvdi    /etc/cryptsetup-keys.d/backups.key

step 2

sudo mkdir /etc/cryptsetup-keys.d && sudo touch /etc/cryptsetup-keys.d/backup.key && sudo nano /etc/cryptsetup-keys.d/backup.key

and write password in this file and save

  • after that restart backup-vm

Well I tried this but can’t get automatic decrypt device :pensive:

11

If you don’t want to mess with crypttab, you could just put something like this in your dom0 script instead:

qvm-run -p -u root <BACKUP_QUBE> 'echo <LUKS_PASSPHRASE> | cryptsetup open /dev/xvdi backup'
sleep 5
qvm-run -p -u root <BACKUP_QUBE> 'mount /dev/mapper/backup <DESIRED_DIRECTORY>'
1 Like
12

this command back with error

13

last command I use it

on dom0 terminal

 qvm-start [backupVM] --skip-if-running && qvm-block attach [backupVM] [device-name] && qvm-run --service -- [backupVM] qubes.StartApp+org.gnome.Terminal && sleep 10s && qubes-backup && qvm-shutdown [backupVM] && exit

and when terminal [backupVM] start I write y and on it and press enter

*before all step above I was make some preparation steps on [backupVM] :point_down:

on [backupVM] terminal

sudo nano .bashrc

and insert this command

#part of backup command
alias y='sudo cryptsetup open /dev/xvdi1 backup --type luks --key-file /etc/cryptsetup-keys.d/backup.key && sudo mount /dev/mapper/backup /mnt && exit'

every thing working good now

thanks friends for help me :grinning: @brian @adw

14

What is the error? Note that you have to replace things like <EXAMPLE> with your own values (including the pointy brackets).

15

error is
not found phrase
No key available with this passphrase

but from backupVM terminal everything is working