Script to automate the backup of the data inside a qube

General Discussion
1

Continuing the discussion from How do you organize your backups?:

@unman Do you have a script that automates that whole process ? (and any update in your setup since your last message in 2023?)

As a new user, one of the things that’s scaring me in transitioning to QubesOS is the lack of incremental backup.

I have read a number of related posts on this forum. There seems to have been quite some effort around wyng, however after reading the most recent threads (e.g. Wyng for Noobs) it is (from my perspective) far from user-friendly.

I have seen you advocate instead in several threads for an approach that backs up the data inside the qubes directly, instead of the qubes themselves, which I find attractive. If I understand correctly, part of your reasoning is that the qubes creation process on your end is mostly automated and reproducible via salt.
(Honestly, as a new user salt in QubesOs feels very unintuitive, and the docs are very minimal (imo), but I can always use bash scripts in dom0 to recreate the same outcome.)

I find your argument compelling - but I’m not quite sure how you’re implementing this in practice, and am guessing you have a script to automate most of the steps.

What I had in mind was to conceptually:

  • Plug a usb device and attach it to a disposable AppVM “BackupVM” Bk
  • For each Qube Q to backup:
    • “Mount Q as read-only in Bk” (whatever that means - somehow make Q’s files of interest readable by Bk)
    • Use a backup tool (Kopia in my case, that’s what I’m used to) to backup those files to the usb device, in a folder named after the VM being backed up (=> how to handle renames, is there a unique ID ?)

Would you have any input/advice in that regard ?

2

I could not find a satisfactory Qubes OS file‑backup solution, so I wrote my own script:
My Script and a short Video to see how it works.

1 Like
3

I’ve already made clear my views on Wyng, and why it is not a backup
tool suitable for many use cases.

I do have a script that automates the process, and it does almost
exactly match your concept.

The piece you may be missing is that you can attach private storage to
another qube as detailed here:

Note that while the qube is running you may want to back up both the
private and private-snap volumes. It’s essential to attach these and
mount them ro, for obvious reasons.

I use zpaqfranz, but you can use whatever tool you are familiar with.

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

1 Like
4

Thank you for your reply.

I attempted to build a backup script based on those indications (although frankly my understanding of everything I’m doing is weak - I’m mostly piecing together disparate pieces of docs and forum posts)

#!/bin/bash
set -eux

vms=("name-of-vm")  # to backup
backup_vm="Kopia"  # TODO make disposable, but for POC is ok. Assumed running.
vault="vault-kopia"  # where to store the backup (another VM)

# Attach Vault
VAULT_PATH="/dev/qubes_dom0/vm-${vault}-private"
VAULT_DEV=$(basename $(readlink $VAULT_PATH))
qubesdb-write /qubes-block-devices/$VAULT_DEV/desc "Backup-$vault"
qvm-block attach "$backup_vm" "dom0:$VAULT_DEV"

for vm_name in "${vms[@]}"; do

echo "Start backup for $vm_name"

# Attach VM to backupvm
SOURCE_PATH="/dev/qubes_dom0/vm-${vm_name}-private"
SOURCE_DEV=$(basename $(readlink $SOURCE_PATH))
qubesdb-write /qubes-block-devices/$SOURCE_DEV/desc "Backup-$vm_name"
qvm-block attach --ro "$backup_vm" "dom0:$SOURCE_DEV"

# Run on backup VM
set +e
qvm-run -p -u root "$backup_vm" bash << EOF
set -eux
mkdir -p /mnt/source /mnt/vault
mountpoint -q /mnt/vault || mount /dev/xvdi /mnt/vault
mount -o ro /dev/xvdj /mnt/source

kopia repository connect filesystem --path=/mnt/vault/home/user/kopia --password "0"
kopia snapshot create /mnt/source

umount /mnt/source /mnt/vault
EOF
set -e

# Cleanup
qvm-block detach "$backup_vm" "dom0:$SOURCE_DEV"
qubesdb-rm /qubes-block-devices/$SOURCE_DEV

echo "End backup for $vm_name"

done

qvm-block detach "$backup_vm" "dom0:$VAULT_DEV"
qubesdb-rm /qubes-block-devices/$VAULT_DEV

echo 'OK'

This script barely works - more precisely, it will likely work once after reboot, but I can’t answer for any later run.

In particular, there’s several things I am very unsure of (which likely reflect my lack of understanding of what I am doing exactly with the mounting)

  • Do we need the backed-up VM to be shut down ? If not, what were to hypothetically happen if we modified /mnt/source (assuming no --ro) while the VM is running ?
  • Although we qubesdb-rm /qubes-block-devices/..., is it normal that qvm-block list still shows the qubesdb-write-created entries afterwards ? I feel like my cleanup is not working properly, even though I think I am following the docs properly. I also feel like this is why any run past the first one leads to issues like can't connect to storage: cannot access storage path: stat /mnt/vault/home/user/kopia: input/output error.

Thank you.

5

Edit: (as separate message because iirc @unman can’t see edits)

Concerning my second point I think I found the issue: I am missing a “/” at the end of qubesdb-rm:

qubesdb-rm /qubes-block-devices/$VAULT_DEV > qubesdb-rm /qubes-block-devices/$VAULT_DEV/

I’d still be happy to receive feedback on the rest.

6

This is good as poc, and I suggest only minor improvements. You might
like to make the backup and cleanup functions, so that you can call them
as needed.

Definitely use a disposable - I use a named disposable for the
purpose, to make things easier.

I run the mount and backup calls directly from mgmt, but I see the
benefit in the script approach. Make the mount call use ro,noload.

You need some error checking to make sure that xvdi and xvdj are
what you want them to be before taking any actions. If not, roll back
immediately.

This will work fine, whether the source is running or not. It’s
essential that the qvm-block attack and mount calls are ro/ro,noload.
If the source is running, then you will only get a backup from when the
source was started. If you want to run live backup then use the
private and private-snap volumes: the backup tool must be able to
cope with changing files. (I used to use snapshots here, but now I do
not. I run the live backups hourly in some systems with no issues.
YMMV.)

qvm-block and the device GUI both suffer from ghost entries after this
process. It’s annoying, and a bug that I should report. But it looks to
me as if you are doing the write thing. I also remove the description
and then remove the device in separate calls.

Make sure that you are running this against test data before you run
against your working qubes. At the moment it looks good to me.

I never presume to speak for the Qubes team.
When I comment in the Forum I speak for myself.

1 Like