How to use Qubes OS solidly?

I only use Qubes OS to work now and I can’t miss it. How to make it work solidly?

  • Backup before every change but this will consume a lot time. Rsync can make it faster but you also need to shutdown VMs to backup so I think this can’ be totally automatically.
  • Backup change that you think may break up the system. I backup before upgrading to 4.1 which I think is a big change. But you may not predict it, recently a normal debian 11 upgrade made me can’t work.
  • Duplicate essential VMs, upgrade one VM first, if there is any problem, you can still use the other VM if dom0 doesn’t have any problem.
  • Prepare two computers, upgrade one computer first, if there is any problem, you can still use the other computer

I have …

  • two identical laptops
  • bash scripts that automatically create all my templates
  • bash script that gathers all configuration files in [dom0] /etc that I have touched and copies them into the [dom0] /home before starting the backup
  • full compressed and encrypted backup of everything to an external 2 TB drive … every night while I sleep; in my case that’s ~10-12 days of backups
  • restore (–verify) of the most recent backup every morning in the background, while I catch up on emails and are in meetings
  • every 2-3 months I switch from one laptop to the other (restore all qubes and [dom0] /home takes ~3 hours)
4 Likes

Are those the security of your’s, Sven?

Yes, that’s what I do. It sounds like a lot but it’s less than a minute of my time every day. Once you have a setup like this, it’s not much effort to keep it going.

Yes, it wears out the external drive. I don’t care. When it starts failing I’ll through it away and get a new one. Worth it.

4 Likes

@Sven, I feel like this could be a very successful qubes-contrib-repo package for those who want to be able to do what you do, because honestly, it sounds pretty awesome.

qubes-sven-decoy or something like that.

There’d be a healthy amount of libsven in it.

1 Like

Are you doing any RAID setup for your backup drives?

Don’t you shutdown VMs when backup? This will backup running state. It will waste a lot time if you shutdown and then start and recover your session.

1 Like

@alzer89 there isn’t any code I could contribute. When it comes to the scripts that build my templates, they are specific to my use case and I have made clear in other threads why I’d rather teach people to create their own scripts then publish mine in a readily usable form. I do share which packages are needed for what on my website (work in progress).

The script that copies files from /etc to /home is also highly individual. In my case I copy:

/etc/environment
/etc/X11/Xresources
/etc/X11/xorg.conf.d/99-synaptics-overrides.conf
/etc/thinkfan.conf
/etc/lightdm/lightd*.conf
/etc/qubes-rpc/policy/qubes.*
/etc/qubes/policy.d/*

… into a directory in my /home.

@Rhys-Hussain yes I shutdown all qubes before backup. All I have to do is to make sure the backup drive is mounted in sys-usb and then I run the script in dom0 that shuts down all qubes (except sys-usb which is disposable anyway), copies the above files and then starts qvm-backup. I don’t have to sit in front of the computer for that.

In the morning I start my qubes again. This too is automated. I have a script that starts all the qubes/apps I am normally using and devilspie2 makes sure they show on the workspace and in the position I want them too. You will see that e.g. Chrome simply asks whether you want to restore the previous session (of course not in disposables).

So I wake up and run the script to restore my working environment while I make tea. Once I am back at my desk I start the verify/restore and dive into emails/feeds etc.

These bash and devilspie2 scripts are specific to my machine and environment and hold little value to others.

2 Likes

@Sven, that’s more than enough to get the community started :sunglasses:

2 Likes

These are good tips on migrating one’s current Qubes setup to another Qubes install. Thanks for spelling these out.

1 Like

No RAID, but i verify every backup.

1 Like

I’m also interested in automating some setup for dom0 and certain qubes. My use case is similar to Sven’s but with a twist: I keep my backups on a NAS. Thus in order to be able to restore my qubes, I need a working qubes installation including sys-net, sys-firewall and the templates these are based on. When migrating from one machine to the next, this has resulted in the original $distro template for my personal AppVM to be restored to $distro-personal because $distro already existed as template for the freshly installed sys-net. It would be unwise, it seems, to restore an AppVM without also restoring its corresponding template. Is that correct?

I have the impression that settings in dom0 are less coveniently to restore than other VMs. In particular launchers and the panel setup for xfce4. So anything Sven has worked out easing this would be helpful.
I also backup my sys-firewall customizations to a /home/user folder in an AppVM so I can more easily restore these into a freshly installed sys-firewall. I’d be intersted, how @Sven would solve this when you can’t create sys-firewall from dom0 because the custom dom0 can not be created without sys-firewall.

To make things worse, the $distro template is now a different version than the current 4.1 $distro template, so I ended up maintaining two major versions of templates for the same distro. On the other hand, having a separate $distro-personal template has advantages since other AppVMs don’t see the apps I installed in that template. Or should I move to standalone VMs, as these are easier to restore?

$distro already existed

This is one of the main reasons why I never use the original templates that get installed by Qubes OS, but always clone them. Another good reason would be to always being able to start from a clean, unmodified baseline by making a clone of the Qubes OS provided template.

settings in dom0

It makes sense to restore them the way they are… so you are able to pick and choose what to restore. Otherwise, if you messed something up in dom0 your only options would be to restore the mistake or start from scratch.

launchers and the panel setup for xfce4

At login screen user Ctrl+Alt+F2 to go to a text terminal. Login. Nuke or rename the ~/.config directory and then move the restored .config folder into your ~/ home. Logout. Ctrl+Alt+F1 and login with XFCE. :slight_smile:

I also backup my sys-firewall customizations to a /home/user folder in an AppVM so I can more easily restore these into a freshly installed sys-firewall. I’d be intersted, how @Sven would solve this when you can’t create sys-firewall from dom0 because the custom dom0 can not be created without sys-firewall.

I am not entirely sure what you are asking. In my case modifications to sys-firewall are either in the respective template (e.g. apt-cacher-ng package) or they are documented in a bash script in dom0 I can use to recreate my sys-firewall from scratch (along with all other system qubes).

After 5 years of Qubes OS usage this would be one of my main recommendations: document all of your modifications regardless of how “unimportant” they are. Either write them down in a notebook/text file or better automate them in a bash script or salt recipe if you can.

Following this one guideline takes most of the sting out of restoring/upgrading/switching template distros.

$distro template is now a different version than the current 4.1 $distro template

That is a good thing, see above.

so I ended up maintaining two major versions of templates for the same distro.

Don’t use or modify the Qubes OS provided template. You can even remove it after cloning if you are tight on space. The key here is to always be able to go back to a clean version and your template name not being in the way when installing/restoring the Qubes OS provided version of the template.

standalone VMs, as these are easier to restore?

And then you have to update/maintain each qube separately? With it’s root partition keeping state and loosing a major advantage of Qubes OS architecture?

:slight_smile:

Standalone VMs are useful for specific use cases, but by no means should they be your default option.

Thanks a lot, Sven. I shall remember at least two lessons:

  1. Changes to any of the standard VMs e.g. (sys-firewall) should be made in a programmatic way so that they can be re-played on another fresh install.
  2. The standard templates should only serve as templates for dispVMs and the system qubes. Any personal app VMs should be based on clones.

As to (1), I wonder since the filesystems are copy-on-write, shouldn’t it be possible to obtain a diff of e.g. the pristine sys-firewall (i.e. the template it is based on) and the current state as a tar-ball? If so, could that be made into a Qubes tool?

1 Like