Don’t home folders represent practically all data in App qubes anyway? When you backup an App qube, the overhead is tiny, since it has no /root etc included.
1 Like
ideally, you could write a script to install everything in your template, you would only have to backup these few scripts
there is qvm-backup command line, which uses the same library as the backup GUI tool, you can totally script it
as @fslover pointed out, the storage in appvm is only the home directory + a few things in /rw/ that are really small
3 Likes
the inherent trade-off between security and convenience, […]
If it was inherent, it would apply to all cases. The fact is it does not.
Simple proofs:
- Does HTTPS make web browsing less convenient than HTTP? - No.
- Does having a good UI prevent user from mistakes which can cost data loss, thus improving security? - Yes.
- I also gave a very specific example about backup.
So, there is no inherent trade-off in the sense of some inevitable omnipotent principle. There are only case-specific trade-offs.
If the software does not provide the functionality the user needs, he either will look for alternatives, which indeed may be less secure, or not backup at all - also less secure. So, by being “more secure” (the justification of less functional/usable) it inherently leads to less security.
2 Likes
this would probably be hard for someone like me
i am a low skill qubes user. if there were a community guide to do this, i would implement it.
if i scripted a backup for non-templates, would i install cronie in dom0?
could i script a cronjob to open up each template, run dpkg list copy the contents to a file in some format that could be read along with the template name, copy any additional repositories, copy the settings for the VM, and save it to a external backup drive folder as well? this would need to happen without all template opening at once.
if there could be a standard database that could be read by another script it would make restoring templates easier and backups most likely
it would be such a useful feature, it’s unfortunate there’s not a command or gui for doing this. qubes crashes at times. we who use qubes often all know it and have been there. even a community guide would be so good
i have the skill to setup a cronjob but not skill to automate template program listings to database that could be restored easily. without the templates configured correctly, if have a full crash then restoring takes much longer. without script for copying template information backup must be very large and so will be infrequent
1 Like
2 Likes
Don’t home folders represent practically all data in App qubes anyway? When you backup an App qube, the overhead is tiny, since it has no
/rootetc included.
The question is does one really need to backup all data or just important data.
For example, some programs store a huge amount of cache in subdirs of home just to speed up their work. Loosing that cache is fine, as the program will recreate it automatically. Copying it during every backup is not tiny and it can be much bigger than essential data.
2 Likes
This is parallel to my situation in an unexpected way. I explained above how I back up only AppVMs (and there are few of them), however one AppVM I don’t bother with is sys-cacher, for the same reason: It will regenerate the next time I do updates anyway. And yes, it’s fairly big as AppVMs go, at least on my system.
Thanks @kenosen.
2 Likes
I don’t bother with is sys-cacher, for the same reason: It will regenerate the next time I do updates anyway.
That’s easy.
What I meant though was that in the same AppVM which stores important data, there can be unimportant cache much bigger than the important data.
Some mail clients, for example, can cache locally your whole IMAP folder structure which exists on the mail server. That cache can be a few GB. What you need to backup though is just the messages that exist only locally (e.g. POP accounts) which can be 100 times less data.
2 Likes
It does, but you have to organize your qubes such that a set of qubes contains all and only your important data (or nearly so), then you can back up only that set of qubes when you want a smaller backup.
In general, no such data should exist.
That’s why it’s recommended to keep a list of changes you make to your templates. A text file is tiny compared to templates.
An alternative is to use Salt or scripts. That’s much more technically challenging, but the same principle applies.
That’s not true. In fact, it’s basically the opposite: The home directory is the main thing backed up from normal app qubes, and almost everything else is left out, except for some other specific directories. Just doing a default backup already achieves what you want.
Btw, in case it’s not obvious, Qubes has no way of knowing which data you consider crucial (neither does any OS, for that matter), so you have to tell it. One way to do this is to put crucial data in one set of qubes and non-crucial data in a different set of qubes, then back up only the former set when you want a backup of only the crucial data.
Again, not true.
I’m not sure what you’re asking for here. What kind of database are you envisioning?
Why would a crash require restoring templates (or anything, for that matter)? If a system crash is somehow wiping out your installation, that might be a symptom of a deeper problem.
2 Likes
Because what is the worth of a properly backed up qube that will not work when restored under 4.2, and even 4.3, we now know? Or, it doesn’t need to be brought up to attention? If I knew that, even that I asked for it at the time of upgrading, I wouldn’t actually back up and upgrade to v4.2, but no one answered.
It’s like “properly backing up/removing” the car hood in order to install new engine in the car, (not) knowing that it won’t be properly restored because the engine is now oversized.
2 Likes
As long as we’re on the subject of backups, I will air my (admittedly petty) gripe.
On the command line, if I back up one qube, the thing insists on listing all of the qubes I am not backing up. (How many commands out there respond by telling you what you didn’t ask it to do?) If, later, I want to see what I actually did back up (e.g., maybe I want to back up something different now and want to make sure I’m not repeating things), I have to scroll back through a useless list of dozens of qubes. There’s a “quiet” option on the command line that seems to have no effect (unless it has been changed in the last few months), perhaps it could be used to shut this worthless output off?
1 Like
I really like Qubes and wish I had more time to use it.
Keeping a list of each program in each template is another thing I would have to keep track of.
It’s easier for me to just do biannual backups but I wish there were an easier way.
I am not technically advanced user of Qubes. I am likely among least technically sophisticated Qubes users on this forum.
2 Likes
if you install programs in templates using the command line, you could just make a script appending the command lines in it.
1 Like
I’ll split this into sections for hopefully easier reading/for you to skip a part you’re not here for 
1. People lack motivation to make backups
What it comes down to 9 times out of 10 is motivation vs. perceived difficulty. Everyone “knows” they should be making backups. My mom, my partner, my digital artist friend, and others in my life, all “know”. They own an external hard drive, they’ve made backups in the past, and when the topic comes up they’ll say “oh, yeah it’s been a year or two… I’m really bad about that haha”. My mom has important business records on her PC, and yet only makes backups when I hound her into doing it. My partner will drop their phone on the floor for the second time in a day, I’ll say “I hope you have that backed up”, and they’ll remark that it’s hard to make it a habit, or just chuckle nervously.
After all, what’s the difference between backing up today vs. tomorrow? Not much at all really. The chances you’ll have a catastrophic data loss in the next 24 hours are extremely small. And you have 10 other things that need doing urgently! So surely you can put it off just a little, while you work on something more pressing.
It’s really easy to make this justification to yourself today… and tomorrow… and any other day you happen to think about it.
2. Why don’t I backup?
You’d think I could practice what I preach, but no. 
I adopted Qubes OS in February of this year, and haven’t once made a backup. I will, that’s the plan anyway, but I’m STILL in the middle of configuring my system. Formulating a backup plan is sitting about 10 more points down on my system config todo list. TBH in setting my priorities I just kind of made a huge assumption that backups would be non-trivial for me, given that I have a pretty non-standard system configuration (like root on ZFS), and will likely want a custom solution. I haven’t so much as opened the backup tool to look at it yet.
But also TBH I had no idea this is where I’d be at six months after moving to Qubes, and one full year after I started planning my build lmao.
3.1. How to motivate people (with anxiety)
Motivation is an emotional issue so convincing/teaching people to be motivated isn’t effective imo. What works is anxiety. People know there’s a potential to lose data, but they need to feel it too, feel in their bones. They need to backup TODAY, not tomorrow, because they’re not only doing it for that imperceptible bump to their data’s durability, but also to calm their nerves. One hard loss is enough to motivate most people into forming a backup regimen. We’d like them to reach that same emotional understanding without having to learn the hard way, and that’s the tricky part.
There are a lot of vectors but, unfortunately as an OS, there’s only so much you can do without being overbearing. But reminders certainly help.
Good: “Remember to back up your qubes regularly”
Better: Using scary language “Your data is unsafe! You haven’t made any recent backups!”
Best: Get people to actually think about what they stand to lose “Take a minute to ask yourself: Which files on your PC are most important to your life? Which have the most sentimental value? If your computer died a horrible death tomorrow, do you have a backup of them safe somewhere?”
3.2. How to enable people (abate perceived difficulty)
The other side of the equation is the perceived difficulty/effort of making backups. And at no time is the perceived difficulty higher than the first time. Separately, at some point in the Qubes adoption process, it’d be good if users were greeted with a friendly super simple infographic showing 3 easy steps for backing up a qube. (Or something like that-- I haven’t used the backup tool so idk exactly lmao). The more it looks like it was made for babies to understand, while still being assuring the backup will be secure, the less resistance there’ll be for the user’s puny motivation to overcome. I suspect the reason friendly/minimalist infographics are popular on landing pages of modern websites is because they’re effective at making products/systems/processes seem more approachable.
A good time to do that might be on the login screen during the 2nd week of use-- late enough that the user is not suffering information overload from running a new OS so they’ll actually take notice, and also late enough that they might have something worth backing up.
Btw I have to say @adw you went through point-by-point dispelling many of the things brought up by @qubalker and @dispuser , but I encourage you to consider how insightful their posts are. These problems they’re talking about might have simple solutions, or be exaggerated or imagined, but this all demonstrates exactly how perceived difficulty can be a massive hurdle to some users. Most people aren’t going to be on a forum that is feeding them the easy answers to their concerns (even then, some people will want to independently verify an answer given to them, which takes effort too). In qubalker’s case I think the questions were even meant somewhat rhetorically; that is, to an extent, part of the perceived difficulty is in not knowing what you don’t know. Some users won’t even know all the questions they should be asking, and these are just some of the things that could flood their mind.
Of course perceived difficulty is partly downstream from actual difficulty, so anything that can be reasonably done to make backups actually easier is a plus. That is our secondary concern imo.
4. On lists of packages installed in templates
replying to:
I’ve started keeping notes of all system configuration/setup I do, and I’ve found it incredibly helpful. Highly recommended! And yes I keep a list of all the packages I’ve installed in my templates too. No time like the present to start. I wouldn’t worry too much if you forget a package. If you ever have to restore your system, it’ll usually become apparent when you go to use a program and the app or command is missing, and you can add it then.
Btw for debian/apt you can get a fairly legible list of things you’ve installed with:
zcat /var/log/apt/history.log.*.gz | cat - /var/log/apt/history.log | grep ^Commandline:
There’s probably something analogous to that for fedora/dnf but I’m not as familiar with that system.
5 Likes
another thing that would improve quality of life:
when backing up, the backup program should offer to also verify the backups (instead of me opening the backup manaer a second time, punching in the password and ticking the verify box, selecting what i want to restore and…)
3 Likes
It sounds like that one aspect could be improved. However it might be the case that inputting the password a second time is the minimum possible amount of user interaction for verification. That could be to lower the possibility of user error with the password or, if for security the password should not persist in memory during backup, to receive it again to make sure it can correctly derive the encryption key.
Quick comment on something mentioned earlier in the thread too, about automating backups. Backups cannot be fully automated! It would be a disaster waiting to happen, if the software were to silently fail for any reason, the user may not notice for months that no backups have taken place. Always-connected solutions are not optimal either, even if it’s to a cloud. If you have keys to backup to a server, in most cases those same keys can be used to destroy the backups by accident or when your system is compromised by a disk encryption ransom attack, so be careful. And no matter what you do there needs to be some involvement by the user for verification, or there is no verification-- even if it is as simple as clicking an OK box.
1 Like
you enter the password twice upon setting up the backup
and i am not sure but i guess when packing the backup file the password remains in ram while doing this, keeping it longer for a requested verification of just created backup would be acceptable to me.
1 Like
Speaking of backups. The Transcend m.2 2240 on Elitebook 820 G1 development machine broke. And it is not the controller. It has worn out (too many templates & keeping them updated do wear-out NAND). My last backup is from 30th July. Nothing serious on this machine. It is primarily used for Qubes OS developments and tests…
4 Likes
It still has 17% free unused spare cells, but I think I never sad a SSD drive used that much 
Could you calculate how much writes this is? (maybe it’s indicated somewhere)
3 Likes