When I run sudo smartctl -x /dev/nvme0 I get some output, but this part is concerning: Data Units Written: (redacted) [2.19 TB]
I had to install Qubes 2 times (mistyped my encryption password on installer), which might explain the 2TB of usage, as (iirc) disks are wiped by random data to thwart some cryptanalyses. However, I installed Qubes just a few days ago (3ish?) and apparently it wrote 70GB in total? (I used this computer before, so some of that might have been from previous OSes, but I can’t imagine they wrote more than 120GB)
This amount of usage is atypical, but I’d explain it by the fact that I need to update many different templates.
I already disabled swap on dom0 and templates (and by extension in basically all qubes), but writes are still coming in (slower, but still atypically large amounts)
Does anybody know why Qubes writes so much?
P.S. I’m aware of this thread:
but I have no idea if I should post in that one or if that would be necroposting and that I should create a new one.
disks are wiped by random data to thwart some cryptanalyses
As you may know, SSDs never really erase data, they just mark the block and write on the next one. So, for the wiping to be effective, the whole disk must be rewritten. And even that “whole disk” is still not whole because of over-provisioning, i.e. even more writes may be necessary. All that is bad for SSD.
If the disk was encrypted, it is highly unlikely that anyone can recover data from it without the decryption key. Or at least I don’t know how that can happen. Plus, while you work you keep writing and writing new encrypted data in different places.
Does anybody know why Qubes writes so much?
By default logs and volatile volumes are written to persistent storage. Perhaps download cache during updates too. I don’t think that could easily result in 2TB of data writes though.
You may be interested in:
You can also manually create a RAM-based pool and your AppVMs through CLI, specifying the volatile volume to be on the RAM pool.
A midrange SSD (Western Digital Blue NVMe 1TB) has a TBW rating of somewhere in the 300-600 range, so “rated life” is 300-600 TB written (different spec sheets seem to have different values).
“Two installs, two overwrites of your disk” sounds very plausible to me, so unless you’re seeing a ton of write traffic after things are stable, I wouldn’t worry about it.
But swap is the likely suspect. If you’re short on RAM, memory balancing is likely to start reducing the RAM-per-VM, and that will drive swap use - which can lead to a lot of writes in a short period of time.
I had to reinstall because for some reason I mistyped the encryption password twice with the same error into the installer LOL.
So, even you don’t know the secret
you die from old age before the drive.
That very much depends on actual usage. If one works with big files (e.g. large media), 30GB a day is not so much - it is the size of a fairly small (as per today’s professional standards) SD memory card. Editing images/video uses a lot of writes. Some years ago I worked on stitching hi-res panoramas (not on Qubes) which costed > 500GB daily writes. With 800TBW, using your formula, that is 4.38 years.
I seem to be burning like 200GB per 3 days. Somewhat concerningly high, because I will also use this drive for high-writing applications like video editing like you said, and I’d like it to last more than 4 years LOL (warranty goes to 5). Is there a tutorial somewhere for reducing writes?
Is there a tutorial somewhere for reducing writes?
I don’t know. I can tell you how I used to do it: I use ramdrive for saving work while working on it. Then, when ready (or for the day), I save the result to persistent storage. Not only is this faster but also better for SSD.
The problem: you need enough RAM. The bigger the files, the more you need.
I kind of want to do this but I’m scared to OOM and explode all my data. Is there a tutorial for that? Or can you point me to certain commands I can use?
Arguably they also die, they just don’t have a “TBW” rating on them.
I kind of want to do this but I’m scared to OOM and explode all my data.
You have to be careful when setting up your ramdrive. Don’t size it bigger than available RAM and always have some reserve, also don’t fill it to the top. Consider the size of the file you are working on and have a monitor for free space.
Is there a tutorial for that? Or can you point me to certain commands I can use?
Well, that’s not Qubes specific. It is basic file save operation on a tmpfs mount.
Arguably they also die, they just don’t have a “TBW” rating on them.
Unarguably, everything dies one day. Even RAM. The point here is, HDDs don’t wear as a result of writes per se.
Another option is to buy a separate SSD and use it as a scratch disk. Then make enough money from videos, so you don’t care when it dies
I thought you were talking about saving logs and stuff which will get deleted anyway to the tmpfs. Is there a tutorial to do that? You said (way earlier):
Won’t the motor burn out or something at some point?
I thought you were talking about saving logs and stuff which will get deleted anyway to the tmpfs. Is there a tutorial to do that? You said (way earlier):
Outside certain “terrible” drives from the early 2000s (DeathStars, commonly sold as DeskStar IIRC, that weren’t rated for more than 18h/day operation), the motor just spins, and for longevity, “letting it spin” is better than constantly starting/stopping it. Easier on everything, and less thermal stress.
Hard drive death is independent of amounts of data written. A “spinning rust” style drive is storing data magnetically on a ferromagnetic surface, and there’s no practical limit to how often you can do that (I’m sure there’s some theoretical limit, somewhere, but it doesn’t impact practical storage use). You can just rewrite a hard drive constantly and not bother it.
SSDs store data by “shoving electrons through an insulator” into a little electron container. The problem with them is that the insulator wears out over time from the abuse of electrons being shoved through it, so the containers eventually start to leak - which means data loss. It’s worse on cheaper SSDs because they’re storing three or four bits per container based on the charge level, so they’re going to “practically wear out” sooner than a flash device only storing one bit per container (“Charged or not” is easier to figure out than “Which of 8 or 16 levels does the charge in this container correspond to?”).
Oh, TIL about the letting it spin thing. Also I thought the bottleneck was the magnetic endurance, good to know I won’t need to worry about another thing (lol).
Thanks for your explanations, they were very informative!
Gaming is mostly reading, only updates are writes. I want to save writes so I can do heavy writing as a user, not have the system burn all my writes before I can even use the SSD for what I wanted to use it for (even if it’s somewhat in the future)
I hope this doesn’t have security implications though.