Backup device: external HDD with LUKS via USB 3.0 with external power supply
My Qubes Backup crashes randomly between 20 and 60% during the backup routine.
The crash itself shows briefly a terminal screen (like the one when you do a normal shutdown) and finally perform a Qubes shutdown.
Note: I did not check the option “turn computer off after backup is finshed”. The AppVM I use for the backup is sys-usb.
Do you think it is due to the LUKS encryption, too high CPU or RAM load?
Update: Tests with
both end with the same crash.
Hello @whoami ,
I suggest you follow the VM log and dom0 log while doing the backup. Maybe you’ll see a related message…
Yea, thanks for the suggestion. Could you please point me to the log files? Is it ok for you to direct message you the content?
in dom0 : /var/log/qubes, /var/log/xen/console/, sudo dmesg -Tw.
in the VM : sudo dmesg -Tw.
No.
I had a look into these logs but I can’t see any issue … I almost understand nothing in there which could point me to the root of these crashes.
I could perform some small backup with a smaller total backup size (just i.e. personal, vault)
Meanwhile, I noticed an additional crash using freetube running in the background for music: Boom, silence and computer shut down. Maybe it is a general memory or screensaver issue?
I would appreciate any help or further suggestion where I can look into (logs, AppVM settings etc.)
System: T430, 16GB, SSD, default QOS installation / configuration
HI @whoami ,
when you say crashing, please describe what is crashing, only the domU (AppVM)? dom0?
Which QubesOS version? maybe 4.0.4?
Which domU template? maybe fedora-33 ?
You should detect an action which do the crash (example for you: a big backup).
Then you select what to monitor (VM logs, dom0 logs, disk writing, memory usage, cpu load, …).
The origins of a crash are huge, so you should reduce the possible origins (not only by hypotheses but by facts, traces in the logs). For example, maybe you got disk writing failure logs for your external HDD…
You can follow in live the resources (tail -f <mylog>, journalctl -fxe, dmesg -Tw, top, …) and filter some events (see journalctl options, | grep <mypattern>, …). Else a post-mortem analysis is also possible, journalctl allows to access the n-1 boot logs (--boot option).
Yes it’s difficult if you don’t know all theses commands, but you will learn a lot…
as described in my original post
It shuts down my notebook so I’d name it a dom0 crash.
Yes.
Yes, fedora-33, sys-usb. (also tried other AppVMs; same crash)
That’s my actual goal here but I’ve no clue where to start debugging and what keyword I’ve to looking for. Anyway, I’m willing to learn and I’ll try to go dig into it with your log commands.
Did you have a look at journalctl in dom0?
I’m just in there with journalctl -fxe. Well, it is a huge list… Is there a way to launch a backup routine from dom0 to get more info or set some time stamps to better frame the log?
I am not very good at managing output from journalctl (maybe someone else can help). What I usually do is write the output of it into a file:
journalctl -f > file.txt
and then try to repeat the crash. After reboot, I can easily see the place of reboot in file.txt and what happened before.
I found the root of the issue: Compress the backup
I won’t say this is the solution but I’ve at least found a workaround - just finished a complete 26 GB backup by simply unchecked the compress option.
Is this a known issue?
Any further tests I can do for you to provide more feedback to the dev?
Unfortunately, backup compression has been problematic for a long time. However, I can’t find any bug reports about it causing a crash, so please feel free to open one.
Ok, thank you. I will create an issue in the coming days.