As far as I can tell, the next step needed is developer review, which is already reflected on the issue with the “S: needs review” label.
1 Like
After a week of battle testing it, it works well. Deduplication works fine even with a remote repository, it’s incredibly fast 
For the trust issue, I wonder if we could use restic in dom0 but make the network access through a qube. I need to think about a way 
3 Likes
Probably, but then you’d have something in dom0 speaking (what I’m going to assume is) a complex bidirectional application-level protocol. Connecting to the potentially malicious side through qrexec - instead of through the TCP/IP stack - only takes care of the lower-level part of the connection.
1 Like
Something based on borg may work better because it has a sender, locally compressing and encrypting files, and a receiver doing stuff with the backend.
The send could be used from dom0 and the receiver could be in an appvm.
dom0 could still be attacked through a vulnerability in borgbackup, but the backup itself could be verified.
Restic is more limited as it does everything with a single process
1 Like
I restate that efforts should be made into testing and upstreaming wyng and wyng-tools-qubes so that official backup system switches to wyng as soon as possible…
Here, my “special” configured volumes, specially configured to include volumes to be backup in manual runs, outside of the wyng-util-qubes that I run as often as noted.
Also note that calling wyng-backups manually could include even “volatile” volumes, which in qubesos backend terminology means the live lvm you are actually working on, and where wyng-util-qubes run as official backup system does, meaning on the volumes as seen on last qubes shutdown:
(130)$ time sudo wyng --dest=qubes://wyng-wdcloud/home/user/nv41 --local qubes_dom0/vm-pool --dedup send --all
Wyng 0.8wip release 20240415
Encrypted archive 'qubes://wyng-wdcloud/home/user/nv41'
Last updated 2024-04-23 12:13:05.568645 (-04:00)
Preparing snapshots in '/dev/qubes_dom0/'...
Warning: Local '/dev/qubes_dom0/wyng-qubes-metadata' does not exist!
Acquiring deltas.
Sending backup session 20240423-121405:
———————————————————————————————————————————————————————————
[...redacted for privacy reason...]
[...]
0.5MB | 4s | vm-deb12_no_gnat-heads-docker-private
552.5MB | 21s | vm-deb12_no_gnat-heads-docker-root
no change | - | vm-deb12_with_gnat-heads-docker-private
no change | - | vm-deb12_with_gnat-heads-docker-root
no change | - | vm-debian-11-private
no change | - | vm-debian-11-root
no change | - | vm-debian-12-extrepo-heads-private
no change | - | vm-debian-12-extrepo-heads-root
no change | - | vm-debian-12-minimal-private
no change | - | vm-debian-12-minimal-root
0.0MB | 4s | vm-debian-12-xfce-private
29.5MB | 5s | vm-debian-12-xfce-root
no change | - | vm-debian-12-xfce-vpn-private
no change | - | vm-debian-12-xfce-vpn-root
no change | - | vm-debian-luban3d-private
no change | - | vm-debian-luban3d-root
no change | - | vm-default-dvm-private
no change | - | vm-dev-private
no change | - | vm-dvm-browser-private
no change | - | vm-dvm-debian-minimal-private
no change | - | vm-dvm-dev-private
no change | - | vm-dvm-fedora-minimal-private
no change | - | vm-dvm-fedora-private
no change | - | vm-dvm-fetcher-private
no change | - | vm-dvm-media-private
no change | - | vm-dvm-mgmt-private
no change | - | vm-dvm-qubes-builder-private
no change | - | vm-dvm-reader-private
no change | - | vm-dvm-sys-audio-private
no change | - | vm-f37-builder-private
no change | - | vm-f37-builder-root
no change | - | vm-fedora-37-private
no change | - | vm-fedora-37-root
no change | - | vm-fedora-37-xfce-printer-private
no change | - | vm-fedora-37-xfce-printer-root
no change | - | vm-fedora-38-xfce-private
no change | - | vm-fedora-38-xfce-root
no change | - | vm-fedora-39-minimal-private
no change | - | vm-fedora-39-minimal-root
no change | - | vm-fedora-39-private
no change | - | vm-fedora-39-root
no change | - | vm-heads-tests-deb11-private
no change | - | vm-heads-tests-deb12-clean-private
no change | - | vm-heads-tests-deb12-nix-private
no change | - | vm-heads-tests-deb12-private
[...]
no change | - | vm-luban-private
no change | - | vm-media-private
no change | - | vm-nk3-private
[...]
no change | - | vm-qubes-builder-private
no change | - | vm-qubes-builder1-private
no change | - | vm-qusal-private
no change | - | vm-spotify-private
no change | - | vm-sys-cacher-browser-private
no change | - | vm-sys-cacher-private
no change | - | vm-sys-git-private
no change | - | vm-sys-pgp-private
no change | - | vm-sys-print-private
no change | - | vm-sys-syncthing-browser-private
no change | - | vm-sys-syncthing-private
2.7MB | 4s | vm-sys-whonix-private
[...]
no change | - | vm-testing-conf-private
no change | - | vm-tpl-browser-private
no change | - | vm-tpl-browser-root
no change | - | vm-tpl-dev-private
no change | - | vm-tpl-dev-root
no change | - | vm-tpl-fetcher-private
no change | - | vm-tpl-fetcher-root
no change | - | vm-tpl-media-private
no change | - | vm-tpl-media-root
no change | - | vm-tpl-mgmt-private
no change | - | vm-tpl-mgmt-root
no change | - | vm-tpl-qubes-builder-private
no change | - | vm-tpl-qubes-builder-root
no change | - | vm-tpl-reader-private
no change | - | vm-tpl-reader-root
no change | - | vm-tpl-sys-audio-private
no change | - | vm-tpl-sys-audio-root
no change | - | vm-tpl-sys-cacher-private
no change | - | vm-tpl-sys-cacher-root
no change | - | vm-tpl-sys-git-private
no change | - | vm-tpl-sys-git-root
no change | - | vm-tpl-sys-pgp-private
no change | - | vm-tpl-sys-pgp-root
no change | - | vm-tpl-sys-syncthing-private
no change | - | vm-tpl-sys-syncthing-root
no change | - | vm-untrusted-private
no change | - | vm-vault-private
no change | - | vm-vpub6-conf-private
no change | - | vm-whonix-gateway-17-private
no change | - | vm-whonix-gateway-17-root
no change | - | vm-whonix-workstation-17-dvm-private
no change | - | vm-whonix-workstation-17-private
no change | - | vm-whonix-workstation-17-root
no change | - | vm-work-private
[...]———————————————————————————————————————————————————————————
[... volume number doesn't match because redacted...]
122 volumes, 2052116——>585 MB in 272.5 seconds.
real 4m35.081s
user 1m50.656s
sys 1m29.784s
How can anything beat that?!?!
With wyng-util-qubes helper (fully redacted, no diff)
Note here that -i means taking all qubes manager qubes/templates tagged to be included in backups:
$ time sudo wyng-util-qubes --dest=qubes://wyng-wdcloud/home/user/nv41 --pool qubes_dom0/vm-pool backup -i --authmin 720 --dedup
wyng-util-qubes v0.9wip rel 20240415
Skipping dom0
Wyng 0.8wip release 20240415
Encrypted archive 'qubes://wyng-wdcloud/home/user/nv41'
Last updated 2024-04-23 12:17:49.241332 (-04:00)
Preparing snapshots in '/dev/qubes_dom0/'...
Queuing full scan of import 'wyng-qubes-metadata'
Acquiring deltas.
Sending backup session 20240423-122518:
————————————————————————————————————————————————————————
[...]
0.0MB | 1s | wyng-qubes-metadata
————————————————————————————————————————————————————————
42 volumes, 1248276——>0 MB in 107.4 seconds.
real 1m56.495s
user 0m42.645s
sys 0m28.375s
Please test wyng-backups/wyng-util-qubes as much as possible and focus on replacing/extending official backup system with something being 2024 and competitng and beating up any other backup tools not carved for qubesos…
5 Likes
Is it able to restore qubes with their settings?
…yes. IF wyng-util-qubes used, which is the wrapper meant to eventually be wrapped by qubes os
this is
part, taking snapshot of qubes.xml and keeping reversion at each backup run from the tool, and used when restoring a session.
@solene : you;re exactly the kind of person that could test it, write about it and make things go forward because of your technical writer/end user directed documentation easiness. Please do, and the whole QubesOS ecosystem will thank you for it forever 
Oh, and also read about --sparse-write and --sparse from official docs, @solene !!! If I got your technical background enough, you will be amazed.
The beauty of this magical tool is that if a user clones a qube/template to test reverted changes upon a specific volume, --sparse-write (usage encouraged if backup archive is local as oppsed to --sparse if remote) will only write the changes on that volume instead of writing all the backup archive to the clone.
What this means, technically, is that the snapshot volume will actually only “grow” by the size of the diff of the archive vs occupying the whole archived volume space… which qubes-backup simply cannot do and won’t do. Exploding disk requirements upon restore and requiring users to choose carefully what to restore when they do. It shouldn’t have to be like that, neither on backup nor backup restoration.
This is pure magic, really. Just needs to be challenged on the documentation side and made for general usage now. Not just used by technical people.
1 Like
There is a restore option that you can uncheck which lets you restore unencrypted backups. Could this help in this case?
no because you need to make an not encrypted backup first which is not possible with the current tooling.
It seems it wasn’t always the case, so maybe the restore button to uncheck encryption is for retro-compatibility purpose.