Data Sync Approach in General

Dear all,

I am interested in the general and your individual approach for data syncing with Qubes-OS. Of course, following the qubes approach, I can backup every qube which contains data.

But it is not that simple in today’s world - some of the data I want also to have on e.g.

  • my mobile device (graphene os)
  • special systems e.g. for rendering (debian, mac, win)
  • and backup purposes (truenas, debian based nas ).

Currently I am using syncthing (wan) and ssh based (lan) solutions. This allows me to differentiate which device and/or virtual system has access to which data.

This might sound more complex as it is in reality. Today I have only 9 data types. For each type I can define which device or v-system has access. So the admin part (after setting up everything) is very low.

A) How do you solve this with Qubes-OS in general?

Here is one example for discussion:
Living in the world of projects, I have one project which is allowed to have access to one data type. Currently, my non Qubes-OS approach is that this project is a virtual system and is syncing against the data on the laptop where the virtual system is running. Yes, the data are on the laptop twice (I dont mind this because there is an advantage in performance for syncing included).

B) How would you solve this example project syncing?

Sync everything to NAS and backup the NAS.
You could backup working NAS to backup NAS and as second precaution to offline backup system.
Backuping NAS could be fully automatic (ZFS snapshots syncing). Offline backup could be to DAS system or tape.

thank you for your feedback. Yes, my NAS setup covers this all.

  • The question I have is, how do you sync everything with your NAS (protocols and tools) - one config individually per qube?
  • My NAS is only accessible in the LAN, so how do you sync with your mobile on the road?
  • How do you update your laptop on the road?

Mobile I sync by Syncthing and WiFi.
Qubed OS I make backup that I copy to NAS.
Gaming PC I don’t backup.

make a VPN

use the VPN

  • to syncthing with your mobile, is this an individual cube?
  • do you exchange data between qubes?
  • do you have something like one cube for all data, where the other qubes syncing against?

BTW - why not to backup your game save points - if accessible - with syncthing?

for the laptop this would be an option now - because of the functionality of CubesOS to handle easily multiple vpn connections in parallel (thank you for your good ProtonVPN how-to).

for my mobile I am not convinced yet - because currently I am using vpn on my mobile on both profiles. using instead a vpn with my office for syncing would (to avoid switching between vpn settings) also require to use the office as internet exit point…

do you sync data between qubes on your Qubes OS system?

Syncthing backup server is on NAS (local only discovery).
Mobile is syncing to NAS.
Only one qube has access to NAS (firewall allow local connecting only).
Inter qube data syncing - standard manual file copy/move.

No, I’m using peergos to store my data so I can reach files when I need instead of storing everything everywhere. I still have a full copy in my qubes as a backup though.

And one can increase number of snapshots for said qube with important data.
Such qube could be connected to NAS and do rsync cronjob for data only.
One could edit data directly on NAS (zfs raidz5 on ssd’s and lagg x4 on sfp 10G connection).

@solene
thanks - do you know when the peergos self hosting solution will be available for testing?

@ KitsuneNoBaka
I will follow your path for the time being…

  • inter qube data syncing
  • only one qube has access to nas / syncthing etc

I understand that this is still a question where everybody has to find his own way. For new users like me a “how to sync your data” like “how to organize your qubes” would be helpful.

My final question for the moment:

Which inter qube data syncing solution (automatic) do you recommend?

Peergos reached 1.0, I self host my own. And just to make it clear, peergos uses a federated network with a PKI, my instance is federated so I can share stuff to other peergos users instead of using public links, but it’s possible to self host the entire stack.

Perfect - thank you. Looks very promising, I will use it.

Here is my first summary for other newbie’s:

  • one central qube holds all data
  • this qubes syncs with encrypted cloud service (self hosted?)
  • inter qube data exchange for all qubes which work with data
  • for device syncing use syncthing with central data qube

they are qubes, not cubes :wink:

:wink:

Just another one about peergos - do you have a nice link to a how to for a test installation (self hosting - docker - lan only)?

Do you sync with proton drive as well? Currently there is nor linux version available. In github there is a tool available but there seems to be issues to download it…

You can give a try with the demo server https://peergos-demo.net

It’s possible to run peergos locally (as a cache or instance), but it will federate with the PKI so any username created is global to the federation, if you want to be hosted by peergos.org later for instance, you would have to migrate your account from the local system to their. Just saying, it was not easy to grasp and I’ve lost my preferred username due to a quick local test :joy: