Implementation of `qvm-sync`

Before this discourse existed, I emailed about this on the qubes-devel mailing list, but I thought I’d post it here too.
I implemented qvm-sync . Code is here. I have been using it in my day-to-day work for the last 6 months or so without finding any more kinks to sort out.

The usecase that I wanted to solve, is that I have one VM for development (work). I have another qube for building docker images (dockervm). From time to time, I want to sync a folder (a full github repo) from work to dockervm to build it. I found that qvm-copy was a clunky tool for this purpose, as I had to manually delete the old folder and copy everything, even if the changes were minimal.

qvm-sync instead operates a bit like rsync. It also always operates on exactly one folder (not a list of files). It sends over a filelist + crc32 cheksums, and the remote side updates the folder by deleting files that are no longer present and requesting the files that have been changed. It also uses snappy compression (optional) during transmission.

However, I decided to implement it in golang, which turned out to be a bit problematic with regards to threading: golang uses goroutines which do not map 1:1 to threads, which means that things like “dropping privileges for a certain thread” does not work. Instead I had to do other work-arounds, to sandbox the executable (more info).

Feedback appreciated!

1 Like

I do remember your post. I’ve never tried it yet but certainly soon. I’m also doing such things multiple times when I’m working with dockers. I would also suggest you to have a look to https://www.qubes-os.org/doc/package-contributions/ in case you would be interested to have it packaged into QubesOS-contrib repository.

If you actually want rsync working over qubes-rpc, without the overhead
of golang, it’s simple - https://github.com/unman/qubes-sync
No need for workrounds, and it’s just rsync

1 Like

@unman that looks neat, I’ll look into it!

For this use-case you can also use the “Git connection between VMs”