How to replicate dotfiles across qubes?

[tanky0u]

@unman

There are existing solutions for rsync and git over qrexec which allow
for this without using internal Qubes network.
You could find them by searching the forum.

Ooh, I am interested in reading more about this. I will take a look at the forum search.

[idarwin]

Since I use both Qubes and a number of other *Nix systems (mostly OpenBSD), I have a “dotfiles” git repo someplace secure enough, where I can access it from all machines. The internet is fast enough these days that I can do a ‘git pull’ in my login scripts so I don’t have to manually update, yet it’s always up to date. There’s a Makefile to symlink the files into place when I clone it onto a new machine. For stuff that shouldn’t be on all the machines, this doesn’t apply, but you could extend it to have a few repos or just handle those few files manually.

[tanky0u]

I stumbled upon another way to manage the dotfiles across qubes: using GNU stow and git:

Piped

[Sven]

Moved to User Support.

[tanky0u]

Can you explain if using rsync to move git repo locally from qube1 to qube2 would overwrite the qube2’s local git repo version? I mean, let’s say I make a change to my dotfiles git repo on qube1. Then I use rsync to sync that change to qube2. What happens? Is it going to be overwritten?

I am trying to see if it is possible to keep track of a git repo for my dotfiles, with a “remote” server being one of my local qubes os qubes. Can I do git push/pull/fetch/merge operations that way?

[tanky0u]

Or, let’s say I simply qvm-copy the ~/.dotfiles folder or, ~/.dotfiles/.git folder (?) to other qubes that have the same ~/.dotfiles folder but outdated git repo version.

What do I do with the ~/.dotfiles folder or, ~/.dotfiles/.git in the ~/QubesIncoming directory now? Do I move that to the ~/.dotfiles? If I do that is it going to automatically merge the commits or something?

[tanky0u]

I might be in the process of devising a solution for my use case. I am making use of git bundle command. See here for an introduction: Git - Bundling

As I keep testing this workflow, I will straighten the kinks and if I am satisfied with it, probably will write a full guide. But for now, here’s some quick notes on it:

Syncing dotfiles across qubes without a remote server on the internet:

on the dotfiles qube:
cd into ~/.dotfiles dir.
You already have a git repo in there. You have your dotfiles organized according to stow program.
~/.dotfiles/zsh/.config/zsh/ → this dir contains your zsh dotfiles.

So, on ~/.dotfiles dir:
$ git bundle create dotfiles.bundle HEAD master

this will create a dotfiles.bundle file containing the version of ~/.dotfiles directory as it is on the dotfiles qube.

You then qvm-move that dotfiles.bundle into another qube that you want to propagate your dotfiles to.

Let’s say you qvm-move 'd that file to “mydebian” qube. On the mydebian qube, create the ~/.dotfile dir. Then, move the dotfiles.bundle file there:

$ mv ~/QubesIncoming/dotfiles/dotfiles.bundle ~/.dotfiles

Then, cd into ~/.dotfiles dir and do

$ git pull dotfiles.bundle

You will have the same dir with the same git history and files as in dotfiles qube now in your mydebian qube.

Let’s say, you change some files in ~/.dotfiles, in this mydebian qube, and you want to propagate those changes back to the dotfiles qube. You commit your changes, and create a git bundle in mydebian:

$ git bundle create dotfiles.bundle HEAD master

Basically qvm-move that dotfiles.bundle file back to dotfiles qube. Move that bundle file again to the ~/.dotfiles dir, and git pull from it in the dotfiles dir.

You can further assign following git usernames for keeping track of which commits came from which qubes:

(on dotfiles ) $ git config user.name ‘dotfiles’
(on dotfiles ) $ git config user.email ‘dotfiles@localhost’

(on mydebian ) $ git config user.name ‘mydebian’
(on mydebian ) $ git config user.email ‘mydebian@localhost’

that way, your commit history will show the qubes’ names as the commit owners.

[unman]

Can you explain if using rsync to move git repo locally from qube1 to qube2 would overwrite the qube2’s local git repo version? I mean, let’s say I make a change to my dotfiles git repo on qube1. Then I use rsync to sync that change to qube2. What happens? Is it going to be overwritten?

Yes - rsync will copy over the destination. It’s designed to put the
source and destination in sync. (There are configuration options to deal
with various use cases.)
It’s efficient and fast.
For git I would recommend git over qrexec.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.

[anon22772651]

New idea that I might implement. Currently I send a script to each new VM and run it, and it’ll configure all my settings by running commands and editing files. This makes updating an issue as someone pointed out, but I rarely update my dotfiles after being set up so it hasn’t bothered me yet. However you could centralize it by placing the scripts in dom0, then running some sort of for loop on each VM you want to update, where it sends the new centralized file from dom0 and then executes the file in each VM. That way you only have to update one file, and then run the update loop.

[unman]

This is where salt comes in to its own.
You can place the configuration files in dom0, and then target delivery
, writing configuration as needed, and changing in accord with the
target system, either by name or by other features.
You can read about Salt here - I
have a basic tutorial here covering many
uses and examples.

I never presume to speak for the Qubes team. When I comment in the Forum or in the mailing lists I speak for myself.

[Kiryoko]

I’m currently using a solution very similar to the one described here, but it’s a bit cumbersome to keep all of my VMs up-to-date with the main config.

Does your current workflow include a convenient way to automate all of this too?

Here’s the approach I’m trying to build/configure:

How To Integrate a Centralized Dotfiles Template with Disposable Coding VMs and Isolated Project Data

I have a setup where I manage multiple coding projects and need tight isolation. My workflow involves having a dedicated AppVM for each project so that malicious code can’t tamper with my global dotfiles configuration. My dotfiles configuration is centralized in a standalone vault-dotfiles VM from which I pull config updates. The problem is that, as the number of projects and client goes up, also the amount of time and complexity to keep everything up-to-date goes up like crazy. I’d like a si…

Specifically this:

Ideally, I’d have a disposable coding environment that attaches project data as a block device (like from a project-data VM) while always pulling fresh dotfiles from a known template source.

But I’m still tinkering around to understand how to do this and whether it’s possible at all.

Are you still using the same method?

[tanky0u]

@Kiryoko

Does your current workflow include a convenient way to automate all of this too?

It’s been a while since I posted that. Nowadays, I don’t use git bundles for dotfiles replication. Instead, I have a dotfiles qube which is allowed to send files to any other qubes (except dom0 ofc). I have a simple script on dom0 which remote commands my dotfiles qube to send the select dotfile files to select qubes, which they then automatically stow into ~/.config/ directory.