Persisting docker images in App VM

Hello everyone!
I’m currently trying to setup a development environment in a specialized VM, based on a clone of the debian 10 template.
I’ve tried several configuration locations while setting up the docker folders /etc/docker and /var/lib/docker as bind-dirs, but none of them seem to work.

My final goal would be to configure the template, so that I can just create new dev environments each with their own docker images and containers and not have all the images inside the template. It was my interpretation, that configuring the directories in a 50_user.conf file located at /etc/qubes-bind-dirs.d/ would achieve exactly that but I cannot seem to get this to work.
Configuring the app VM via a config file located at /re/config/qubes-bind-dirs.d does also not work.

I’m using the information from the documentation at https://www.qubes-os.org/doc/bind-dirs/

I tried running sudo /usr/lib/qubes/init/bind-dirs.sh inside the app vm and that outputs a syntax error. As the documentation states that running this script does not work without running umount first and discourages running the script with the umount parameter (unless I’m reading that part of the docs wedding?) I don’t know if this should happen or not.

Can anyone tell me if I’m doing something wrong?

1 Like

This is not the answer you are looking for, but rather an alternative path.

In my case, I’ve been using podman on fedora as docker alternative in an AppVM. I’ve had not major issues since then.

As opposed to docker, it is daemonless and allows you to run as other users. Additionally (and the thing that should solve your problem) it stores the docker images on your user’s home

Storage root dir in which data, including images, is stored (default: “/var/lib/containers/storage” for UID 0, “$HOME/.local/share/containers/storage” for other users). Default root dir configured in /etc/containers/storage.conf.

– man page of podman

So if you use this there should be no need to play with bind-dirs. But if you choose to go this path, my advice is for you to read up on the differences between the two to see if it suits your needs.

2 Likes

Thanks for the suggestion, I’ll check it out. I guess as another alternative, I could also configure the docker storage directory for images and containers somehow.

But the underlying question remains. If I ever wanted to persist other parts of an App VM, did I do something fundamentally wrong?
Can I somehow check the bind-dir config for errors, are there logs on VM Startup where information regarding the bind-dirs.sh script can be found?

I think you are in the right direction. But I’ve never explored bind-dirs myself.

To anyone interested, I did indeed have a syntax error in the configuration file (a space between binds+= and the following opening parenthesis). I fixed the error and can now use the configuration in the template VM to persist images inside the derived app VMs

/close

1 Like

@laser-brain
I have the same problem as yours but I don’t understand why except I not fluent in Docker :slight_smile:

I install Docker in my a clone of a debian-10-template like this :

sudo apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-common
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo “$ID”)/gpg | > sudo apt-key add -
sudo add-apt-repository “deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo “$ID”) $(lsb_release -cs) stable”
sudo apt update
sudo apt install -y docker-ce
sudo groupadd docker
sudo usermod -aG docker user
sudo systemctl enable docker
sudo mkdir -p /rw/config/qubes-bind-dirs.d
sudo bash -c ‘cat << EOF > /rw/config/qubes-bind-dirs.d/50_user.conf
binds+=( ‘/var/lib/docker’ )
binds+=( ‘/etc/docker’ )
EOF’
sudo curl -L “https://github.com/docker/compose/releases/download/1.28.2/docker-compose-$(uname -s)-$(uname -m)” -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo mv /usr/local/bin/docker-compose /usr/bin/docker-compose

But in my AppVM nothing is persistent…

Is there a difference between my code and yours?

Just like @unman I do use podman for OCI containers.

With podman-docker you get a simple alias that helps you when you follow docker based instruction manuals. You can even use docker-compose with podman. I would recommend using Fedora for that. It is as simple as:

# TemplateVM 
$ sudo dnf install podman-docker docker-compose
# App VM
$ bash <(echo "sudo systemctl enable --now podman.socket" | sudo tee -a /rw/config/rc.local)

Most oft those containers, that I am occasionally running, are rootless, which means they are running as the AppVMs user. Thus I have not applied any bind-dirs configuration. I think I have run a rootful container for something a while ago, but I have to look into whether I had to apply any bind-dirs configuration for that.

2 Likes

@spergynerd,

But in my AppVM nothing is persistent

I think that’s because you created /rw/config/qubes-bind-dirs.d/50_user.conf in the template VM? That’s the wrong VM — you need to create it in the AppVM instead.

Look: How to make any file persistent (bind-dirs) | Qubes OS

Inside the app qube.
Make sure folder /rw/config/qubes-bind-dirs.d exists.
Create a file […]

Hmm that was 1 year 1 month ago — I wonder if you got it working eventually or what did you do?

Works fine for me — and I accidentally ran the commands in the template VM first. Maybe app qube could be made bold in the docs :- )

Have not read what is up there lol. But judging from the title, what i did may be strongly related.

I automated my docker workflow in qubes. Here is my script. Maybe this helps.

Oh, and welcome to the forum @KajMagnus!

Thanks @Suspicious_Actions. Interesting script — I didn’t know one could run scripts in the app-VMs from dom0. Yes I see the bind mounts, and seems like a nice way to, … hmm, I’ll reply in that other topic :slight_smile: