How to store docker/podman containers and images in a second drive

Why:

  • to make the containers and images persistent across reboots of the AppVM
  • while excluding said containers and images from the backups (since you can download them again anytime)

Example use case:

I use it for selfhosting my Nostr front-end (Iris).

Docker is very common the selfhosting community. If you have a secondary drive, you can selfhost many services while excluding them from the backups.

HOWTO:

In /rw/config/rc.local:

exec 1> >(systemd-cat --identifier=rw-config-rc-local) 2>&1
echo "Start of /rw/config/rc.local"

# Exit if there's an error
set -e

mkdir -v /mnt/removable
mount -v /dev/xvdi /mnt/removable
mkdir -pv /mnt/removable/containerd
mkdir -pv /mnt/removable/containers
mkdir -pv /mnt/removable/docker
rm -rfv /var/lib/containerd
rm -rfv /var/lib/containers
rm -rfv /var/lib/docker
ln -fsv /mnt/removable/containerd /var/lib/
ln -fsv /mnt/removable/containers /var/lib/
ln -fsv /mnt/removable/docker /var/lib/


# From https://github.com/containers/podman/issues/2788
# From https://github.com/stackhpc/ansible-slurm-appliance/issues/234
chmod 4755 /usr/bin/newgidmap
chmod 4755 /usr/bin/newuidmap

systemctl unmask podman
systemctl unmask podman.socket
service podman start

I mask podman in the template and unmask it in the VMs that need it. I do this with other services too that aren’t needed in most VMs.

In dom0:

qvm-block attach <your-appvm> <device-for-the-storage> --persistent --option frontend-dev=xdvi

Troubleshooting:

sudo journalctl --identifier=rw-config-rc-local --follow

Even better if you have lnav installed:

sudo journalctl --identifier=rw-config-rc-local --follow | lnav

Let me know of any improvements to this guide.

1 Like