─── ❖ ── ✦ ── ❖ ───

This note is about how to use offen/docker-volume-backup (GitHub, Documentation) to back up (and restore) docker volumes. Offen works with mounted folders as well as docker volumes and offers many backup targets.

Useful docker commands

I have compiled a list of useful docker commands.

"Fatal error running command: permission denied"
I encountered once that the Offen container had no read access:
level=INFO msg="Removed tar file `/tmp/valknut_containers-2024-05-30T15-14-49.tar.gz`."
level=ERROR msg="Fatal error running command: permission denied" error="main.(*command).runAsCommand: error running script: main.runScript.func4: error running script: main.(*script).createArchive: error walking filesystem tree: open /backup/valknut_containers: permission denied"
I solved it by running the container with --privileged

Manual backup

Into current folder

Let’s say we want to back up the volume volumename into an archive file that’s placed inside our current directory.

Set shell variable with volume name, useful if you want to backup multiple volumes (different syntax on Windows?)

DVAR='volumename'

Make sure the corresponding containers are stopped. If the volume you want to backup contains a database, it might be advisable to create a database backup inside this volume now.
Create the backup file. It will be named like volumename-2024-05-23T11-45-02.tar.gz.

Create Backup in current folder (no subdirectory)

docker run --rm \
  -v $DVAR:/backup/$DVAR:ro \
  -v .:/archive \
  --env BACKUP_ARCHIVE="/archive" \
  --env BACKUP_COMPRESSION="gz" \
  --env BACKUP_FILENAME="$DVAR-%Y-%m-%dT%H-%M-%S.{{ .Extension }}" \
  --env BACKUP_FILENAME_EXPAND="true" \
  --entrypoint backup \
  offen/docker-volume-backup:v2

Restart the previously stopped containers.

Into another volume

I use this approach on remote hosts to create backups inside a volume (offen_backup_syncthing) that’s mounted to a Syncthing container, which copies the backups to my home (or another backup location).

Required folder structure

The folder offenbackup inside the volume offen_backup_syncthing must be present, the copy script inside offen/docker-volume-backup fails otherwise. Workaround: → replace BACKUP_ARCHIVE="/archive/offenbackup" with BACKUP_ARCHIVE="/archive", which changes where the backup archives are stored inside the volume offen_backup_syncthing.

The contents of offen_backup_syncthing are structured like this to separate the backup archives from the syncthing files (.stfolder & .stversions):

offen_backup_syncthing
 ├── .stfolder
 │   └── ...
 ├── .stversions
 │   └── ...
 └── offenbackup
     ├── volumename-2024-05-23T11-45-02.tar.gz
     └── ...

Set shell variable with volume name.

DVAR='volumename'

Make sure the corresponding containers are stopped. If the volume you want to backup contains a database, it might be advisable to create a database backup inside this volume now.
Create the backup file.

Create Backup inside syncthing volume (with subdirectory)

docker run --rm \
  -v $DVAR:/backup/$DVAR:ro \
  -v offen_backup_syncthing:/archive \
  --env BACKUP_ARCHIVE="/archive/offenbackup" \
  --env BACKUP_COMPRESSION="gz" \
  --env BACKUP_FILENAME="$DVAR-%Y-%m-%dT%H-%M-%S.{{ .Extension }}" \
  --env BACKUP_FILENAME_EXPAND="true" \
  --entrypoint backup \
  offen/docker-volume-backup:v2

Restart the previously stopped containers.

Of course Syncthing has to be configured properly to sync the backup archives to remote locations.

Add container information to volume before backup

My idea here is to add some information about the container that’s using the target volume to said volume before the backup.

Get the RepoDigest corresponding to the volume $VOLUMENAME that is to be backed up:

Warning

The command fails if the volume $VOLUMENAME is mounted into multiple containers simultaneously, even if they’re stopped!

docker image inspect --format '{{index .RepoDigests 0}}' $(docker inspect --format='{{.Id}} {{.Name}} {{.Image}}' $(docker ps -aq) | grep $(docker ps -aq --filter volume=$VOLUMENAME) | awk '{print $3}')

The components of this command
Get ContainerID of container(s) that use the volume $VOLUMENAME
docker ps -aq --filter volume=$VOLUMENAME
Get ContainerID, ContainerName & ImageID of the container $CONTAINERID.
docker inspect --format='{{.Id}} {{.Name}} {{.Image}}' $(docker ps -aq) | grep $CONTAINERID
We only need the ImageID, so we add
| awk '{print $3}'
List image (and RepoDigest) of the local image $IMAGEID (also works with $IMAGENAME)
docker image inspect --format '{{index .RepoDigests 0}}' $IMAGEID

We write that information into a new file:

mkdir Offen-Backup-Info && docker image inspect --format '{{index .RepoDigests 0}}' $(docker inspect --format='{{.Id}} {{.Name}} {{.Image}}' $(docker ps -aq) | grep $(docker ps -aq --filter volume=$VOLUMENAME) | awk '{print $3}') >> ./Offen-Backup-Info/repodigest.txt

With two different methods we reconstruct the docker run command that created the container and save those as well.

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro assaflavie/runlike $(docker ps -aq --filter volume=$VOLUMENAME) >> ./Offen-Backup-Info/docker_run_runlike.txt

docker inspect --format "$(curl -s https://gist.githubusercontent.com/efrecon/8ce9c75d518b6eb863f667442d7bc679/raw/run.tpl)" $(docker ps -aq --filter volume=$VOLUMENAME) >> ./Offen-Backup-Info/docker_run_inspect.txt

We save the output of docker volume inspect & docker inspect as well.

docker volume inspect $VOLUMENAME >> ./Offen-Backup-Info/docker_inspect_volume.txt

docker inspect $(docker ps -aq --filter volume=$VOLUMENAME) >> ./Offen-Backup-Info/docker_inspect_container.txt

We copy the folder ./Offen-Backup-Info into the volume (works with busybox and alpine)

docker run --rm -v .:/src -v $VOLUMENAME:/data alpine cp -r /src/Offen-Backup-Info /data

Run the desired backup.
Cleanup

docker run --rm -v $VOLUMENAME:/data/ alpine /bin/sh -c "rm -rf /data/Offen-Backup-Info"

rm -r Offen-Backup-Info

Inspect or extract backup archives

The backup gets stored as a .tar.gz file, which can be extracted with tar -xvf backup.tar.gz. This creates a new folder backup (Note: NOT with the name of the archive, but with the name of the highest folder inside it!) and within this a folder with the volume name. The volume’s contents are inside this folder.

You can also view the content of backup.tar.gz like this:

tar -tvf backup.tar.gz

or using Docker (if for some reason tar is not installed):

docker run --rm -v ./backup.tar.gz:/data/backup.tar.gz alpine tar -tvf /data/backup.tar.gz

Restore a backup manually

Assumptions:

In your working directory is only one backup file per volume (so $DVAR-*.tar.gz is unambiguous).
Your archive file created by offen/docker-volume-backup has its contents nested like in this example (the volume’s name, $DVAR, is here ‘volumename’):

# tar -tvf backup.tar.gz
tar: Removing leading `/' from member names
drwxr-xr-x root/root         0 2024-05-23 13:45 /backup
drwxr-xr-x 1000/users        0 2024-05-23 13:38 /backup/volumename
-rw-r--r-- 1000/users        4 2024-05-23 13:15 /backup/volumename/config.json

With local extraction (docker cp)

Warning

Depending on your working directory, you might run into problems related to the filesystem’s path length limit.

Set shell variable with the name of the volume you want to restore to

DVAR='volumename'

Remove old volume contents (after stopping related containers!)

docker run --rm -v $DVAR:/data/ alpine /bin/sh -c "rm -rf /data/*"

Extract backup into local folder. It’s folder structure should look like this: ./tmp/$DVAR/backup/$DVAR/[volume contents].

mkdir -p ./tmp/$DVAR
 
tar -C ./tmp/$DVAR -xvf $DVAR-*.tar.gz

Copy files into the empty volume using docker cp.

docker run -d --name temp_restore_container -v $DVAR:/backup_restore alpine
  
docker cp -a ./tmp/$DVAR/backup/$DVAR/. temp_restore_container:/backup_restore

Cleanup.

docker rm temp_restore_container
 
rm -r ./tmp/$DVAR
 
rm -r $DVAR-*.tar.gz

Without local extraction

It’s possible to restore the backup directly to the volume instead of extracting on the host first.

Set shell variable with the name of the volume you want to restore to.

DVAR='volumename'

Remove old volume contents (after stopping related containers!)

docker run --rm -v $DVAR:/data/ alpine /bin/sh -c "rm -rf /data/*"

Rename the backup archive that you want to restore to backup.tar.gz. cp because we might want to archive the backup file.
This step is necessary because Docker won’t accept -v ./$DVAR-*.tar.gz:/archive/backup.tar.gz:ro, instead Docker creates a new folder called volumename-*.tar.gz.

cp $DVAR-*.tar.gz backup.tar.gz

Restore the backup. Without the flag --strip-components=2 the contents of our restored volume would look like this: /backup/$DVAR/[contents of original volume] instead of [contents of original volume]. Depending on how you’ve nested the leading folders in your backup, you might have to change this. You can inspect your backup file first with tar -tvf backup.tar.gz.

docker run --rm -v $DVAR:/target -v ./backup.tar.gz:/archive/backup.tar.gz:ro alpine tar --strip-components=2 -C /target -xzvf /archive/backup.tar.gz

Cleanup

rm backup.tar.gz

Personal backup scripts

This section is about my personal backup scripts, adapted to the layout of my systems.

Unraid

/mnt/user/appdata/Jellyfin-AMD-Intel-Nvidia
/mnt/user/appdata/tailscale
/mnt/user/appdata/scrutiny2/config
/mnt/user/appdata/stash2
  config
  metadata
  cache

stash

Stop container

docker stop stash

VOLUMENAME='/mnt/user/appdata/stash/config'

mkdir /mnt/user/medien/offen-backup && cd /mnt/user/medien/offen-backup

Prepare additional info

mkdir Offen-Backup-Info && docker image inspect --format '{{index .RepoDigests 0}}' $(docker inspect --format='{{.Id}} {{.Name}} {{.Image}}' $(docker ps -aq) | grep $(docker ps -aq --filter volume=$VOLUMENAME) | awk '{print $3}') >> ./Offen-Backup-Info/repodigest.txt && docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro assaflavie/runlike $(docker ps -aq --filter volume=$VOLUMENAME) >> ./Offen-Backup-Info/docker_run_runlike.txt && docker inspect --format "$(curl -s https://gist.githubusercontent.com/efrecon/8ce9c75d518b6eb863f667442d7bc679/raw/run.tpl)" $(docker ps -aq --filter volume=$VOLUMENAME) >> ./Offen-Backup-Info/docker_run_inspect.txt && docker inspect $(docker ps -aq --filter volume=$VOLUMENAME) >> ./Offen-Backup-Info/docker_inspect_container.txt && mv ./Offen-Backup-Info /mnt/user/appdata/stash

Create backup archive

docker run --rm \
  -v /mnt/user/appdata/stash:/backup/stash:ro \
  -v `pwd`:/archive \
  --env BACKUP_ARCHIVE="/archive" \
  --env BACKUP_COMPRESSION="gz" \
  --env BACKUP_FILENAME="stash-%Y-%m-%dT%H-%M-%S.{{ .Extension }}" \
  --env BACKUP_FILENAME_EXPAND="true" \
  --entrypoint backup \
  offen/docker-volume-backup:v2

delete Offen-Backup-Info

rm -r /mnt/user/appdata/stash/Offen-Backup-Info

Start container

docker start stash

jellyfin

Stop container

docker stop jellyfin

VOLUMENAME='/mnt/user/appdata/jellyfin'

mkdir /mnt/user/medien/offen-backup && cd /mnt/user/medien/offen-backup

Prepare additional info

mkdir Offen-Backup-Info && docker image inspect --format '{{index .RepoDigests 0}}' $(docker inspect --format='{{.Id}} {{.Name}} {{.Image}}' $(docker ps -aq) | grep $(docker ps -aq --filter volume=$VOLUMENAME) | awk '{print $3}') >> ./Offen-Backup-Info/repodigest.txt && docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro assaflavie/runlike $(docker ps -aq --filter volume=$VOLUMENAME) >> ./Offen-Backup-Info/docker_run_runlike.txt && docker inspect --format "$(curl -s https://gist.githubusercontent.com/efrecon/8ce9c75d518b6eb863f667442d7bc679/raw/run.tpl)" $(docker ps -aq --filter volume=$VOLUMENAME) >> ./Offen-Backup-Info/docker_run_inspect.txt && docker inspect $(docker ps -aq --filter volume=$VOLUMENAME) >> ./Offen-Backup-Info/docker_inspect_container.txt && mv ./Offen-Backup-Info $VOLUMENAME

Create backup archive

docker run --rm \
  -v $VOLUMENAME:/backup/jellyfin:ro \
  -v `pwd`:/archive \
  --env BACKUP_ARCHIVE="/archive" \
  --env BACKUP_COMPRESSION="gz" \
  --env BACKUP_FILENAME="jellyfin-%Y-%m-%dT%H-%M-%S.{{ .Extension }}" \
  --env BACKUP_FILENAME_EXPAND="true" \
  --entrypoint backup \
  offen/docker-volume-backup:v2

delete Offen-Backup-Info

rm -r /mnt/user/appdata/jellyfin/Offen-Backup-Info

Start container

docker start jellyfin

Missing topics → https://offen.github.io/docker-volume-backup/how-tos/

automatic backups with flags in docker compose

backup rotations & retention schedules

backups to remote locations → pcloud.com

encrypting backups

run custom commands during backup/restore

Steps to improve this guide

Make stuff like $VOLUMENAME & ContainerID consistent

write consistently ‘we’ or ‘you’

add unraid section

add synology section

─── ❖ ── ✦ ── ❖ ───

🍺zoy.sh📙

Explorer

Backup & restore Docker volumes with Offen

Table of Contents

Manual backup

Into current folder

Into another volume

Add container information to volume before backup

Inspect or extract backup archives

Restore a backup manually

With local extraction (docker cp)

Without local extraction

Personal backup scripts

Unraid

stash

jellyfin

Graph View

Table of Contents

Backlinks