jkaninda/pg-bkup repository overview

⁠PG-BKUP

PG-BKUP backs up, restores and migrates PostgreSQL databases. It streams pg_dump output through compression and encryption straight into the storage: no temporary dump file, constant memory, whatever the database size.

Documentation: https://jkaninda.github.io/pg-bkup⁠ · MySQL: mysql-bkup⁠

⁠Features

• Storage: local, Amazon S3 and S3-compatible (MinIO, Wasabi, R2...), Google Cloud Storage, Azure Blob, SFTP, FTP/FTPS.
• Streaming: pg_dump | gzip/zstd | GPG/age | upload, and back for restores. Nothing is written to disk.
• Encryption: GPG (passphrase or public key) and age⁠ (recipients or passphrase).
• Safe restores: psql -v ON_ERROR_STOP=1 --single-transaction (custom archives too, through pg_restore -f -): a failing or corrupted backup changes nothing, and a dump from another server is not stopped by its roles or its pg_dump version.
• Every PostgreSQL version: the image ships the 16, 17 and 18 clients and picks, per server, the lowest pg_dump that is not older than the server (servers 9.2 and newer). No more "server version mismatch".
• Backup index and retention: by age (BACKUP_RETENTION_DAYS) or count (BACKUP_RETENTION_COUNT); only backups recorded in the index are ever deleted.
• Commands: backup, restore, migrate, verify, list, delete, prune, reconcile, config, health.
• Scheduling: --cron-expression daemon with a Docker HEALTHCHECK and optional Prometheus metrics.
• Notifications: email, Telegram, Slack, Discord, ntfy, generic webhook, healthchecks.io pings.
• Formats: plain SQL (default, .sql) or custom (PG_DUMP_FORMAT=custom, .dump); schema-only, data-only, selected or excluded tables, all databases one file each, or the whole instance with pg_dumpall.

⁠Quick start

docker run --rm --network your_network \
  -v $PWD/backup:/backup \
  -e DB_HOST=postgres -e DB_USER=postgres -e DB_PASSWORD=password \
  jkaninda/pg-bkup backup -d database_name

The last line of a successful backup always names the artifact:

level=INFO msg="Backup completed" database=database_name file=database_name_20261027_031500.sql.gz size=12.3MiB duration=2.1s storage=local location=/backup

Restore it (the target database must exist):

docker run --rm --network your_network \
  -v $PWD/backup:/backup \
  -e DB_HOST=postgres -e DB_USER=postgres -e DB_PASSWORD=password \
  jkaninda/pg-bkup restore -d database_name -f database_name_20261027_031500.sql.gz

restore accepts .sql, .sql.gz, .sql.zst, .dump (custom format) and any of them encrypted (.gpg, .age). The format is detected from the content, not the name.

A connection URL works too: DB_URL=postgres://user:p%40ss@host:5432/db?sslmode=require or the 1.x JDBC form jdbc:postgresql://host:5432/db?user=user&password=pass.

⁠Backup variants

--all-databases skips templates and the postgres database; databases are listed from DB_AUTH_DATABASE (default postgres). An --all-in-one file is restored with restore -f all_databases_....sql.gz (no -d): it is replayed without a single transaction and, by default, without ON_ERROR_STOP, because pg_dumpall output always contains statements such as CREATE ROLE postgres that fail on an existing server (RESTORE_ON_ERROR_STOP).

The custom format is dumped with pg_dump -Fc -Z0 and compressed by the pipeline (parallel gzip or zstd), so COMPRESSION applies to both formats and every artifact is decompressed the same way. Custom-format archives are turned into SQL by pg_restore --no-owner -f - and replayed by the same psql as plain dumps (RESTORE_NO_OWNER=false keeps ownership).

⁠Migrate

migrate pipes pg_dump into psql on the target (TARGET_DB_*), with no intermediate file:

docker run --rm --network your_network \
  -e DB_HOST=old -e DB_USER=postgres -e DB_PASSWORD=pw -e DB_NAME=shop \
  -e TARGET_DB_HOST=new -e TARGET_DB_USER=postgres -e TARGET_DB_PASSWORD=pw -e TARGET_DB_NAME=shop \
  jkaninda/pg-bkup migrate

• migrate --all-databases (-a) copies every database into the same name, creating missing ones.
• migrate --entire-instance (-I) copies roles, tablespaces and every database with pg_dumpall (add --no-role-passwords to keep the target's passwords).
• A single database is copied in one transaction; the target database is created if it does not exist.
• TARGET_DB_URL, TARGET_DB_SSL_MODE, TARGET_DB_SSL_CA/CERT/KEY and TARGET_DB_AUTH_DATABASE are honoured.

⁠Storage

# S3 / MinIO
-e AWS_S3_ENDPOINT=https://s3.amazonaws.com -e AWS_S3_BUCKET_NAME=backups -e AWS_REGION=us-east-1 \
-e AWS_ACCESS_KEY=... -e AWS_SECRET_KEY=... \
jkaninda/pg-bkup backup --storage s3 -d shop --path /prod/shop
# MinIO: add AWS_DISABLE_SSL=true (plain http) and AWS_FORCE_PATH_STYLE=true.
# Without keys, the AWS default credential chain (IAM role, IRSA, profile) is used.

# SFTP (host keys are verified: mount a known_hosts file)
-e SFTP_HOST=backup.example.com -e SFTP_USER=backup -e SFTP_IDENTITY_FILE=/config/id_ed25519 \
-e SFTP_KNOWN_HOSTS=/config/known_hosts \
jkaninda/pg-bkup backup --storage sftp -d shop --path /home/backup/shop

# FTP, Azure Blob, Google Cloud Storage
jkaninda/pg-bkup backup --storage ftp   -d shop --path /backups   # FTP_HOST, FTP_USER, FTP_PASSWORD [FTP_TLS]
jkaninda/pg-bkup backup --storage azure -d shop --path backups    # AZURE_STORAGE_ACCOUNT_NAME, _KEY (or _SAS_TOKEN), _CONTAINER_NAME
jkaninda/pg-bkup backup --storage gcs   -d shop --path backups    # GCS_BUCKET, GCS_CREDENTIALS_FILE

restore takes the same --storage and --path plus the bare file name: restore --storage s3 --path /prod/shop -d shop -f shop_20261027_031500.sql.gz.gpg.

⁠Encryption

ENCRYPTION=auto (default) picks from the keys that are set; GPG wins when both are set. Encryption fails closed: a configured but unusable key aborts the backup. Encrypted files end in .gpg or .age.

⁠Backup index and retention

Each storage directory holds an index (BACKUP_INDEX_NAME, default pg-bkup-metadata.json, placeholders {tool} {engine} {name} {db}) and a <artifact>.meta.json sidecar per backup, with size, sha256, server version and encryption.

• BACKUP_RETENTION_DAYS=7 and/or BACKUP_RETENTION_COUNT=10 prune after each backup; the newest backup is always kept.
• Only files recorded in the index are deleted. Backups made before the index existed are imported with BACKUP_RETENTION_ADOPT_LEGACY=true; BACKUP_INDEX=false falls back to name-pattern retention.
• list, verify -f FILE, delete -f FILE, prune [--dry-run] and reconcile work from the index.

⁠Scheduled backups

docker run -d --name pg-bkup --network your_network -v $PWD/backup:/backup \
  -e DB_HOST=postgres -e DB_USER=postgres -e DB_PASSWORD=password -e BACKUP_RETENTION_DAYS=7 \
  jkaninda/pg-bkup backup -d database_name --cron-expression "@daily"

Cron expressions take 5 or 6 fields, @daily/@hourly/... or @every 6h. BACKUP_RUN_ON_START=true runs one backup immediately. A failed run is logged and notified and the daemon keeps running. docker inspect shows the health status (pg-bkup health); METRICS_ADDR=:9090 serves Prometheus metrics.

On Kubernetes, prefer a CronJob running a one-shot backup (see the docs⁠).

⁠Several databases in one run

# /config/config.yaml (or --config / BACKUP_CONFIG_FILE)
cronExpression: "@daily"      # optional
databases:
  - host: lldap-db
    name: lldap
    user: lldap
    password: ${LLDAP_DB_PASSWORD}   # ${VAR} and ${VAR:-default} are expanded
    path: /s3-path/lldap
  - name: keycloak                    # host/user/password from DB_HOST_KEYCLOAK or KEYCLOAK_DB_HOST, ... then DB_HOST
    path: /s3-path/keycloak
    sslMode: require

For each entry: the YAML field wins, then <KEY>_<NAME> / <NAME>_<KEY> env vars (e.g. DB_PASSWORD_KEYCLOAK, KEYCLOAK_DB_PASSWORD), then the global variable.

⁠Exit codes

--result-file PATH writes the result as JSON; LOG_FORMAT=json makes the logs machine-readable.

⁠Copyright

Copyright (c) [2024] [Jonas Kaninda]