Skip to content

Improve postgres backup portability and add Metabase restore runbook#4850

Open
themightychris wants to merge 2 commits intomainfrom
metabase-migration
Open

Improve postgres backup portability and add Metabase restore runbook#4850
themightychris wants to merge 2 commits intomainfrom
metabase-migration

Conversation

@themightychris
Copy link
Contributor

Description

  • Switch the postgresql-backup CronJob from pg_dumpall to pg_dump --no-owner --no-privileges so dumps are directly importable into Cloud SQL without needing to extract database sections or remap roles.
  • Add a runbook documenting the full Metabase restore process for the Cloud Run + Cloud SQL deployment.

Resolves #4757

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

How has this been tested?

  • The restore process was tested end-to-end: imported the K8s prod database (205 users, 312 dashboards, 4,662 cards) into Cloud SQL
  • Metabase v0.58.7 auto-migrated the schema from v0.50.26.1, and the service is live at https://metabase.dds.dot.ca.gov/
  • I was able to log in with my existing credentials and view recent dashboards

Post-merge follow-ups

  • No action required
  • Actions required (specified below)

@github-actions
Copy link

github-actions bot commented Feb 27, 2026

The following changes will be applied to the production Kubernetes cluster upon merge.

BE AWARE this may not reveal changes that have been manually applied to the cluster getting undone—applying manual changes to the cluster should be avoided.

monitoring-grafana, postgresql-backup, CronJob (batch) has changed:
...
                    wget -q -O - https://healthchecks.ops.k8s.jarv.us/ping/9095f868-4c8e-443c-80a4-9b9a22d9a8d8/start || echo "Failed to ping start"
                  

                    # snapshot current database
                    echo "Snapshotting Database"
-                   pg_dumpall --clean \
+                   pg_dump --clean --if-exists --no-owner --no-privileges \
                      | gzip --rsyncable \
                      | restic backup \
                        --host monitoring-grafana:postgresql-backup \
                        --stdin \
-                       --stdin-filename pg_dumpall.sql.gz
+                       --stdin-filename pg_dump.sql.gz

                    sql_snapshot_status=$?

                    # prune aged snapshots
                    echo "Pruning aged snapshots"
...
metabase, postgresql-backup, CronJob (batch) has changed:
...
                    wget -q -O - https://healthchecks.ops.k8s.jarv.us/ping/f223e786-4fa4-44f9-a3db-3db5988a12cf/start || echo "Failed to ping start"
                  

                    # snapshot current database
                    echo "Snapshotting Database"
-                   pg_dumpall --clean \
+                   pg_dump --clean --if-exists --no-owner --no-privileges \
                      | gzip --rsyncable \
                      | restic backup \
                        --host metabase:postgresql-backup \
                        --stdin \
-                       --stdin-filename pg_dumpall.sql.gz
+                       --stdin-filename pg_dump.sql.gz

                    sql_snapshot_status=$?

                    # prune aged snapshots
                    echo "Pruning aged snapshots"
...
sentry, postgresql-backup, CronJob (batch) has changed:
...
                    wget -q -O - https://healthchecks.ops.k8s.jarv.us/ping/93c6d460-e014-4b30-bf32-b7974bf5fa4d/start || echo "Failed to ping start"
                  

                    # snapshot current database
                    echo "Snapshotting Database"
-                   pg_dumpall --clean \
+                   pg_dump --clean --if-exists --no-owner --no-privileges \
                      | gzip --rsyncable \
                      | restic backup \
                        --host sentry:postgresql-backup \
                        --stdin \
-                       --stdin-filename pg_dumpall.sql.gz
+                       --stdin-filename pg_dump.sql.gz

                    sql_snapshot_status=$?

                    # prune aged snapshots
                    echo "Pruning aged snapshots"
...
metabase-test, postgresql-backup, CronJob (batch) has changed:
...
                    wget -q -O - https://healthchecks.ops.k8s.jarv.us/ping/ff63f290-fe1c-4011-8405-0ba8b2636ea0/start || echo "Failed to ping start"
                  

                    # snapshot current database
                    echo "Snapshotting Database"
-                   pg_dumpall --clean \
+                   pg_dump --clean --if-exists --no-owner --no-privileges \
                      | gzip --rsyncable \
                      | restic backup \
                        --host metabase-test:postgresql-backup \
                        --stdin \
-                       --stdin-filename pg_dumpall.sql.gz
+                       --stdin-filename pg_dump.sql.gz

                    sql_snapshot_status=$?

                    # prune aged snapshots
                    echo "Pruning aged snapshots"
...

@ohrite ohrite force-pushed the metabase-migration branch from bb6cb3e to 913eb64 Compare February 27, 2026 22:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Analyst sees dashboards.calitp.org content on metabase.dds.dot.ca.gov

2 participants