Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation on migrations #882

Merged
merged 2 commits into from
Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion docs/database-migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,16 @@
> The migration process is currently manual.

## What are they?
Service Catalogue uses the [Prisma](https://www.prisma.io) ORM for database migrations,

Service Catalogue uses the [Prisma](https://www.prisma.io) ORM for database migrations,
to profit from the audit trail it provides.

Migrations are located in [../packages/common/prisma/migrations](../packages/common/prisma/migrations).

Prisma tracks the migration history in the `_prisma_migrations` table in the database.

## Creating a new migration

See https://www.prisma.io/docs/guides/migrate/developing-with-prisma-migrate#create-migrations for the recommended process for developing a migration.

To create a migration file, run:
Expand All @@ -34,6 +36,16 @@ npx -w common prisma generate

## Applying a migration to CODE or PROD

### Migrations as part of CD

Upon a RiffRaff deploy (to either CODE or PROD), a `prisma.zip` containing the Prisma schema and migration scripts is uploaded into the account artifact bucket. This action triggers an [ECS task](../packages/cdk/lib/prisma-migrate-task.ts) running the [`prisma-migrate`](../containers/prisma-migrate/Dockerfile) container, which retrieves the `prisma.zip` and applies the corresponding migration to the CODE/PROD RDS instance.

This migrations can fail for a number of reasons - for example attempting to add a `NOT NULL` column to a table with existing data. In this case, manual intervention will be necessary. Prisma provides some useful documentation on [patching and hotfixing](https://www.prisma.io/docs/orm/prisma-migrate/workflows/patching-and-hotfixing).

### Manual migrations

Although migrations are applied automatically on deploy, these documentation is preserved here in case a manual migration is necessary.

Prerequisite:

1. You have an approved Pull Request
Expand All @@ -46,6 +58,12 @@ You can apply a migration to CODE or PROD using the [CLI](../packages/cli)):
npm -w cli start migrate -- --stage [CODE|PROD]
```

The command above will not apply the migration without a `--confirm` flag. If satisfied with the output of the above, you can run:

```bash
npm -w cli start migrate -- --stage [CODE|PROD] --confirm
```

See also:

- https://www.prisma.io/docs/concepts/components/prisma-migrate/migrate-development-production#production-and-testing-environments
Expand Down
8 changes: 8 additions & 0 deletions packages/cdk/lib/cloudquery/images.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,14 @@ export const Images = {
postgres: ContainerImage.fromRegistry(
'public.ecr.aws/docker/library/postgres:16-alpine',
),
/**
* This image is built in CI by the service-catalogue, and tagged with the
* SHA of the corresponding commit
*
* To use a new image, update the SHA here. The full set of possible tags
* can be found at
* https://github.com/guardian/service-catalogue/pkgs/container/service-catalogue%2Fprisma-migrate
*/
prismaMigrate: ContainerImage.fromRegistry(
'ghcr.io/guardian/service-catalogue/prisma-migrate:sha-06b60dc3f47bfdfe8997d3d622e91f5508733ece',
),
Expand Down
Loading