Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Squashing port-to-postgresql further #1199

Closed
wants to merge 34 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
2cb78c3
Update GitHub Actions for migrating to PostgreSQL
brucebolt Dec 21, 2023
38b468f
Update Dockerfile for migration to PostgreSQL
brucebolt Dec 21, 2023
7250e2f
Remove MongoDB gems
brucebolt Dec 21, 2023
ac7e15d
Add `.byebug_history` to .gitignore
brucebolt Dec 27, 2023
1589c84
Update Jenkinsfile for migration to PostgreSQL.
brucebolt Dec 21, 2023
bd37f08
Add basic ActiveRecord configuration for PostgreSQL
aldavidson Mar 20, 2023
81acc74
Port ContentItem.create_or_replace to ActiveRecord
aldavidson Mar 30, 2023
64d1210
Port find_by_path to ActiveRecord
aldavidson Apr 4, 2023
772e7cc
Make PublishIntent compatible with ActiveRecord
aldavidson Apr 5, 2023
ebe5467
Make content_items_controller 404 when it can't find the contentitem
aldavidson Apr 5, 2023
5dacee2
Update find_specific_term for PostgreSQL
aldavidson Apr 6, 2023
5c9f464
Simplify queries in data hygiene code
aldavidson Apr 6, 2023
9adfa8a
Enforce stringified keys in content_item.as_json
aldavidson Apr 6, 2023
a69d057
Add explicit specs for ContentItem.create_and_replace
aldavidson Apr 6, 2023
ea2f82b
Add DatabaseCleaner config to the Pact tests
aldavidson Apr 11, 2023
29aa93a
Add _id field to content_items, to store imported mongo ID
aldavidson Apr 21, 2023
13981e7
Add Mongo import task & supporting classes
aldavidson Apr 21, 2023
06208fb
Add performance:compare_hosts rake task
aldavidson Apr 28, 2023
797853a
Add optimised indexes on JSONB array routes/redirects columns
aldavidson Apr 28, 2023
6eb07aa
Remove mongo export files
aldavidson Jun 16, 2023
9c34535
Allow JsonImporter to import all files in a given dir, and add specs
aldavidson Jun 16, 2023
13757bc
Allow null timestamps on records imported from mongo
aldavidson Aug 30, 2023
a4e4e96
Fix issue that can cause doubly-nested description Hashes on PostgreSQL
aldavidson Aug 14, 2023
fbaa77d
Make JsonImporter only import gzipped files
aldavidson Aug 29, 2023
d5c3e84
Don't override default attributes with nil
richardTowers Aug 25, 2023
572a0fc
Speed up FindByPath.find on K8s/RDS environments
aldavidson Oct 12, 2023
175f5cc
Fix Integer field overflow on scheduled_publishing_delay_seconds
aldavidson Oct 16, 2023
f2f67e3
Don't dump the schema after running migrations in production
aldavidson Oct 17, 2023
624012b
Fix error when auth_bypass_ids is nil
aldavidson Oct 24, 2023
29ba644
Upgrade rails from 7.0.8 to 7.1.2
aldavidson Oct 31, 2023
a4f1d63
Fix logger custom field config
mike29736 Nov 23, 2023
9820d4b
Maintain compatibility with earlier nil-values fix from main (#1136)
aldavidson Dec 22, 2023
bf06dd0
Retain compatibility with #bbe2fda / PR #1152 on main
aldavidson Dec 22, 2023
7919408
update comment about exposing id outside the model
aldavidson Jan 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ on:
description: 'Commit, tag or branch name to deploy'
required: true
type: string
default: 'main'
environment:
description: 'Environment to deploy to'
required: true
Expand All @@ -18,14 +19,6 @@ on:
- staging
- production
default: 'integration'
ecrRepositoryName:
description: 'ECR repo name to push image to'
required: true
type: choice
options:
- content-store
- content-store-postgresql-branch
default: 'content-store'
release:
types: [released]

Expand All @@ -35,7 +28,7 @@ jobs:
name: Build and publish image
uses: alphagov/govuk-infrastructure/.github/workflows/build-and-push-image.yml@main
with:
ecrRepositoryName: ${{ inputs.ecrRepositoryName || 'content-store' }}
ecrRepositoryName: content-store
gitRef: ${{ inputs.gitRef || github.ref_name }}
secrets:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_GOVUK_ECR_ACCESS_KEY_ID }}
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/rspec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,9 @@ jobs:
name: Run RSpec
runs-on: ubuntu-latest
steps:
- name: Setup MongoDB
uses: alphagov/govuk-infrastructure/.github/actions/setup-mongodb@main
with:
version: 2.6
- name: Setup Postgres
id: setup-postgres
uses: alphagov/govuk-infrastructure/.github/actions/setup-postgres@main

- name: Checkout repository
uses: actions/checkout@v3
Expand All @@ -44,10 +43,12 @@ jobs:
- name: Initialize database
env:
RAILS_ENV: test
TEST_DATABASE_URL: ${{ steps.setup-postgres.outputs.db-url }}
run: bundle exec rails db:setup

- name: Run RSpec
env:
RAILS_ENV: test
TEST_DATABASE_URL: ${{ steps.setup-postgres.outputs.db-url }}
GOVUK_CONTENT_SCHEMAS_PATH: vendor/publishing-api/content_schemas
run: bundle exec rake spec
12 changes: 8 additions & 4 deletions .github/workflows/verify-pact.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,9 @@ jobs:
env:
RAILS_ENV: test
steps:
- name: Setup MongoDB
uses: alphagov/govuk-infrastructure/.github/actions/setup-mongodb@main
with:
version: 2.6
- name: Setup Postgres
id: setup-postgres
uses: alphagov/govuk-infrastructure/.github/actions/setup-postgres@main

- name: Checkout repository
uses: actions/checkout@v3
Expand All @@ -44,11 +43,14 @@ jobs:

- name: Initialize database
run: bundle exec rails db:setup
env:
TEST_DATABASE_URL: ${{ steps.setup-postgres.outputs.db-url }}

- name: Verify pact consumer version
if: inputs.pact_artifact == ''
env:
PACT_CONSUMER_VERSION: ${{ inputs.pact_consumer_version }}
TEST_DATABASE_URL: ${{ steps.setup-postgres.outputs.db-url }}
run: bundle exec rake pact:verify

- name: Download pact artifact
Expand All @@ -61,3 +63,5 @@ jobs:
- name: Verify pact artifact
if: inputs.pact_artifact != ''
run: bundle exec rake pact:verify:at[tmp/pacts/publishing_api-content_store.json]
env:
TEST_DATABASE_URL: ${{ steps.setup-postgres.outputs.db-url }}
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,5 @@

/coverage
/spec/reports

.byebug_history
3 changes: 0 additions & 3 deletions .rubocop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,3 @@ inherit_mode:
# See https://github.com/alphagov/rubocop-govuk/blob/main/CONTRIBUTING.md
# **************************************************************

# Mongoid doesn't support this
Rails/FindBy:
Enabled: false
9 changes: 2 additions & 7 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,8 @@ RUN bootsnap precompile --gemfile .

FROM $base_image

# TODO: remove this temporary MongoDB package once we no longer need mongoexport (once the migration to Postgres is done).
ARG mongo_package=mongodb-database-tools-ubuntu2204-x86_64-100.7.2.deb
ARG mongo_package_repo=https://fastdl.mongodb.org/tools/db
WORKDIR /tmp
RUN curl -LSsf "${mongo_package_repo}/${mongo_package}" --output "${mongo_package}" && \
apt-get install -y --no-install-recommends "./${mongo_package}" && \
rm -fr /tmp/*
# Need psql for importing from Mongo exports
RUN install_packages postgresql-client

ENV GOVUK_APP_NAME=content-store

Expand Down
7 changes: 3 additions & 4 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
source "https://rubygems.org"

gem "rails", "7.0.8"
gem "rails", "7.1.2"

gem "bootsnap", require: false
gem "deepsort"
gem "gds-api-adapters"
gem "gds-sso"
gem "govuk_app_config"
gem "mongo", "2.15.0" # Later releases require Mongo >= 3.6
gem "mongoid"
gem "pg"
gem "plek"
gem "rack-cache"
gem "uuidtools"
Expand All @@ -17,7 +16,7 @@ gem "whenever", require: false
group :development, :test do
gem "ci_reporter_rspec"
gem "climate_control"
gem "database_cleaner-mongoid"
gem "database_cleaner-active_record"
gem "factory_bot"
gem "govuk_schemas"
gem "govuk_test"
Expand Down
Loading