Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(agora): update agora-mongo image to include Agora dataset #2440

Closed

Conversation

tschaffter
Copy link
Member

@tschaffter tschaffter commented Jan 23, 2024

Description

Update the docker image generated by the project agora-mongo so that it includes a specific version of the Agora dataset stored on Synapse (~500 MB).

Preview

Build the docker image:

nx build-image agora-mongo

Run the container locally:

nx serve-detach agora-mongo

Stop the container:

docker rm -f agora-mongo

cc @sagely1 @JessterB

@tschaffter
Copy link
Member Author

@sagely1 Here is the code that we wrote together + cleanup.

@tschaffter
Copy link
Member Author

@sagely1 Something we may have overlooked: when building the docker image, MongoDB is not running, so the dataset files can't be load with a MongoDB CLI. Is there a way to export the Agora dataset files into a file format so that MongoDB can understand these data when it's instance is starting? Other the data processing may need to be done outside of the MongoDB Dockerfile.

@tschaffter
Copy link
Member Author

Closing this PR that actually doesn't need to be merged. The code of this PR can always be retrieved from this closed PR.

Note

I'm have recently merged PRs that bring a similar feature to iAtlas, i.e. the seeding of a Postgres DB using a container that pulls data from Synapse, validate it against a schema (with Schematic) and push the data to the DB.

cc: @sagely1

@tschaffter tschaffter closed this Feb 13, 2024
@tschaffter tschaffter deleted the agora-mongo-with-data branch July 22, 2024 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants