Skip to content

Commit c51dcf4

Browse files
Kyrylo-KtlKyrylo-Ktl
authored
[Feature] Main worksheet implementation (#1)
1 parent a217216 commit c51dcf4

12 files changed

+1319
-1
lines changed

Diff for: ‎.env.example

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
POSTGRES_DB=
2+
POSTGRES_USER=
3+
POSTGRES_PASSWORD=
4+
5+
PGADMIN_DEFAULT_EMAIL=
6+
PGADMIN_DEFAULT_PASSWORD=
7+
8+
POSTGRESQL_HOST=postgres
9+
POSTGRESQL_PORT=5432

Diff for: ‎.github/workflows/lint-yaml.yaml

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
name: Run yaml linter
3+
on: [push] # yamllint disable-line rule:truthy
4+
jobs:
5+
lintAllTheThings:
6+
runs-on: ubuntu-latest
7+
steps:
8+
- uses: actions/checkout@v3
9+
- name: yaml-lint
10+
uses: ibiqlik/action-yamllint@v3

Diff for: ‎.gitignore

+162
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
6+
# C extensions
7+
*.so
8+
9+
# Distribution / packaging
10+
.Python
11+
build/
12+
develop-eggs/
13+
dist/
14+
downloads/
15+
eggs/
16+
.eggs/
17+
lib/
18+
lib64/
19+
parts/
20+
sdist/
21+
var/
22+
wheels/
23+
share/python-wheels/
24+
*.egg-info/
25+
.installed.cfg
26+
*.egg
27+
MANIFEST
28+
29+
# PyInstaller
30+
# Usually these files are written by a python script from a template
31+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
32+
*.manifest
33+
*.spec
34+
35+
# Installer logs
36+
pip-log.txt
37+
pip-delete-this-directory.txt
38+
39+
# Unit test / coverage reports
40+
htmlcov/
41+
.tox/
42+
.nox/
43+
.coverage
44+
.coverage.*
45+
.cache
46+
nosetests.xml
47+
coverage.xml
48+
*.cover
49+
*.py,cover
50+
.hypothesis/
51+
.pytest_cache/
52+
cover/
53+
54+
# Translations
55+
*.mo
56+
*.pot
57+
58+
# Django stuff:
59+
*.log
60+
local_settings.py
61+
db.sqlite3
62+
db.sqlite3-journal
63+
64+
# Flask stuff:
65+
instance/
66+
.webassets-cache
67+
68+
# Scrapy stuff:
69+
.scrapy
70+
71+
# Sphinx documentation
72+
docs/_build/
73+
74+
# PyBuilder
75+
.pybuilder/
76+
target/
77+
78+
# Jupyter Notebook
79+
.ipynb_checkpoints
80+
81+
# IPython
82+
profile_default/
83+
ipython_config.py
84+
85+
# pyenv
86+
# For a library or package, you might want to ignore these files since the code is
87+
# intended to run in multiple environments; otherwise, check them in:
88+
# .python-version
89+
90+
# pipenv
91+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
92+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
93+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
94+
# install all needed dependencies.
95+
#Pipfile.lock
96+
97+
# poetry
98+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
99+
# This is especially recommended for binary packages to ensure reproducibility, and is more
100+
# commonly ignored for libraries.
101+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
102+
#poetry.lock
103+
104+
# pdm
105+
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
106+
#pdm.lock
107+
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
108+
# in version control.
109+
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
110+
.pdm.toml
111+
.pdm-python
112+
.pdm-build/
113+
114+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
115+
__pypackages__/
116+
117+
# Celery stuff
118+
celerybeat-schedule
119+
celerybeat.pid
120+
121+
# SageMath parsed files
122+
*.sage.py
123+
124+
# Environments
125+
.env
126+
.venv
127+
env/
128+
venv/
129+
ENV/
130+
env.bak/
131+
venv.bak/
132+
133+
# Spyder project settings
134+
.spyderproject
135+
.spyproject
136+
137+
# Rope project settings
138+
.ropeproject
139+
140+
# mkdocs documentation
141+
/site
142+
143+
# mypy
144+
.mypy_cache/
145+
.dmypy.json
146+
dmypy.json
147+
148+
# Pyre type checker
149+
.pyre/
150+
151+
# pytype static type analyzer
152+
.pytype/
153+
154+
# Cython debug symbols
155+
cython_debug/
156+
157+
# PyCharm
158+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
159+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
160+
# and can be added to the global gitignore or merged into this file. For a more nuclear
161+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
162+
.idea/

Diff for: ‎README.md

+131-1
Original file line numberDiff line numberDiff line change
@@ -1 +1,131 @@
1-
# postgres-worksheet
1+
[![Docker](https://img.shields.io/badge/Docker-%230db7ed.svg?logo=docker&logoColor=white)](https://www.docker.com/)
2+
[![Docker Compose](https://img.shields.io/badge/Docker%20Compose-%230db7ed.svg?logo=docker&logoColor=white)](https://docs.docker.com/compose/)
3+
[![PostgreSQL](https://img.shields.io/badge/PostgreSQL-%23316192.svg?logo=postgresql&logoColor=white)](https://www.postgresql.org/)
4+
[![PgBouncer](https://img.shields.io/badge/PgBouncer-%23316192.svg?logo=postgresql&logoColor=white)](https://www.pgbouncer.org/)
5+
[![pgAdmin](https://img.shields.io/badge/pgAdmin-%23316192.svg?logo=postgresql&logoColor=white)](https://www.pgadmin.org/)
6+
<br>
7+
![OS](https://img.shields.io/badge/OS-Linux%2C%20Windows%2C%20MacOS-0078D4)
8+
![CPU](https://img.shields.io/badge/CPU-x86%2C%20x64%2C%20ARM%2C%20ARM64-FF8C00)
9+
![GitHub last commit](https://img.shields.io/github/last-commit/Kyrylo-Ktl/advent-of-code)
10+
11+
# Postgres worksheet
12+
13+
This project provides a preconfigured environment for working with PostgreSQL database infrastructure using Docker Compose.
14+
It includes the following services:
15+
16+
+ [PostgreSQL](https://www.postgresql.org/) - the primary database service with custom configuration;
17+
+ [PgBouncer](https://www.pgbouncer.org/) - a lightweight connection pooler for PostgreSQL;
18+
+ [pgAdmin](https://www.pgadmin.org/) - a web-based PostgreSQL database management tool.
19+
20+
21+
## Architecture
22+
23+
### Components
24+
25+
![Architecture diagram](https://github.com/user-attachments/assets/d20e472b-6c2d-4561-b31c-f8cd13c6a020)
26+
27+
### Features
28+
29+
This preconfigured project environment provides the following features:
30+
31+
+ **PostgreSQL**
32+
+ Custom configuration through a mounted `postgresql.conf` file;
33+
+ Automated data model creation with scripts from mounted `docker-entrypoint-initdb.d` folder;
34+
+ Persistent data model storage via volumes;
35+
+ Resources optimal setup - CPU (2) and memory (4G);
36+
+ Health checks to ensure the database is ready before other services start;
37+
+ Isolated in the internal Docker network;
38+
39+
40+
+ **PgBouncer**;
41+
+ Connection pooling for database to optimize resource usage;
42+
+ SCRAM-SHA-256 authentication for secure connections;
43+
+ Resources optimal setup - CPU (0.5) and memory (1G);
44+
+ Accessible via the external Docker network;
45+
46+
47+
+ **pgAdmin**;
48+
+ Persistent configurations storage via volumes;
49+
+ Resources optimal setup - CPU (0.5) and memory (1G);
50+
51+
### Data model
52+
53+
In this project, an e-commerce data model is implemented by default to manage customers, products and related orders.
54+
The data model is created automatically when the database is first started and is populated with synthetic records:
55+
56+
+ Customers - [table](config/postgresql/scripts/01-create-0001-customers-table.sql) with information about 10,000 [randomly generated](config/postgresql/scripts/02-fill-0001-customers-table.sql) customers;
57+
+ Products - [table](config/postgresql/scripts/01-create-0002-products-table.sql) with information about 100,000 [randomly generated](config/postgresql/scripts/02-fill-0002-products-table.sql) products;
58+
+ Orders - [table](config/postgresql/scripts/01-create-0003-orders-table.sql) with information about 1,000,000 [randomly generated](config/postgresql/scripts/02-fill-0003-orders-table.sql) orders;
59+
60+
![postgres-worksheet-er](https://github.com/user-attachments/assets/dd8c7574-8d83-4bc8-b134-2d17b6686fa3)
61+
62+
63+
## Usage
64+
65+
### Prerequisites
66+
67+
Make sure you have [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) installed on your machine.
68+
69+
### Setup
70+
71+
Project uses an `.env` file with environment variables to manage configurations and tool settings.
72+
The file should be created using the `.env.example` file provided in the root directory using the following command:
73+
74+
75+
```shell
76+
cp .env.example .env
77+
```
78+
79+
And then filled with the corresponding values:
80+
81+
```dotenv
82+
POSTGRES_DB=<database-name>
83+
POSTGRES_USER=<postgres-user>
84+
POSTGRES_PASSWORD=<postgres-password>
85+
86+
PGADMIN_DEFAULT_EMAIL=<pgadmin-email>
87+
PGADMIN_DEFAULT_PASSWORD=<pgadmin-password>
88+
89+
POSTGRESQL_HOST=postgres
90+
POSTGRESQL_PORT=5432
91+
```
92+
93+
### Run
94+
95+
Build and start the database services infrastructure by running:
96+
97+
```shell
98+
docker compose up -d --build
99+
```
100+
101+
> [!NOTE]
102+
> First run will take some time, as all necessary docker images will be loaded and the data model will be created and populated with synthetic records.
103+
104+
Check that all services are successfully started by running:
105+
106+
```shell
107+
docker compose ps
108+
```
109+
110+
To stop all running services, execute:
111+
112+
```shell
113+
docker compose down
114+
```
115+
116+
> [!NOTE]
117+
> After stopping the services, the data will not be deleted and will be available when the services are restarted.
118+
119+
If you want to stop the services and remove all associated data use:
120+
121+
```shell
122+
docker compose down --volumes
123+
```
124+
125+
> [!WARNING]
126+
> This command deletes all service data (e.g. data model, stored passwords and configurations).
127+
128+
129+
## License
130+
131+
This project is licensed under the [MIT License](LICENSE).

0 commit comments

Comments
 (0)