OParl Index

Scrapes data from OParl parliamentary information APIs and serves it via a read-only REST API.

Built with FastAPI, SQLModel, PostgreSQL, and Alembic.

Setup

Requires Docker and Docker Compose.

docker-compose up --build
docker-compose exec app alembic upgrade head

• Frontend: http://localhost:8000
• API docs: http://localhost:8000/docs
• PostgreSQL: localhost:5432

Scraping Data

The scraper walks the OParl navigation chain (System → Bodies → entity lists) and stores Organizations, People, Memberships, Meetings, Agenda Items, Papers, Consultations, and Files.

# Full scrape
uv run python -m backend.scripts.oparl_scraper https://oparl.stadt-muenster.de/system

# Only recent items (stops at items older than the given date)
uv run python -m backend.scripts.oparl_scraper https://oparl.stadt-muenster.de/system --since 2026-01-01

Database Statistics

uv run python -m backend.scripts.db_stats

Prints row counts and oldest/newest created timestamps per table.

uv run python -m backend.scripts.db_check

Checks for orphaned references, missing fields, broken JSON array refs, and duplicate names.

API Endpoints

All list endpoints support skip and limit query parameters. Detail endpoints use {id:path} to accept OParl URLs as IDs.

• GET /organizations/, GET /organizations/{id}
• GET /persons/, GET /persons/{id}
• GET /memberships/, GET /memberships/{id}
• GET /meetings/, GET /meetings/{id}
• GET /agenda-items/, GET /agenda-items/{id}
• GET /papers/, GET /papers/{id}
• GET /consultations/, GET /consultations/{id}
• GET /files/, GET /files/{id}

Development

# Install dependencies locally
uv sync

# Run migrations
docker-compose exec app alembic upgrade head

# Run tests
docker-compose exec app pytest

License

MIT