Crypto Reconciliation Platform

A full-stack platform for ingesting, normalizing, and reconciling cryptocurrency trades across multiple exchanges (Binance, Coinbase, Kraken).

Architecture

┌──────────────────┐
│    Dashboard      │  (Next.js - port 3002)
└────────┬─────────┘
         │
┌────────▼─────────┐     ┌──────────────────┐
│   API Gateway     │◄────│   Ingestion       │
│   (port 3000)     │     │   (port 3001)     │
└────────┬─────────┘     └──────┬───────────┘
         │                       │
┌────────▼─────────┐     ┌──────▼───────────┐
│   PostgreSQL      │     │  Binance API      │
│   (port 5432)     │     │  Coinbase API     │
└──────────────────┘     │  Kraken API       │
                          └──────────────────┘

Tech Stack

• Backend: NestJS, TypeScript, Prisma ORM, PostgreSQL
• Frontend: Next.js, React, Tailwind CSS
• Infrastructure: Docker Compose, GitHub Actions CI
• Monorepo: pnpm workspaces

Project Structure

crypto-reconciliation-platform/
├── apps/
│   └── dashboard/              # Next.js web dashboard
├── packages/
│   └── shared/                 # Shared types & DTOs
├── services/
│   ├── api-gateway/            # REST API — CRUD, reconciliation
│   └── ingestion/              # Multi-exchange trade ingestion
├── docker-compose.yml
└── .github/workflows/ci.yml

Features

Multi-Exchange Ingestion

• Fetches real trades from Binance, Coinbase, and Kraken APIs
• Unified symbol format (BTC-USDT) across all exchanges
• Abstract BaseExchangeService class — each exchange implements only getRecentTrades() and normalizeTrades()
• Upsert logic to avoid duplicate trades
• Scheduled ingestion via cron (default: every 5 minutes)
• Error handling with proper HTTP status codes

Trade Management

• Full CRUD API for trades
• Filtering by exchange, base asset, quote asset, side
• Validation via DTOs and class-validator

CSV Reconciliation

• Upload a CSV file and compare with database trades
• Two-step matching: exact match by externalId + exchange, then fuzzy match by timestamp proximity (±5s window)
• Confidence scoring (0-100%) with breakdown by field: amount (30pts), price (30pts), side (20pts), timestamp (20pts)
• Trades scoring ≥80% are matched, below are mismatched, no match found are missing
• Saves reconciliation reports with confidence scores and match type (exact / fuzzy) to database
• Report history with detail view

Web Dashboard

• Home — Overview with trade count, report count, exchange count
• Trades — Table of all ingested trades with color-coded BUY/SELL
• Ingestion — Select exchange and symbol, trigger ingestion from UI
• Reports — Upload CSV, view reconciliation history, drill into report details

Getting Started

Prerequisites

• Node.js 20+
• pnpm
• Docker & Docker Compose

Run with Docker

# Start everything (API Gateway + Ingestion + Dashboard + PostgreSQL)
docker-compose up --build

• API Gateway: http://localhost:3000
• Ingestion Service: http://localhost:3001
• Dashboard: http://localhost:3002

API Endpoints

Trades

GET    /trades                  # List all trades (supports filters)
GET    /trades/:id              # Get trade by ID
POST   /trades                  # Create trade
POST   /trades/upsert           # Create or update trade
PUT    /trades/:id              # Update trade
DELETE /trades/:id              # Delete trade

Ingestion

GET    /binance/trades?symbol=BTC-USDT      # Fetch trades from Binance
POST   /binance/ingest?symbol=BTC-USDT      # Ingest Binance trades to DB
GET    /coinbase/trades?symbol=BTC-USD       # Fetch trades from Coinbase
POST   /coinbase/ingest?symbol=BTC-USD       # Ingest Coinbase trades to DB
GET    /kraken/trades?symbol=BTC-USD         # Fetch trades from Kraken
POST   /kraken/ingest?symbol=BTC-USD         # Ingest Kraken trades to DB

Reconciliation

POST   /reconciliation/upload                # Upload CSV and reconcile
GET    /reconciliation/reports               # List all reports
GET    /reconciliation/reports/:id           # Get report with details

CSV Format

externalId,exchange,amount,price,side
6108653502,binance,0.04330000,71701.74000000,SELL

Key Design Decisions

• Abstract class for exchanges — BaseExchangeService provides saveTrades() and ingestTrades(). Each exchange only implements API-specific logic. Adding a new exchange takes ~30 lines of code.
• Unified symbol format — All exchanges use BASE-QUOTE (e.g., BTC-USDT). Each service converts to the exchange's native format internally.
• Tolerance-based reconciliation — Compares amounts/prices as floats with ±0.0001 tolerance instead of exact string matching, accounting for decimal formatting differences.
• Reconciliation kept in API Gateway — Direct database access avoids unnecessary inter-service calls. Pragmatic choice given project scope.
• Upsert for deduplication — Composite unique constraint on (externalId, exchange) prevents duplicate trades on re-ingestion.
• gRPC for inter-service communication — The ingestion service communicates with the API Gateway via gRPC instead of HTTP, using a shared protobuf contract (packages/shared/proto/trades.proto).

Testing

# Unit tests
cd services/api-gateway && npm test

Tests cover TradeService (CRUD operations) and TradeController (HTTP layer, 404 handling).

End-to-End Tests

E2E tests run against a real PostgreSQL database — no mocks. They test the full HTTP request → NestJS → Prisma → DB chain.

# Start the test database (port 5433, isolated from dev)
docker compose -f docker-compose.test.yml up -d

# Run e2e tests
DATABASE_URL="postgresql://antonyjin:password@localhost:5433/crypto_recon_test" pnpm test:e2e

# Stop the test database
docker compose -f docker-compose.test.yml down

Coverage (24 tests):

• Health — Server boot, /health endpoint
• Trades — Full CRUD, query filters, upsert deduplication, validation (400), not found (404)
• Reconciliation — CSV upload with matched/mismatched/missing detection, mixed results, report persistence and retrieval

CI runs automatically on every push via GitHub Actions.

Scheduled Ingestion

The ingestion service automatically fetches trades from all exchanges on a cron schedule (default: every 5 minutes).

Configure via environment variables:

Variable	Default	Description
INGESTION_CRON	*/5 * * * *	Cron expression for ingestion frequency
BINANCE_SYMBOLS	BTC-USDT,ETH-USDT	Comma-separated symbols to ingest from Binance
COINBASE_SYMBOLS	BTC-USD,ETH-USD	Comma-separated symbols to ingest from Coinbase
KRAKEN_SYMBOLS	BTC-USD,ETH-USD	Comma-separated symbols to ingest from Kraken

Roadmap

• End-to-end tests
• Scheduled ingestion (cron jobs)
• gRPC communication between services
• Advanced reconciliation (timestamp matching, confidence scoring)
• Authentication (JWT)

License

MIT