SkellySubs 💀💬

Browser-based multilingual video translation and subtitling tool

Important

This project is still in early development and may not be fully functional yet. Expect broken things and missing features 😌

Overview

Skellysubs provides in-browser video translation and subtitling capabilities. Video processing is handled client-side using ffmpeg.wasm, keeping server load light enough to wrangle significant-ish traffic on a free-tier GCP Cloud Run instance 🤞😅

Demo videos

Original Announcement video

2025-01-27-jsm-video_subtitled.mp4

Roughtly how it works

2e69e2563cb2d378200eb4afdb0086d2.mp4

Development Setup

Backend

1. Install uv: https://docs.astral.sh/uv/getting-started/installation/
2. Create and activate virtual environment:

uv venv
.venv/Scripts/activate  # Windows
source .venv/bin/activate  # Unix

3. Install dependencies:

uv sync

4. Run the server:

python skellysubs/__main__.py

The API documentation will be available at http://localhost:8080

Frontend

1. Navigate to the UI directory:

cd skellysubs-ui

2. Install dependencies:

npm install

3. Start development server:

npm run dev

Deployment

Commits to the production branch automatically trigger deployment to Google Cloud Run.

Data Privacy & Usage

• No video, audio, or translation data is stored on our servers
• Translations are processed through OpenAI's API and subject to their standard privacy policy
• All video processing occurs in your browser (using a WebAssembly version of ffmpeg bundled along with the webpage)
• OpenAI API costs are currently covered by the FreeMoCap Foundation as a service to the community
• If usage grows beyond our free-tier infrastructure capacity or token costs become prohibitive... we'll figure something out ❤️

Docker

For local containerized testing:

docker build -t skellysubs . && docker run -p 8080:8080 --name skellysubs-docker skellysubs

Software Architecture overview

[!WARNING] This section was generated by feeding the repository map into an LLM (deepseek or claude 3.5 Sonnet), may not be totally accurate.

SkellySubs Architecture Overview

Overall Architecture

A full-stack web application with:

• Python FastAPI backend
• React/TypeScript frontend
• Redux state management
• FFmpeg.wasm client-side video processing
• Containerized deployment (Docker + Cloud Run)

Key Architectural Components

Client-Side (skellysubs-ui/)

see - typescript App.tsx

• Core Stack: React + TypeScript + Vite + TailwindCSS
• State Management:
  • Redux Toolkit with slices for processing status, subtitles, logs
  • Thunks for async operations (file processing, API calls)
• Video Processing:
  • FFmpeg.wasm integration via context providers
  • Subtitle editor components with timeline visualization
• UI System:
  • Configurable panel layouts
  • Multi-stage processing workflow
  • Real-time logging terminal

Backend Service (skellysubs/)

see - server_manager.py

• Core Framework: FastAPI with REST/WS endpoints
• Key Modules:
  • AI Service integration (OpenAI/HuggingFace/Ollama)
  • Video processing pipeline (transcription/translation/subtitles)
• Main Components:
  • core/: Business logic
  • api/: Endpoint routers
  • ai_clients/: AI Provider strategies

Notable Architectural Patterns

Client-Side Video Processing

see - useFfmpeg.ts

• FFmpeg.wasm for browser-based processing
• Avoids server-side resource usage
• Enables offline-capable workflows

AI Service Abstraction

see - ai_client_strategy.py

• Strategy pattern implementation
• Unified interface for multiple providers
• Easy integration of new AI services
• Currently focused on OpenAI API, but has the beginnings an Ollama, HuggingFace, Deepseek, and others

Deployment Architecture

see Dockerfile

• Single container deployment:
  • Python backend (UVicorn)
  • Pre-built React frontend
  • FFmpeg.wasm dependencies
• Cloud Run optimized
• Stateless design:
  • Client-side state management
  • No persistent storage

Key Dependencies

Frontend

see - package.json

• @ffmpeg/ffmpeg - WASM video processing
• @mui/material - UI components
• react-resizable-panels - Layout system

Backend

see - pyproject.toml

• fastapi - Web framework
• openai - AI integrations