Your private AI assistant that knows your emails, projects, writing style, and remembers your conversations. Runs 100% locally on your hardware.
| Feature | Description |
|---|---|
| Conversation Memory | AI remembers past conversations - learns your name, preferences, context |
| Memory Viewer | Browse, search, and delete specific memories |
| Customizable Personality | Name your AI, set its personality and traits |
| Document Upload | Drag & drop PDFs, TXT, MD files directly in the web UI |
| Audio/Video Transcription | Upload MP4, MP3, WAV files - transcribed with Whisper |
| Meeting Summarization | Auto-extract key points, action items, decisions from transcripts |
| Web Search | Search the internet with "search: query" (toggle in Settings) |
| Image/Screenshot OCR | Extract text from images with Tesseract OCR |
| Export/Import | Backup and restore all conversations and settings |
| Personal Knowledge Base | 61,000+ documents from emails, transcripts, blogs, datasheets |
| Your Writing Style | LoRA fine-tuned on your emails - writes like you |
| Performance Metrics | TTFT, tokens/sec, and total time on every response |
| Gmail Integration | OAuth-based Send & Draft buttons - email directly from the UI |
| Streaming Responses | Real-time word-by-word output |
| Hybrid RAG Search | BM25 + semantic + reranking for best results |
| Auto-Remember | Toggle to automatically save conversations to memory |
| Meeting Summarizer | Transcribe audio/video with Whisper + summarize |
| Document Generator | Create specs, proposals, reports from your knowledge |
| Datasheet Ingestion | Ingest PDFs - reference manuals, app notes |
| Daily Digest | Automated morning summary of your AI activity |
| Web UI | Clean interface with settings panel and conversation sidebar |
- GPU: NVIDIA RTX 3080+ (16GB+ VRAM recommended)
- RAM: 32GB+
- Storage: 50GB+ for models and knowledge base
- OS: Ubuntu 22.04+ (or similar Linux)
# Clone the repo
git clone https://github.com/kylefoxaustin/personal-ai-framework.git
cd personal-ai-framework
# Run setup (installs dependencies, creates config)
./setup.sh
# Download model (Qwen 2.5 14B Q4)
./run.sh download-model
# Start services
./run.sh start
# Open web UI
xdg-open http://localhost:8765Give your AI a name and personality! Click ⚙️ Settings in the web UI:
- AI Name: "Skippy", "Jarvis", "Friday", or whatever you want
- Personality Prompt: "You are Skippy, a sarcastic but helpful AI who loves sci-fi references..."
The personality is injected into every prompt automatically.
Your AI remembers past conversations:
- Auto-Remember: Toggle to automatically save all chats to memory
- Manual Save: Click 💾 on any conversation to save it (turns to 🧠 when saved)
- Memory Viewer: Click 🧠 in header to browse/delete specific memories
- Clear Memory: 🗑️ button to wipe all memories (with confirmation)
- Memory Search: AI searches past conversations when answering
Ask "What's my name?" or "What did we discuss about X?" - the AI remembers!
Upload documents, audio/video, and images to your knowledge base from a single dedicated panel:
- Click 📤 Upload in the header
- Choose a tab: 📄 Documents, 🎤 Audio/Video, or 📷 Images
- Drag & drop files, click to browse, or use 📂 Upload Folder to select an entire folder
- Multi-file and folder uploads show a progress bar with per-file results
- Files are automatically processed and indexed in ChromaDB
- Select document type (General, Datasheet, Transcript, Email)
- Supported formats: PDF, TXT, MD
- Enter a recording title (optional)
- Toggle meeting summary generation (key points, action items, decisions)
- Whisper transcribes using CPU to avoid VRAM conflicts
- Results show full summary, key points, action items, and decisions
- Copy Summary / Copy Transcript / Send Email / Save Draft buttons
- Collapsible full transcript view
- Transcripts saved to
~/knowledge/transcripts/and searchable by Skippy! - Supported formats: MP4, MP3, WAV, M4A, WEBM, MKV, MOV, AVI, OGG, FLAC
- Enter an image title (optional)
- Optional summarization of extracted text (checkbox)
- Tesseract extracts text and auto-ingests to ChromaDB
- Copy Text / Copy Summary / Send Email / Save Draft buttons
- Perfect for whiteboard photos, screenshots, scanned docs, or book pages!
- Supported formats: PNG, JPG, GIF, BMP, TIFF, WEBP
Never lose your AI's memories:
- 📤 Export Conversations: Download all chats as JSON
- 💾 Full Backup: Export conversations + settings in one file
- 📥 Import Backup: Restore from any backup file
Find these in ⚙️ Settings → Backup & Restore
personal-ai-framework/
├── docker/ # Docker configurations
├── knowledge/ # Your knowledge base (mounted volume)
│ ├── documents/ # General documents
│ ├── emails/ # Exported emails (.eml, .mbox)
│ ├── transcripts/ # Meeting/video transcripts
│ ├── datasheets/ # PDF datasheets & manuals
│ └── writing/ # Your blog posts, articles
├── pipeline/ # Core Python services
│ ├── llm_server.py # FastAPI LLM server
│ ├── rag_service.py # ChromaDB RAG
│ ├── advanced_rag.py # Hybrid search + reranking
│ ├── conversation_store.py # SQLite conversation storage
│ ├── memory_service.py # Memory ingestion & search
│ ├── meeting_summarizer.py
│ ├── doc_generator.py
│ ├── email_service.py
│ ├── daily_digest.py
│ └── settings_manager.py
├── web/ # Web UI
├── training/ # LoRA fine-tuning
└── run.sh # Main CLI
./run.sh start # Start all services
./run.sh stop # Stop all services
./run.sh status # Check service status
./run.sh logs # View logs./run.sh sync # Sync knowledge base
./run.sh ingest-datasheets # Ingest PDF datasheets
./run.sh ingest-emails # Ingest email exports# Email drafting
./run.sh email draft "project status update" -t recipient@email.com
./run.sh email draft "follow up on meeting" --open # Opens in email client
# Document generation
./run.sh generate technical_spec "i.MX display system"
./run.sh generate project_proposal "embedded AI project"
./run.sh generate status_report "Q1 development progress"
# Meeting summarization
./run.sh summarize recording.mp4 -t "Team Meeting" --add-to-kb
# Daily digest
./run.sh digest # Print digest
./run.sh digest -m mailto -t you@email.com # Open in email client
./run.sh digest --schedule 08:00 -t you@email.com # Schedule dailyAccess at http://localhost:8765
- Conversation Sidebar: All your chats, with save/delete options
- Chat: Ask questions, get answers from your knowledge base
- Email Drafting: Type "Draft an email about..." → Copy/Send/Draft buttons
- Streaming: Responses appear word-by-word in real-time
- Performance Metrics: See TTFT, tok/s, and total time on every response
- ✨ New: Start a new conversation
- 📤 Upload: Upload documents, audio/video, and images
- 🔄 Sync: Sync knowledge base (green dot = auto-sync enabled)
- 🧠 Memory: View/delete stored memories
- 🌐 Web: Toggle web search on/off (green = enabled)
- ⚙️ Settings: Personality, backup, and more
- All buttons have hover tooltips
- ✍️ Write like me: Use your personal writing style
- 🧠 Auto-remember: Automatically save conversations to memory
- AI Personality: Name and personality prompt
- Backup & Restore: Export/import conversations and settings
- Email Providers: Gmail OAuth connection
- Daily Digest: Schedule and email settings
- Auto-Sync: Knowledge base sync interval
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Web UI │────▶│ LLM Server │────▶│ Mixtral │
│ :3000 │ │ :8080 │ │ 8x7B Q4 │
└─────────────┘ └──────┬──────┘ └─────────────┘
│
┌───────────┼───────────┐
▼ ▼ ▼
┌───────────┐ ┌─────────┐ ┌─────────┐
│ ChromaDB │ │ SQLite │ │ Gmail │
│ (RAG) │ │ (Chats) │ │ API │
│ 61K docs │ │ Memory │ │ │
└───────────┘ └─────────┘ └─────────┘
Settings are stored in ~/.personal-ai/settings.json and can be configured via:
- Web UI Settings panel (⚙️ button)
- CLI commands
- Direct file editing
Key settings:
- AI Personality: Name and system prompt
- Daily Digest: Enable, time, email address
- Auto-Sync: Enable, interval (1/4/12/24 hours)
- Context Window: 8K/16K/32K tokens
Tested on RTX 5090 (32GB VRAM) running Qwen 2.5 7B v4 (production fine-tune):
- TTFT: ~0.03-0.12s (time to first token)
- Throughput: 180-215 tokens/sec sustained (183.9 median)
- Peak VRAM: ~5.5 GB at 16K context
- Context: 16K tokens default (32K max)
- Knowledge Base: 61,500+ documents
- Datasheet Ingestion: 37 PDFs (6,129 chunks) in ~5 min
Production model upgraded from Qwen 2.5 14B to Qwen 2.5 7B v4 in the v5.10.0 methodology campaign — full reasoning at docs/skippy-white-paper.md and docs/recipe-taxonomy.md. 14B candidate scored higher on headline (76.2% vs 73.8%) but fabricated fictional peripherals; production trades headline for safety calibration.
| Version | Highlights |
|---|---|
| v5.10.0 | Methodology + research checkpoint — gotcha-#7 closure with N=7 two-factor recipe predictor (ceiling reasoning OR family-match); cross-family v4 campaign (Gemma 9B, Mistral 7B, Llama 3.1 8B, Yi-1.5-9B, Phi-4 14B); Qwen-family substring-grader bias finding + eval/regrade_semantic.py semantic-regrade tooling; voice-metrics tooling (eval/voice_metrics.py); recipe-taxonomy framework (docs/recipe-taxonomy.md); Skippy white paper (docs/skippy-white-paper.md); private-anchor secrets spec for cross-app NPU measurement architecture (docs/private_anchor_secrets_spec.md); 29-slide use-case deck + presenter script for tech-mgmt audiences; cross-deck reference pattern with Keyhole. Production = Qwen 2.5 7B v4 (73.8% pass / voice ✓ / safety ✓); 14B v4 candidate documented but not shipped (fabrication). No application-feature changes — engine is unchanged. |
| v5.9.2 | Auth middleware bypasses CORS preflight (OPTIONS) — browser was getting 401 on every cross-origin preflight, silently hiding per-user data |
| v5.9.1 | Multi-user polish — /health is public (no more spurious session wipes), header wraps cleanly when buttons don't fit, 401 handler only logs out when a token was actually sent |
| v5.9.0 | Multi-user support — bcrypt login, per-user data isolation under users/<name>/, admin can add/remove users and reset passwords, auto-migrate single-user installs into users/kyle/ |
| v5.8.0 | Agent platform + observability — Google Calendar, write-capable tools (sandboxed), reminders, multi-step agent loop, RLHF 👍/👎, training dashboard, selective training exclusion, mobile-friendly UI, Phone Access helper, Prometheus /metrics + Grafana dashboard |
| v5.7.0 | Smarter RAG, light mode, streaming TTS, training hardening — multi-doc synthesis, conv-aware RAG, citation score breakdown, theme toggle, conversation export |
| v5.6.0 | Model upgrade — Qwen 2.5 14B replaces Mixtral 8x7B, ChatML format, 32K context, unified training |
| v5.5.0 | Voice mode — push-to-talk mic input with Whisper STT, browser TTS for spoken responses |
| v5.4.0 | Agent tools — Skippy can read files, list directories, search web, and check git status (sandboxed, read-only) |
| v5.3.0 | Multi-turn [INST] format, RAG query rewriting, conversation search, model auto-restore, auto-training schedule, training venv, DuckDuckGo retry |
| v5.2.0 | Instruct template fix (facts/web search now reliable), integration test suite, training data dedup fix, RAG hybrid search fix |
| v5.1.0 | Learned Facts layer, inference stability fixes, model warmup, Skippy personality |
| v5.0.0 | Automated LoRA retraining — Train Now button, GPU handoff, GGUF conversion pipeline |
| v4.1.0 | Bulk upload (multi-file + folder), enhanced results UI (copy/email/summary), web search toggle, auto-sync indicator, tooltips |
| v4.0.0 | UI Refactor - Dedicated Upload panel with tabbed Documents/Audio/Images |
| v3.9.0 | Web Search - DuckDuckGo integration with privacy toggle |
| v3.8.0 | Meeting Summarization - key points, action items, decisions |
| v3.7.0 | Image/Screenshot OCR - Tesseract text extraction |
| v3.6.0 | Audio/Video Transcription - Whisper-powered web upload |
| v3.5.0 | Document Upload - drag & drop PDFs/TXT/MD in web UI |
| v3.4.0 | Export/Import - backup & restore conversations and settings |
| v3.3.1 | Bug fixes: auto-remember, assistant message saving |
| v3.3.0 | Memory Viewer - browse and delete specific memories |
| v3.2.0 | AI Personality (custom name/prompt), Performance metrics (TTFT, tok/s) |
| v3.1.0 | Auto-remember toggle, Clear memory button |
| v3.0.0 | Conversation Memory - AI remembers past chats! |
| v2.1.0 | Gmail OAuth, Send/Draft email buttons |
| v2.0.0 | Streaming, Settings UI, Daily Digest, Datasheet ingestion |
| v1.0.0 | Initial Release (RAG, email drafting, LoRA training) |
MIT License - Use freely for personal projects.
- Qwen 2.5 - Base model (upgraded from Mixtral 8x7B)
- llama.cpp - Efficient inference
- ChromaDB - Vector database
- OpenAI Whisper - Audio transcription
- PyMuPDF - PDF text extraction
Trust the Awesomeness! 🚀
Kyle Fox - @kylefoxaustin
Built with ❤️ and lots of 🍺 in Austin, Texas.