python-chatbot-agentic-ai

A multi-agent customer service AI system built with Python, FastAPI, and LangChain. Specialized agents collaborate to classify customer intents, retrieve knowledge, resolve issues, escalate when needed, and review quality—all with a clean web UI.

Features

✅ 5-Agent Orchestration Pipeline

• Intent Classification Agent: Detects customer intent and confidence
• Knowledge Retrieval Agent: Searches knowledge base for relevant policies/FAQs
• Resolution Agent: Generates customer-safe responses
• Escalation Agent: Handles low-confidence or high-risk scenarios
• Quality Agent: Final review and guardrails

✅ Web UI: Interactive chat interface with quick-action chips and real-time responses

✅ REST API: FastAPI endpoints for programmatic access

✅ Knowledge Base: Sample customer support documents (refunds, shipping, accounts, login troubleshooting)

✅ Admin Knowledge Base View: Browse documents in a readable grouped HTML page or query them as JSON

✅ Custom Q&A Management (Qdrant-backed): Add and delete custom questions/answers from the admin UI, persisted in Qdrant

✅ Testing: Unit and integration tests with a passing CI baseline

✅ OpenTelemetry Observability: Traces + AI metrics (tokens, latency, retrieval, agent/tool/MCP behavior, estimated cost)

Architecture Layers

• api: FastAPI routes and transport models.
• services: Application service layer that coordinates use-cases.
• orchestration: Multi-agent workflow and routing logic.
• agents: Specialized agents (intent, retrieval, resolution, escalation, quality).
• retrieval: Knowledge base and RAG-related retrieval logic.
• memory: Conversation memory interfaces and adapters.
• tools: MCP/tool integrations for external actions.
• infra: Infrastructure adapters (PostgreSQL, Qdrant, Kafka).
• domain: Core request/response models.
• core: Shared config and logging utilities.

Project Structure

.
├── docker/
├── docs/
│   └── knowledge_base/          # Sample support documents
│       ├── faqs/
│       ├── policies/
│       ├── playbooks/
│       └── troubleshooting/
├── scripts/
│   └── bootstrap.sh
├── src/
│   └── customer_service_ai/
│       ├── agents/              # 5 specialized agents
│       ├── api/
│       │   └── routes/          # chat.py, ui.py, admin.py
│       ├── core/
│       ├── domain/
│       ├── infra/
│       │   ├── db/
│       │   └── messaging/
│       ├── memory/
│       ├── orchestration/
│       ├── retrieval/
│       ├── services/
│       ├── tools/
│       └── main.py
├── tests/
│   ├── integration/
│   └── unit/
├── .env.example
├── docker-compose.yml
├── Makefile
├── pyproject.toml
└── README.md

Quick Start

1. Setup Environment

bash scripts/bootstrap.sh

2. Start Local Infrastructure (optional for full stack)

docker compose up -d

3. Run the API

make run

The service will start on http://localhost:8000

4. Access the UI

Open your browser to: http://localhost:8000/ui

5. Run Tests

make test

Observability (OpenTelemetry)

The app now emits OpenTelemetry traces and AI-specific metrics.

Key metrics include:

• ai_requests_total, ai_response_latency_seconds, ai_errors_total
• prompt_tokens_total, completion_tokens_total, total_tokens_consumed, ai_cost_usd_total
• retrieval_latency_seconds, retrieved_documents_count, retrieval_success_total, retrieval_failures_total
• agent_executions_total, agent_iterations_total, agent_failures_total
• tool_calls_total, tool_execution_time_seconds, tool_failures_total
• mcp_requests_total, mcp_tool_calls_total, mcp_errors_total

Example .env overrides:

OTEL_ENABLED=true
OTEL_SERVICE_NAME=customer-service-ai-agent
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
OTEL_METRICS_EXPORT_INTERVAL_MS=10000

# Set non-zero values for paid LLM cost estimation
LLM_INPUT_COST_PER_1K_TOKENS=0.0
LLM_OUTPUT_COST_PER_1K_TOKENS=0.0

See docs/OBSERVABILITY.md for architecture, dashboard recommendations, and alerting guidance.

Run Local Prometheus

docker compose up -d otel-collector prometheus
export OTEL_ENABLED=true
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
make run

Then open Prometheus at http://localhost:9090 and query metrics like ai_requests_total.

API Endpoints

Method	Endpoint	Purpose
GET	/	Service info with links
GET	/health	Health check
GET	/ui	Web chat interface
POST	/v1/chat	Send message to agent pipeline
GET	/v1/admin/knowledge-base	List all knowledge base documents (JSON)
GET	/v1/admin/knowledge-base/ui	Browse knowledge base documents (HTML)
POST	/v1/admin/knowledge-base/entries	Add or update a custom Q&A entry in Qdrant
DELETE	/v1/admin/knowledge-base/entries	Delete a custom Q&A entry from Qdrant

Example Request

curl -X POST http://localhost:8000/v1/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "user_id": "user123",
    "session_id": "session123",
    "message": "My account is locked"
  }'

Example Response

{
  "answer": "Accounts lock after 5 failed login attempts and automatically unlock after 30 minutes. You can also unlock immediately by completing a password reset.",
  "intent": "account_recovery",
  "confidence": 0.8,
  "escalated": false
}

Configuration Overrides (Optional)

The app ships with sensible defaults in src/customer_service_ai/core/config.py. Add environment variables only when you want to override runtime behavior per environment.

Example .env overrides:

# Ollama
OLLAMA_HOST=http://localhost:11434
OLLAMA_MODEL=mistral
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
OLLAMA_EMBEDDING_DIMENSIONS=768
OLLAMA_TIMEOUT=60

# Retrieval and reranking
RETRIEVAL_TOP_K=5
RETRIEVAL_RERANK_TOP_N=3

# Conversation memory depth
CHAT_HISTORY_LIMIT=8
CONVERSATION_STORE_MAX_TURNS=200

# Conversation store backend: memory (default) or postgres
CONVERSATION_STORE_BACKEND=memory
POSTGRES_DSN=postgresql://postgres:postgres@localhost:5432/customer_service_ai

Current Status

Implemented & Tested:

• ✅ FastAPI application with auto-reload
• ✅ 5-agent orchestration pipeline
• ✅ Ollama LLM integration (intent classification + response generation)
• ✅ Vector embeddings with Ollama (nomic-embed-text)
• ✅ RAG (Retrieval-Augmented Generation) with Qdrant semantic search
• ✅ Document ingestion pipeline with automatic chunking
• ✅ Web UI with chat, quick actions, and animations
• ✅ Health check and info endpoints
• ✅ Request/response validation with Pydantic
• ✅ Unit and integration tests
• ✅ OpenTelemetry traces and AI observability metrics
• ✅ Docker Compose setup for infrastructure
• ✅ Conversation store backend selection (memory or postgres)

Coming Next:

• MCP tool integrations for business actions
• Security layer (authentication, PII detection, prompt injection defense)
• Production hardening: HTTPS-only admin access
• Production hardening: secret manager for admin credentials
• Hybrid search with explicit keyword/semantic score blending
• Persistence hardening: schema migrations and connection pooling for conversation storage

Development

Format & Lint

make format  # ruff check + fix
make lint    # ruff + mypy

RAG Setup (Semantic Search)

The system now uses Retrieval-Augmented Generation for intelligent document retrieval:

# 1. Start Ollama + Qdrant
docker compose up -d ollama qdrant

# 2. Pull required Ollama models (first-time only)
docker exec python-chatbot-agentic-ai-ollama-1 ollama pull mistral
docker exec python-chatbot-agentic-ai-ollama-1 ollama pull nomic-embed-text

# 3. Ingest documents into Qdrant (generates embeddings)
python scripts/ingest_documents.py

# 4. Run API
make run

If you prefer host-installed Ollama, use ollama serve and ollama pull ... locally instead.

See RAG_GUIDE.md for detailed setup and troubleshooting.

Next Steps

To extend this system:

1. Harden Persistence: Add migrations/retention jobs and observability for PostgreSQL conversation storage
2. Tools Integration: Add MCP tools in tools/ layer for business actions
3. Conversation History: Switch CONVERSATION_STORE_BACKEND=postgres for shared multi-instance history
4. Hybrid Search: Add explicit keyword + semantic score blending and ranking strategy
5. Reranking Quality: Tune reranker prompts and add evaluation checks
6. Deploy: Package Docker image and run in orchestrated environment

Admin: Knowledge Base Viewer

Admins can inspect all static markdown documents, and can manage custom Q&A entries stored in Qdrant.

Admin endpoints are protected with HTTP Basic Auth. Set these environment variables before running the API:

export ADMIN_BASIC_AUTH_USERNAME='<your-admin-username>'
export ADMIN_BASIC_AUTH_PASSWORD='<your-strong-admin-password>'

Security notes:

• Admin credentials are loaded from environment variables at runtime.
• Credentials are not stored in PostgreSQL or Qdrant.
• Do not commit real credentials to source control.
• The service blocks admin access if the password is left as the default placeholder (change-me).

HTML view (browser)

http://localhost:8000/v1/admin/knowledge-base/ui

Shows all documents grouped by category in collapsible cards, with a section count summary and a link back to the chat UI.

Custom Q&A entries can be managed directly from this page:

• Add via the Add Q&A Entry form
• Delete via the Delete button shown on custom Q&A cards

JSON API

curl -u "$ADMIN_BASIC_AUTH_USERNAME:$ADMIN_BASIC_AUTH_PASSWORD" \
  http://localhost:8000/v1/admin/knowledge-base

{
  "total": 12,
  "documents": [
    {
      "id": "doc_0001",
      "title": "Refund Policy Overview",
      "category": "policies",
      "source": "policies/refund-policy.md",
      "content": "..."
    }
  ]
}

Sample Knowledge Base

The docs/knowledge_base/ directory contains example customer support documents:

• Policies: Refund and shipping policies with escalation triggers
• FAQs: Account access, order tracking, delivery address changes
• Troubleshooting: Login issues and account recovery workflows
• Playbooks: Escalation decision criteria and handoff procedures