AgentOS

TypeScript runtime for autonomous AI agents — multimodal RAG, cognitive memory, streaming guardrails, voice pipeline, and emergent multi-agent orchestration.

Website · Documentation · npm · GitHub

Overview

@framers/agentos is an open-source TypeScript AI agent runtime for building, deploying, and managing production AI agents. It provides multimodal RAG with cognitive memory (Ebbinghaus decay, 8 neuroscience-backed mechanisms including reconsolidation, retrieval-induced forgetting, involuntary recall, metacognitive FOK, temporal gist extraction, schema encoding, source confidence decay, and emotion regulation — all HEXACO personality-modulated), multi-agent orchestration, 37 channel adapters, 6 guardrail packs (PII redaction, ML classifiers, topicality, code safety, grounding guard, content policy rewriter) with prompt injection defense, 21 LLM providers, and 72 curated skills. Self-hostable and production-ready, it handles the full lifecycle from prompt construction through tool execution, safety evaluation, and streaming response delivery.

Key facts:

Property	Value
Package	`@framers/agentos`
Language	TypeScript 5.4+ / Node.js 18+
License	Apache 2.0

Runtime dependencies:

Dependency	Purpose
`@opentelemetry/api`	Distributed tracing and metrics
`ajv` + `ajv-formats`	JSON Schema validation for tool I/O
`axios`	HTTP client for LLM provider APIs
`lru-cache`	High-performance caching (dedup, embeddings, cost tracking)
`natural`	NLP utilities (tokenization, stemming, sentiment)
`pino`	Structured JSON logging
`uuid`	Unique identifier generation
`yaml`	YAML config parsing (agent configs, skill definitions)

Required peer dependency:

Dependency	Purpose
`@framers/sql-storage-adapter`	Cross-platform SQL persistence (SQLite, sql.js, IndexedDB, Postgres) — required for memory, knowledge graph, and trace storage

Optional peer dependencies:

Peer Dependency	Purpose
`graphology` + `graphology-communities-louvain`	GraphRAG community detection and memory graph clustering
`hnswlib-node`	HNSW-based approximate nearest neighbor search (falls back to brute-force without)
`neo4j-driver`	Neo4j-backed knowledge graph and GraphRAG (falls back to SQLite without)

Ecosystem

Package	Description	Links
@framers/agentos	Core orchestration runtime	· GitHub
@framers/agentos-extensions	Official extension registry (45+ extensions)	· GitHub
@framers/agentos-extensions-registry	Curated manifest builder	· GitHub
@framers/agentos-skills-registry	Skills catalog SDK (query helpers + snapshot factories)	· GitHub
@framers/agentos-skills	Skills content (72 SKILL.md files + registry.json)	· GitHub
docs.agentos.sh	Documentation site (Docusaurus)	GitHub · docs.agentos.sh

Guardrail Extensions

Package	What It Does	Links
@framers/agentos-ext-pii-redaction	Four-tier PII detection (regex + NLP + NER + LLM)	· Docs
@framers/agentos-ext-ml-classifiers	Toxicity, injection, jailbreak via ONNX BERT	· Docs
@framers/agentos-ext-topicality	Embedding-based topic enforcement + drift detection	· Docs
@framers/agentos-ext-code-safety	OWASP Top 10 code scanning (25 regex rules)	· Docs
@framers/agentos-ext-grounding-guard	RAG-grounded hallucination detection via NLI	· Docs
@framers/agentos-ext-content-policy-rewriter	Opt-in content policy (8 categories, LLM rewrite/block, 4 presets)	Docs

Quick Start

npm install @framers/agentos

Set any provider's API key and you're ready:

export OPENAI_API_KEY=sk-...        # or ANTHROPIC_API_KEY, GEMINI_API_KEY, GROQ_API_KEY, etc.

1. Generate Text (Provider-First)

No model strings needed — AgentOS picks the best default:

import { generateText, streamText } from '@framers/agentos';

const result = await generateText({
  provider: 'anthropic',
  prompt: 'Explain how TCP handshakes work in 3 bullets.',
});
console.log(result.text);

// Streaming
for await (const chunk of streamText({ provider: 'openai', prompt: 'Explain SYN-ACK.' }).textStream) {
  process.stdout.write(chunk);
}

16 providers supported. Automatic fallback when one fails (402/429/5xx → tries next available key).

2. Agent with Personality & Memory

Agents have HEXACO personality traits that shape their communication style, and cognitive memory with Ebbinghaus decay:

import { agent } from '@framers/agentos';

const tutor = agent({
  provider: 'anthropic',
  instructions: 'You are a patient computer science tutor.',
  personality: {
    openness: 0.9,           // creative, exploratory answers
    conscientiousness: 0.95,  // thorough, well-structured
    agreeableness: 0.85,      // warm, encouraging tone
  },
  memory: {
    enabled: true,
    cognitive: true,          // Ebbinghaus decay, reconsolidation, involuntary recall
  },
});

const session = tutor.session('student-1');
const reply = await session.send('Explain recursion with an analogy.');
console.log(reply.text);

// Memory persists across sessions — the agent remembers context
const followUp = await session.send('Can you expand on that?');
console.log(followUp.text);

3. Multimodal RAG

Ingest documents, images, and structured data — retrieve with HyDE (Hypothetical Document Embedding):

import { agent } from '@framers/agentos';

const analyst = agent({
  provider: 'openai',
  instructions: 'You are a financial analyst. Answer questions using provided documents.',
  rag: {
    enabled: true,
    mode: 'aggressive',       // always retrieves before answering
    topK: 10,
    strategy: 'hyde',         // generates hypothetical answers for better retrieval
  },
});

const session = analyst.session('q4-review');

// Ingest documents
await session.ingest('Q4 revenue grew 23% YoY to $4.2B...');
await session.ingest({ type: 'file', path: './earnings-report.pdf' });

const answer = await session.send('What drove revenue growth in Q4?');
console.log(answer.text);     // Cites specific passages from ingested docs

4. Structured Output (Zod Validation)

Extract typed data from unstructured text:

import { generateObject } from '@framers/agentos';
import { z } from 'zod';

const { object } = await generateObject({
  provider: 'gemini',
  schema: z.object({
    name: z.string(),
    sentiment: z.enum(['positive', 'negative', 'neutral']),
    topics: z.array(z.string()),
  }),
  prompt: 'Analyze: "The new iPhone camera is incredible but the battery life is disappointing."',
});

console.log(object);
// { name: "iPhone Review", sentiment: "mixed", topics: ["camera", "battery"] }

5. Workflows & Graphs

Deterministic multi-step pipelines with branching, parallel execution, and checkpointing:

import { workflow } from '@framers/agentos';

const pipeline = workflow('content-pipeline')
  .step('research', { provider: 'anthropic', instructions: 'Research the topic thoroughly.' })
  .step('draft', { provider: 'openai', instructions: 'Write a blog post from the research.' })
  .step('review', { provider: 'anthropic', instructions: 'Review for accuracy and tone.' })
  .step('publish', { tool: 'blog_publish' });

const result = await pipeline.run('Write a post about WebAssembly in 2026.');
console.log(result.text);

6. Multi-Agent Teams

Coordinate specialized agents with built-in strategies:

import { agency } from '@framers/agentos';

const team = agency({
  agents: {
    researcher: {
      instructions: 'Find relevant facts and data.',
      provider: 'anthropic',
    },
    writer: {
      instructions: 'Write a clear, engaging summary.',
      provider: 'openai',
    },
    reviewer: {
      instructions: 'Check for accuracy and suggest improvements.',
      provider: 'gemini',
      dependsOn: ['writer'],  // runs after writer
    },
  },
  strategy: 'graph',           // dependency-based DAG execution
  memory: { shared: true },    // agents share context
});

const result = await team.generate('Compare TCP vs UDP for game networking.');
console.log(result.text);

6 strategies: sequential, parallel, debate, review-loop, hierarchical, graph.

7. Voice Pipeline

Real-time speech-to-text and text-to-speech with streaming:

import { agent } from '@framers/agentos';

const receptionist = agent({
  provider: 'openai',
  instructions: 'You are a friendly receptionist for a dental clinic.',
  voice: {
    tts: { provider: 'elevenlabs', voice: 'Rachel' },
    stt: { provider: 'deepgram' },
  },
});

// Voice sessions stream audio in real-time
const call = receptionist.voiceSession('call-1');
await call.start();  // begins listening

8. Guardrails & Security

5-tier security with PII redaction, prompt injection defense, and code scanning:

import { agent } from '@framers/agentos';

const secureBot = agent({
  provider: 'anthropic',
  instructions: 'You are a customer support agent.',
  security: { tier: 'strict' },
  guardrails: {
    input: ['pii-redaction', 'ml-classifiers'],     // block PII + detect injection
    output: ['grounding-guard', 'code-safety'],      // prevent hallucination + unsafe code
  },
});

5 tiers: dangerous → permissive → balanced → strict → paranoid.

Default Models Per Provider

Provider	Default Model	Env Var
openai	gpt-4o	`OPENAI_API_KEY`
anthropic	claude-sonnet-4	`ANTHROPIC_API_KEY`
gemini	gemini-2.5-flash	`GEMINI_API_KEY`
ollama	llama3.2	`OLLAMA_BASE_URL`
groq	llama-3.3-70b	`GROQ_API_KEY`
openrouter	openai/gpt-4o	`OPENROUTER_API_KEY`
together	meta-llama/Meta-Llama-3.1-70B	`TOGETHER_API_KEY`
mistral	mistral-large-latest	`MISTRAL_API_KEY`
xai	grok-3-mini	`XAI_API_KEY`

Automatic fallback: when a provider fails, AgentOS tries the next available API key.

System Architecture

Architecture Diagram

graph TB
    subgraph Runtime["AgentOS Runtime"]
        API["AgentOS API<br/><i>Service Facade</i>"] --> Orch["AgentOSOrchestrator<br/><i>Delegation Hub</i>"]
        API --> Stream["StreamingManager<br/><i>Async Gen Mgmt</i>"]
        Orch --> Prompt["PromptEngine<br/><i>Dynamic Prompts</i>"]

        subgraph GMI["GMI Manager — Generalized Modular Intelligence"]
            WM["Working<br/>Memory"] ~~~ CM["Context<br/>Manager"] ~~~ PO["Persona<br/>Overlay"] ~~~ LM["Learning<br/>Module"]
            EM["Episodic<br/>Memory"] ~~~ SM["Semantic<br/>Memory"] ~~~ PM["Procedural<br/>Memory"]
        end

        Stream --> GMI
        Prompt --> GMI

        GMI --> Tools["Tool<br/>Orchestrator"]
        GMI --> RAG["RAG<br/>Memory"]
        GMI --> Plan["Planning<br/>Engine"]
        GMI --> WF["Workflow<br/>Engine"]

        RAG --> Embed["Embedding<br/>Manager"]
        Plan --> ReAct["ReAct<br/>Reasoner"]

        subgraph LLM["LLM Provider Manager"]
            OAI["OpenAI"] ~~~ Anth["Anthropic"] ~~~ Gem["Gemini"] ~~~ Oll["Ollama"] ~~~ OR["OpenRouter"]
        end

        Tools --> LLM
        Embed --> LLM
        ReAct --> LLM

        Guard["Guardrail<br/>Service"] ~~~ CB["Circuit<br/>Breaker"] ~~~ CG["Cost<br/>Guard"] ~~~ SD["Stuck<br/>Detector"]
        Ext["Extension Manager<br/><i>12+ kinds</i>"] ~~~ Chan["Channel Router<br/><i>37 platforms</i>"] ~~~ Call["Call Manager<br/><i>Voice/Tel.</i>"]
        Obs["Observability<br/><i>OpenTelemetry</i>"] ~~~ HITL["HITL Manager<br/><i>Approval/Escal.</i>"] ~~~ Skill["Skill Registry<br/><i>SKILL.md</i>"]
    end

    style Runtime fill:#0e0e18,stroke:#c9a227,stroke-width:2px,color:#f2f2fa
    style GMI fill:#151520,stroke:#00f5ff,stroke-width:1px,color:#f2f2fa
    style LLM fill:#151520,stroke:#8b5cf6,stroke-width:1px,color:#f2f2fa

Request Lifecycle

A single processRequest() call flows through these stages:

flowchart TD
    Input["User Input<br/><code>AgentOSInput</code>"] --> G1

    G1["1 · Input Guardrails<br/><i>evaluateInput()</i>"]
    G1 -->|ALLOW| Ctx
    G1 -.->|BLOCK / SANITIZE| Reject1["⛔ Rejected"]

    Ctx["2 · Context Assembly<br/><i>ConversationManager + RAG</i>"] --> Prompt
    Prompt["3 · Prompt Construction<br/><i>PromptEngine</i>"] --> GMI
    GMI["4 · GMI Processing<br/><i>Working memory · Persona · Adaptation</i>"] --> LLM

    LLM["5 · LLM Call<br/><i>Provider Manager + Circuit Breaker</i>"] --> ToolCheck

    ToolCheck{"Tool call?"}
    ToolCheck -->|Yes| ToolExec["6 · Tool Execution<br/><i>ToolOrchestrator + Permission Guard</i>"]
    ToolExec -->|Loop| LLM
    ToolCheck -->|No| G2

    G2["7 · Output Guardrails<br/><i>evaluateOutput()</i>"]
    G2 -->|ALLOW| Stream
    G2 -.->|BLOCK / SANITIZE| Reject2["⛔ Rejected"]

    Stream["8 · Streaming Response<br/><i>AsyncGenerator yields chunks</i>"] --> Post
    Post["9 · Post-Processing<br/><i>Memory · Cost · Telemetry</i>"]

    style Input fill:#1c1c28,stroke:#c9a227,color:#f2f2fa
    style G1 fill:#1c1c28,stroke:#ef4444,color:#f2f2fa
    style G2 fill:#1c1c28,stroke:#ef4444,color:#f2f2fa
    style LLM fill:#1c1c28,stroke:#8b5cf6,color:#f2f2fa
    style ToolCheck fill:#1c1c28,stroke:#00f5ff,color:#f2f2fa
    style ToolExec fill:#1c1c28,stroke:#00f5ff,color:#f2f2fa
    style Post fill:#1c1c28,stroke:#10b981,color:#f2f2fa

Layer Breakdown

AgentOS is organized into six architectural layers:

Layer	Directory	Purpose
API	`src/api/`	Public-facing facade, input/output types, orchestrator
Cognitive	`src/cognitive_substrate/`	GMI instances, persona overlays, multi-layer memory
Core	`src/core/`	28 subdirectories: LLM, tools, safety, guardrails, planning, workflows, HITL, observability, etc.
Integration	`src/channels/`, `src/voice/`, `src/extensions/`	External platform adapters, telephony, plugin system
Intelligence	`src/rag/`, `src/skills/`	RAG pipeline, GraphRAG, skill loading
Infrastructure	`src/config/`, `src/logging/`, `src/utils/`, `src/types/`	Configuration, structured logging, shared utilities

Core Modules

API Layer

Location: src/api/

The API layer is the primary entry point for all interactions with AgentOS.

File	Role
`AgentOS.ts`	Main service facade (1000+ lines). Implements `IAgentOS`. Initializes all subsystems and delegates to the orchestrator.
`AgentOSOrchestrator.ts`	Delegation hub. Coordinates GMI processing, tool execution, guardrail evaluation, and streaming assembly. Streaming-first design using async generators.
`interfaces/IAgentOS.ts`	Core contract defining `initialize()`, `processRequest()`, `handleToolResult()`, workflow management, persona listing, and conversation history retrieval.
`types/AgentOSInput.ts`	Unified input structure: text, vision, audio, persona selection, user API keys, feedback, workflow/agency invocations, and processing options.
`types/AgentOSResponse.ts`	Streaming output: 11 chunk types covering text deltas, tool calls, progress, errors, workflows, agency updates, and provenance events.

Response chunk types:

enum AgentOSResponseChunkType {
  TEXT_DELTA           // Incremental text tokens
  SYSTEM_PROGRESS      // Internal processing updates
  TOOL_CALL_REQUEST    // Agent requests tool execution
  TOOL_RESULT_EMISSION // Tool execution result
  UI_COMMAND           // Client-side UI directives
  FINAL_RESPONSE       // Aggregated final response
  ERROR                // Error information
  METADATA_UPDATE      // Session/context metadata changes
  WORKFLOW_UPDATE      // Workflow progress notifications
  AGENCY_UPDATE        // Multi-agent coordination events
  PROVENANCE_EVENT     // Immutability/audit trail events
}

Cognitive Substrate (GMI)

Location: src/cognitive_substrate/

The Generalized Modular Intelligence (GMI) is the core agent instance -- the "brain" of each agent. A single AgentOS runtime can manage multiple GMI instances via GMIManager.

graph TB
    subgraph GMI["GMI Instance"]
        PO["PersonaOverlay Manager<br/><i>Personality switching</i>"] ~~~ CTX["Context Manager<br/><i>Dynamic prompts · User context · Session</i>"]
        WM["Working Memory<br/><i>Current session state · Tool context</i>"] ~~~ AM["Adaptation Manager<br/><i>Learning rate · Style drift · Tone</i>"]
        subgraph Mem["Multi-Layer Memory"]
            Ep["Episodic<br/><i>Events</i>"] ~~~ Sem["Semantic<br/><i>Facts</i>"] ~~~ Proc["Procedural<br/><i>Skills</i>"] ~~~ LT["Long-Term<br/><i>Archive</i>"]
        end
        PO --> Mem
        WM --> Mem
    end
    style GMI fill:#0e0e18,stroke:#c9a227,stroke-width:2px,color:#f2f2fa
    style Mem fill:#151520,stroke:#00f5ff,stroke-width:1px,color:#f2f2fa

Key components:

GMI.ts -- Core agent instance (2000+ lines). Manages the complete cognitive loop: context assembly, prompt construction, LLM invocation, tool handling, and response generation.
GMIManager.ts -- Lifecycle manager for multiple GMI instances. Handles creation, pooling, configuration, and teardown.
IGMI.ts -- Interface contract for GMI implementations.
PersonaOverlayManager -- Enables dynamic personality switching at runtime. Ships with 5+ built-in personas:
- Researcher -- Analytical, citation-heavy responses
- Generalist -- Balanced, conversational
- Atlas -- Navigation and spatial reasoning
- Default Assistant -- General-purpose helpful assistant

Memory subsystem (memory/) -- 5 memory types, 4-tier hierarchy, 8 cognitive mechanisms:

Memory types (Tulving's long-term memory taxonomy):

Type	Cognitive Model	Usage
`episodic`	Autobiographical events	Conversation events, interactions
`semantic`	General knowledge/facts	Learned facts, preferences, schemas
`procedural`	Skills and how-to	Workflows, tool usage patterns
`prospective`	Future intentions	Goals, reminders, planned actions
`working`	Baddeley's model	Current session state, active context (7 +/- 2 slots)

4-tier directory hierarchy:

core/ -- Types, config, encoding (Yerkes-Dodson, flashbulb, HEXACO-weighted), decay (Ebbinghaus curves + spaced repetition), working memory, prompt assembly
retrieval/ -- MemoryStore (6-signal composite scoring), memory graph (spreading activation), prospective memory, retrieval feedback
pipeline/ -- Consolidation (merge, prune, promote, schema extraction), observation (buffer, compress, reflect), lifecycle management, infinite context window
io/ -- Document ingestion (PDF, DOCX, HTML, Markdown, URL), import/export (JSON, Obsidian, SQLite, ChatGPT, CSV), facade API, memory tools, extensions

Cognitive mechanisms (memory/mechanisms/) -- 8 optional, all HEXACO personality-modulated:

Mechanism	Citation	Effect
Reconsolidation	Nader et al., 2000	Retrieved memories drift toward current mood
Retrieval-Induced Forgetting	Anderson et al., 1994	Retrieving one memory suppresses similar competitors
Involuntary Recall	Berntsen, 2009	Random surfacing of old high-vividness memories
Metacognitive FOK	Nelson & Narens, 1990	Feeling-of-knowing scoring for tip-of-tongue states
Temporal Gist	Reyna & Brainerd, 1995	Old traces compressed to core assertions
Schema Encoding	Bartlett, 1932	Novel input boosted, schema-matching encoded efficiently
Source Confidence Decay	Johnson et al., 1993	Agent inferences decay faster than observations
Emotion Regulation	Gross, 1998	Reappraisal + suppression during consolidation

See docs/memory/COGNITIVE_MECHANISMS.md for API reference and 30+ APA citations.

LLM Provider Management

Location: src/core/llm/

The LLM layer abstracts multiple AI model providers behind a unified interface.

graph TD
    PM["AIModelProviderManager<br/><i>Provider registry · Routing · Fallback · Model switching</i>"]
    PM --> OAI["OpenAI Provider<br/><code>gpt-4o · gpt-4o-mini · o1</code>"]
    PM --> Anth["Anthropic Provider<br/><code>claude-sonnet-4 · claude-haiku</code>"]
    PM --> Gem["Gemini Provider<br/><code>gemini-2.5-flash · gemini-2.0</code>"]
    PM --> Oll["Ollama Provider<br/><code>llama3.2 · mistral · codellama</code>"]
    PM --> GQ["Groq Provider<br/><code>llama-3.3-70b · gemma2-9b</code>"]
    PM --> OR["OpenRouter Provider<br/><code>200+ models · auto-routing</code>"]
    PM --> Custom["Custom Provider<br/><i>IProvider interface</i>"]
    style PM fill:#1c1c28,stroke:#8b5cf6,color:#f2f2fa

Provider implementations:

Provider	File	Models	Env Var
OpenAI	`providers/implementations/OpenAIProvider.ts`	GPT-4o, GPT-4o-mini, o1, o3	`OPENAI_API_KEY`
Anthropic	`providers/implementations/AnthropicProvider.ts`	Claude Sonnet 4, Claude Haiku	`ANTHROPIC_API_KEY`
Gemini	`providers/implementations/GeminiProvider.ts`	Gemini 2.5 Flash, Gemini 2.0	`GEMINI_API_KEY`
Groq	`providers/implementations/GroqProvider.ts`	Llama 3.3 70B, Gemma2 9B (fast inference)	`GROQ_API_KEY`
Mistral	`providers/implementations/MistralProvider.ts`	Mistral Large, Mistral Small	`MISTRAL_API_KEY`
Together	via OpenAI-compat	Meta-Llama 3.1 70B/8B	`TOGETHER_API_KEY`
xAI	via OpenAI-compat	Grok-2, Grok-2 Mini	`XAI_API_KEY`
Ollama	`providers/implementations/OllamaProvider.ts`	Any locally-hosted model	`OLLAMA_BASE_URL`
OpenRouter	`providers/implementations/OpenRouterProvider.ts`	200+ models from any provider	`OPENROUTER_API_KEY`

Auto-detection: When no provider is specified, AgentOS scans env vars in priority order (OpenRouter > OpenAI > Anthropic > Gemini > Groq > Together > Mistral > xAI > Ollama) and uses the first configured provider. Set one env var and every helper call works automatically.

Additional components:

PromptEngine (PromptEngine.ts) -- Constructs prompts from system instructions, contextual elements, persona definitions, and dynamic user context. Supports template interpolation and conditional element inclusion.
IProvider (providers/IProvider.ts) -- Interface contract for adding custom LLM providers.
Streaming adapters -- All providers support token-level streaming via async generators with backpressure control.

Per-request model override (high-level helpers):

// Provider-first: AgentOS picks the default model
await generateText({ provider: 'gemini', prompt: 'Summarize this document' });

// Override a specific model
await generateText({ provider: 'anthropic', model: 'claude-haiku-4-5-20251001', prompt: '...' });

Per-request model override (full runtime):

for await (const chunk of agent.processRequest({
  userId: 'user-1',
  sessionId: 'session-1',
  textInput: 'Summarize this document',
  options: {
    preferredProviderId: 'gemini',
    preferredModelId: 'gemini-2.5-flash',
  },
})) { /* ... */ }

Tool System

Location: src/core/tools/

Tools are the primary mechanism for agents to interact with the outside world. Every tool implements the ITool interface.

graph TD
    TO["ToolOrchestrator<br/><i>Discovery · Selection · Validation · Execution</i>"]
    TO --> TE["ToolExecutor<br/><i>Validation · Invocation · Result wrap</i>"]
    TO --> TP["ToolPermission Manager<br/><i>RBAC · Per-user access</i>"]
    TO --> TG["ToolExecution Guard<br/><i>Timeout 30s · Circuit breaker</i>"]
    style TO fill:#1c1c28,stroke:#00f5ff,color:#f2f2fa

ITool interface (abbreviated):

interface ITool<TInput = any, TOutput = any> {
  readonly id: string;              // Globally unique identifier
  readonly name: string;            // LLM-facing function name
  readonly displayName: string;     // Human-readable title
  readonly description: string;     // Natural language description for LLM
  readonly inputSchema: JSONSchemaObject;   // JSON Schema for input validation
  readonly outputSchema?: JSONSchemaObject; // JSON Schema for output validation
  readonly category?: string;       // Grouping (e.g., "data_analysis")
  readonly hasSideEffects: boolean; // Whether the tool modifies external state
  readonly requiredCapabilities?: string[]; // Persona capabilities needed

  execute(
    args: TInput,
    context?: ToolExecutionContext
  ): Promise<ToolExecutionResult<TOutput>>;
}

ToolExecutionResult:

interface ToolExecutionResult<TOutput = any> {
  success: boolean;
  output?: TOutput;
  error?: string;
  contentType?: string;  // MIME type (default: "application/json")
  details?: Record<string, any>;
}

ToolExecutionContext provides the tool with calling agent identity (gmiId, personaId), user context, correlation ID for tracing, and optional session data.

Extension System

Location: src/extensions/

The extension system is AgentOS's plugin architecture. Extensions are packaged as packs containing one or more descriptors, loaded via a manifest.

ExtensionManifest
    |
    +-- ExtensionPackManifestEntry[]
            |
            +-- ExtensionPack (resolved via factory/module/package)
                    |
                    +-- ExtensionDescriptor[] (id + kind + payload)
                            |
                            +-- Registered in ExtensionRegistry
                                    |
                                    +-- Consumed by runtime (ToolOrchestrator,
                                        GuardrailService, WorkflowEngine, etc.)

12 extension kinds:

Kind Constant	Value	Payload Type
`EXTENSION_KIND_TOOL`	`"tool"`	`ITool`
`EXTENSION_KIND_GUARDRAIL`	`"guardrail"`	`IGuardrailService`
`EXTENSION_KIND_RESPONSE_PROCESSOR`	`"response-processor"`	Response transform function
`EXTENSION_KIND_WORKFLOW`	`"workflow"`	`WorkflowDescriptorPayload`
`EXTENSION_KIND_WORKFLOW_EXECUTOR`	`"workflow-executor"`	Workflow step executor
`EXTENSION_KIND_PERSONA`	`"persona"`	`IPersonaDefinition`
`EXTENSION_KIND_PLANNING_STRATEGY`	`"planning-strategy"`	Planning algorithm
`EXTENSION_KIND_HITL_HANDLER`	`"hitl-handler"`	Human interaction handler
`EXTENSION_KIND_COMM_CHANNEL`	`"communication-channel"`	Agent-to-agent channel
`EXTENSION_KIND_MEMORY_PROVIDER`	`"memory-provider"`	Memory backend
`EXTENSION_KIND_MESSAGING_CHANNEL`	`"messaging-channel"`	External platform adapter
`EXTENSION_KIND_PROVENANCE`	`"provenance"`	Audit/immutability handler

ExtensionDescriptor:

interface ExtensionDescriptor<TPayload = unknown> {
  id: string;                           // Unique within its kind
  kind: ExtensionKind;                  // One of the 12 kinds above
  priority?: number;                    // Higher loads later (overrides earlier)
  enableByDefault?: boolean;            // Auto-enable on discovery
  metadata?: Record<string, unknown>;   // Arbitrary metadata
  payload: TPayload;                    // The actual implementation
  source?: ExtensionSourceMetadata;     // Provenance (package name, version)
  requiredSecrets?: ExtensionSecretRequirement[]; // API keys needed
  onActivate?: (ctx: ExtensionLifecycleContext) => Promise<void> | void;
  onDeactivate?: (ctx: ExtensionLifecycleContext) => Promise<void> | void;
}

ExtensionManifest:

interface ExtensionManifest {
  packs: ExtensionPackManifestEntry[];  // Pack references
  overrides?: ExtensionOverrides;       // Per-descriptor enable/disable/priority
}

// Packs can be resolved three ways:
type ExtensionPackResolver =
  | { package: string; version?: string }  // npm package
  | { module: string }                     // Local module path
  | { factory: () => Promise<ExtensionPack> | ExtensionPack }; // Inline factory

Loading pipeline:

ExtensionLoader resolves pack entries from the manifest
ExtensionRegistry registers descriptors by kind, applying priority stacking
ExtensionManager provides runtime access: getTools(), getGuardrails(), getWorkflows(), etc.
MultiRegistryLoader supports loading from multiple remote registries

Lifecycle context and shared services:

ExtensionLifecycleContext.getSecret(secretId) gives packs host-resolved secrets at activation time
ExtensionLifecycleContext.services provides a shared ISharedServiceRegistry for lazy singleton reuse across packs
heavyweight dependencies such as NLP pipelines, ONNX models, embedding functions, and NLI models should be loaded through the shared registry rather than per-descriptor globals

Built-in guardrail packs exported by @framers/agentos:

Pack	Import Path	Guardrail ID	Tool IDs	Purpose
PII Redaction	`@framers/agentos-ext-pii-redaction`	`pii-redaction-guardrail`	`pii_scan`, `pii_redact`	Four-tier PII detection and redaction
ML Classifiers	`@framers/agentos-ext-ml-classifiers`	`ml-classifier-guardrail`	`classify_content`	Toxicity, prompt-injection, and jailbreak detection
Topicality	`@framers/agentos-ext-topicality`	`topicality-guardrail`	`check_topic`	On-topic enforcement and session drift detection
Code Safety	`@framers/agentos-ext-code-safety`	`code-safety-guardrail`	`scan_code`	Regex-based code risk scanning across fenced code and tool args
Grounding Guard	`@framers/agentos-ext-grounding-guard`	`grounding-guardrail`	`check_grounding`	RAG-source claim verification and hallucination detection
Content Policy Rewriter	`@framers/agentos-ext-content-policy-rewriter`	`content-policy-rewriter-guardrail`	—	Opt-in content policy: 8 categories (illegal_harmful, adult, profanity, violence, self_harm, hate_speech, illegal_activity, custom), keyword pre-filter + LLM judge/rewriter, 4 presets

Planning Engine

Location: src/orchestration/planner/

The planning engine enables multi-step task decomposition and execution using ReAct (Reasoning + Acting) patterns.

graph TD
    PE["PlanningEngine<br/><i>Task decomposition · Step sequencing · Execution loop</i>"]
    PE --> IF["IPlanningEngine Interface"]
    IF --- M1["createPlan(goal, context) → Plan"]
    IF --- M2["executePlan(plan) → AsyncGenerator‹PlanStepResult›"]
    IF --- M3["revisePlan(plan, feedback) → Plan"]
    style PE fill:#1c1c28,stroke:#10b981,color:#f2f2fa
    style IF fill:#151520,stroke:#c9a227,color:#f2f2fa

Plans are composed of typed steps that the agent executes sequentially, with the ability to revise the plan based on intermediate results. The planning engine integrates with:

Tool system for action execution
Guardrails for step-level safety checks
HITL for human approval of high-risk steps
Memory for persisting plan state across sessions

See docs/orchestration/PLANNING_ENGINE.md for the full planning system specification.

Conversation Management

Location: src/core/conversation/

Manages session state, message history, and long-term memory persistence.

ConversationManager -- Creates and retrieves conversation contexts, manages rolling message windows, and coordinates memory persistence.
ConversationContext -- Immutable snapshot of a conversation: messages, metadata, active persona, user context.
IRollingSummaryMemorySink -- Interface for persisting conversation summaries that compress long conversations into retrievable memory.
ILongTermMemoryRetriever -- Interface for retrieving relevant past conversations during context assembly.

RAG (Retrieval Augmented Generation)

Location: src/rag/

A complete RAG pipeline with pluggable vector stores, embedding management, and document ingestion.

graph TD
    RA["RetrievalAugmentor<br/><i>Ingestion · Retrieval · Document management</i>"]
    RA --> EM["EmbeddingManager<br/><i>Model selection · Caching · Batch</i>"]
    RA --> VSM["VectorStoreManager<br/><i>Multi-provider vector storage</i>"]
    EM --> LLM["LLM Provider<br/><i>Embedding models</i>"]
    VSM --> Mem["InMemory<br/><i>dev</i>"]
    VSM --> SQL["SQL<br/><i>prod</i>"]
    VSM --> HNSW["HNSW<br/><i>local ANN</i>"]
    VSM --> Qd["Qdrant<br/><i>cloud</i>"]
    style RA fill:#1c1c28,stroke:#c9a227,color:#f2f2fa
    style VSM fill:#1c1c28,stroke:#8b5cf6,color:#f2f2fa

Vector store implementations:

Store	Import	Use Case
`InMemoryVectorStore`	`@framers/agentos/rag`	Development and testing
`SqlVectorStore`	`@framers/agentos/rag`	Production (SQLite/Postgres via `@framers/sql-storage-adapter`)
`HnswlibVectorStore`	`@framers/agentos/rag`	High-performance local ANN search
`QdrantVectorStore`	`@framers/agentos/rag`	Cloud-hosted vector database
`Neo4jVectorStore`	`@framers/agentos/rag`	Neo4j 5.x native vector indexes with shared connection pooling
`PostgresVectorStore`	`@framers/agentos/rag`	Postgres + pgvector with HNSW indexes and RRF hybrid search
`PineconeVectorStore`	`@framers/agentos/rag`	Pinecone cloud via fetch API with metadata filtering

Database Persistence: @framers/sql-storage-adapter

All SQL-backed storage in AgentOS — including SqlVectorStore, conversation persistence, and memory archival — is powered by the @framers/sql-storage-adapter package. It provides a unified interface across 7 database backends with automatic runtime detection:

Adapter Runtime Use Case

better-sqlite3 Node.js Production (default)

pg Node.js PostgreSQL for cloud deployments

sql.js Browser/WASM Client-side storage

capacitor Mobile iOS/Android via Capacitor

electron Desktop Electron apps with IPC bridge

indexeddb Browser Fallback browser storage

memory Any Testing and development
import { createDatabase } from '@framers/sql-storage-adapter';
const db = await createDatabase(); // auto-detects best adapter

GraphRAG:

AgentOS supports both in-memory GraphRAGEngine and persistent Neo4jGraphRAGEngine (at src/rag/graphrag/) for knowledge-graph-enhanced retrieval:

Entity and relationship extraction from documents
Community detection via Louvain algorithm (requires graphology peer dependency)
Local search (entity-centric) and global search (community-summarized)
Hybrid retrieval combining vector similarity with graph traversal

import { GraphRAGEngine } from '@framers/agentos/rag/graphrag';
import { Neo4jGraphRAGEngine } from '@framers/agentos/rag/graphrag';
import type { GraphRAGConfig, GraphEntity, GraphRelationship } from '@framers/agentos/rag/graphrag';

See docs/memory/RAG_MEMORY_CONFIGURATION.md and docs/memory/MULTIMODAL_RAG.md for detailed configuration guides.

Memory Scaling:

AgentOS provides a 4-tier vector storage scaling path that grows with your deployment:

Tier	Backend	When to Use
0	SQLite (default)	Development, small datasets (< 1K vectors)
1	HNSW sidecar	Auto-activates at 1K vectors for local ANN search
2	Postgres + pgvector	Production cloud deployments with RRF hybrid search
3	Qdrant / Pinecone	High-scale managed vector databases

One-command migration via MigrationEngine — move data between any two backends with migrate({ from, to })
Docker auto-setup — MigrationEngine.ensureBackend('qdrant') or ensureBackend('postgres') pulls and starts containers automatically
Binary blob embedding storage — 3-4x faster than JSON serialization for high-throughput ingest
HNSW sidecar auto-activation — transparently upgrades from brute-force to approximate nearest neighbors at 1K vectors
embed() config option — opt-in query-time embedding generation for dynamic content

See docs/memory/MEMORY_SCALING.md, docs/memory/POSTGRES_BACKEND.md, docs/memory/QDRANT_BACKEND.md, and docs/memory/PINECONE_BACKEND.md for backend-specific guides.

Bundled Platform Knowledge

Location: knowledge/platform-corpus.json

AgentOS ships with 244 pre-built knowledge entries covering the entire platform surface area. When the QueryRouter initializes, these entries are automatically loaded alongside your project-specific documentation corpus, giving every agent instant knowledge about AgentOS capabilities with zero configuration.

Coverage breakdown:

Category	Count	What it covers
Tools	105	Every tool and channel adapter (Discord, Telegram, LinkedIn, Bluesky, etc.)
Skills	80	All curated skills from the skills registry
FAQ	30	Common questions (voice setup, supported models, streaming, OCR, etc.)
API	14	Core API functions (generateText, streamText, agent, agency, etc.)
Troubleshooting	15	Common errors and their fixes (missing API keys, model not found, etc.)

How it works:

During router.init(), the QueryRouter loads knowledge/platform-corpus.json from the package directory
Platform entries are merged into the same corpus as user docs, making them searchable via both vector and keyword retrieval
The keyword fallback index covers platform entries, so they work even without an embedding API key

Disabling platform knowledge:

const router = new QueryRouter({
  knowledgeCorpus: ['./docs'],
  includePlatformKnowledge: false, // Skip bundled platform entries
});

When disabled, the router only loads your knowledgeCorpus directories. This is useful when you want a purely project-specific knowledge base or are building a non-AgentOS product.

Safety and Guardrails

Location: src/safety/runtime/ and src/safety/guardrails/

AgentOS provides defense-in-depth safety through two complementary systems.

Safety Primitives (`safety/runtime/`)

Five runtime safety components:

Component	Export	Purpose
CircuitBreaker	`CircuitBreaker`	Three-state (closed/open/half-open) wrapper for LLM calls. Configurable failure threshold, reset timeout, and half-open probe count. Throws `CircuitOpenError` when tripped.
ActionDeduplicator	`ActionDeduplicator`	Hash-based recent action tracking with LRU eviction. Prevents redundant tool calls and repeated operations within a configurable time window.
StuckDetector	`StuckDetector`	Detects three patterns: repeated outputs, repeated errors, and oscillation (A-B-A-B cycles). Returns `StuckDetection` with reason and confidence.
CostGuard	`CostGuard`	Per-agent session and daily spending caps. Defaults: $1/session, $5/day. Throws `CostCapExceededError` when limits are hit. Tracks token usage across all LLM calls.
ToolExecutionGuard	`ToolExecutionGuard`	Per-tool timeout (30s default) with independent circuit breakers per tool ID. Reports `ToolHealthReport` for monitoring. Throws `ToolTimeoutError`.

import {
  CircuitBreaker,
  CostGuard,
  StuckDetector,
  ActionDeduplicator,
  ToolExecutionGuard,
} from '@framers/agentos/safety/runtime';

See docs/safety/SAFETY_PRIMITIVES.md for the full safety API reference.

Guardrails (`safety/guardrails/`)

Content-level input/output filtering:

interface IGuardrailService {
  config?: {
    evaluateStreamingChunks?: boolean;
    maxStreamingEvaluations?: number;
    canSanitize?: boolean;
    timeoutMs?: number;
  };

  evaluateInput?(payload: GuardrailInputPayload): Promise<GuardrailEvaluationResult | null>;
  evaluateOutput?(payload: GuardrailOutputPayload): Promise<GuardrailEvaluationResult | null>;
}

GuardrailOutputPayload also carries ragSources?: RagRetrievedChunk[], which enables grounding-aware output checks against retrieved context.

Guardrails run at two points in the request lifecycle:

Pre-processing -- evaluateInput() inspects user input before orchestration
Post-processing -- evaluateOutput() inspects streaming chunks and/or the final response before emission

When multiple guardrails are registered, AgentOS uses a two-phase dispatcher:

Phase 1 (sequential sanitizers) -- guardrails with config.canSanitize === true run in registration order so each sanitizer sees the cumulative sanitized text
Phase 2 (parallel classifiers) -- all remaining guardrails run concurrently with worst-action aggregation (BLOCK > FLAG > ALLOW)

ParallelGuardrailDispatcher powers both input evaluation and output stream wrapping, with per-guardrail timeoutMs fail-open behavior for slow or degraded classifiers.

Multiple guardrails can be composed via the extension system, and each receives full context (user ID, session ID, persona ID, conversation ID, metadata) for context-aware policy decisions.

See docs/safety/GUARDRAILS_USAGE.md for implementation patterns.

Human-in-the-Loop (HITL)

Location: src/orchestration/hitl/

The HITL system enables agents to request human approval, clarification, and collaboration at key decision points.

Core interface: IHumanInteractionManager

Three interaction modes:

Mode	Method	Use Case
Approval	`requestApproval()`	Gate high-risk actions (database mutations, financial operations, external communications)
Clarification	`requestClarification()`	Ask the user for missing information before proceeding
Escalation	`escalateToHuman()`	Hand off to a human operator when the agent cannot proceed

PendingAction structure:

interface PendingAction {
  actionId: string;
  description: string;
  severity: 'low' | 'medium' | 'high' | 'critical';
  category?: 'data_modification' | 'external_api' | 'financial' |
             'communication' | 'system' | 'other';
  agentId: string;
  context: Record<string, unknown>;
  potentialConsequences?: string[];
  reversible: boolean;
  estimatedCost?: { amount: number; currency: string };
  timeoutMs?: number;
  alternatives?: AlternativeAction[];
}

HITL integrates with the planning engine so individual plan steps can require approval, and with the extension system via EXTENSION_KIND_HITL_HANDLER for custom approval UIs.

See docs/safety/HUMAN_IN_THE_LOOP.md for the full HITL specification.

Channels System

Location: src/channels/

Unified adapters for 40 external messaging and social platforms.

40 supported platforms:

Priority	Platforms
P0 (Core + Social)	Telegram, WhatsApp, Discord, Slack, Webchat, Twitter/X, Instagram, Reddit, YouTube, LinkedIn, Facebook, Threads, Bluesky
P1	Signal, iMessage, Google Chat, Microsoft Teams, Pinterest, TikTok, Mastodon, Dev.to, Hashnode, Medium, WordPress
P2	Matrix, Zalo, Email, SMS, Farcaster, Lemmy, Google Business
P3	Nostr, Twitch, Line, Feishu, Mattermost, Nextcloud Talk, Tlon, IRC, Zalo Personal

29 capability flags:

Each adapter declares its capabilities, allowing consumers to check before attempting unsupported actions:

text, rich_text, images, video, audio, voice_notes, documents,
stickers, reactions, threads, typing_indicator, read_receipts,
group_chat, channels, buttons, inline_keyboard, embeds,
mentions, editing, deletion
stories, reels, hashtags, polls, carousel,
engagement_metrics, scheduling, dm_automation, content_discovery

IChannelAdapter -- Unified interface for bidirectional messaging:

connect() / disconnect() -- Lifecycle management
sendMessage() -- Outbound messages with platform-specific formatting
onMessage() -- Inbound message handler registration
getConnectionInfo() -- Connection health monitoring
capabilities -- Declared capability set

ChannelRouter -- Routes inbound messages to the appropriate agent and outbound responses to the correct platform adapter. Supports multi-platform agents (one agent, many channels).

Connection status: disconnected -> connecting -> connected -> reconnecting -> error

Channel adapters are registered as extensions via EXTENSION_KIND_MESSAGING_CHANNEL.

See docs/architecture/PLATFORM_SUPPORT.md for platform-specific configuration.

Voice and Telephony

Location: src/voice/

Enable agents to make and receive phone calls via telephony providers.

Call state machine:

initiated --> ringing --> answered --> active --> speaking <--> listening
    |            |           |          |            |            |
    +------------+-----------+----------+------------+------------+
    |                    (any non-terminal state)                 |
    v
[Terminal States]
completed | hangup-user | hangup-bot | timeout | error |
failed | no-answer | busy | voicemail

Providers:

Provider	Support
Twilio	Full voice + SMS
Telnyx	Full voice
Plivo	Full voice
Mock	Testing/development

Key components:

CallManager -- Manages call lifecycle, state transitions, and event dispatch
IVoiceCallProvider -- Interface for telephony provider adapters
telephony-audio.ts -- Audio stream handling and format conversion

Voice providers are registered via EXTENSION_KIND_TOOL with the voice-call-provider category.

Unified Orchestration Layer

Location: src/orchestration/

Three authoring APIs compile to one CompiledExecutionGraph IR executed by a single GraphRuntime. Persistent checkpointing enables time-travel debugging and fault recovery.

Current status: the builders, IR, checkpoints, and base runtime are real. Some advanced routes are still partial in the shared runtime: discovery edges need discovery wiring, personality edges need a trait source, and extension / subgraph execution still requires a higher-level bridge runtime.

API	Level	Use case
`AgentGraph`	Low-level	Explicit nodes, edges, cycles, subgraphs — full graph control
`workflow()`	Mid-level	Deterministic DAG chains with step/branch/parallel — Zod-typed I/O
`mission()`	High-level	Intent-driven — declare goal + constraints, PlanningEngine decides steps

Differentiators vs LangGraph / Mastra: memory-aware state, capability discovery routing, personality-driven edges, inter-step guardrails, streaming at every node transition.

import { AgentGraph, toolNode, gmiNode, START, END } from '@framers/agentos/orchestration';
import { z } from 'zod';

// Low-level: explicit graph with checkpoints
const graph = new AgentGraph({
  input: z.object({ topic: z.string() }),
  scratch: z.object({ draft: z.string().optional() }),
  artifacts: z.object({ status: z.string().optional(), summary: z.string().optional() }),
})
  .addNode('draft', gmiNode({ instructions: 'Draft a concise summary', executionMode: 'single_turn' }))
  .addNode('publish', toolNode('publish_report'))
  .addEdge(START, 'draft')
  .addEdge('draft', 'publish')
  .addEdge('publish', END)
  .compile();

const result = await graph.invoke({ topic: 'quantum computing' });

// Mid-level: deterministic workflow
import { workflow } from '@framers/agentos/orchestration';

const flow = workflow('onboarding')
  .input(z.object({ email: z.string() }))
  .returns(z.object({ userId: z.string() }))
  .step('validate', { tool: 'email_validator' })
  .then('create', { tool: 'user_service' })
  .compile();

// High-level: intent-driven
import { mission } from '@framers/agentos/orchestration';

const researcher = mission('research')
  .input(z.object({ topic: z.string() }))
  .goal('Research {topic} and produce a cited summary')
  .returns(z.object({ summary: z.string() }))
  .planner({ strategy: 'plan_and_execute', maxSteps: 8 })
  .compile();

const plan = await researcher.explain({ topic: 'AI safety' }); // preview plan without executing

See docs/orchestration/UNIFIED_ORCHESTRATION.md, docs/architecture/AGENT_GRAPH.md, docs/orchestration/WORKFLOW_DSL.md, docs/orchestration/MISSION_API.md, docs/orchestration/CHECKPOINTING.md.

Runnable examples: examples/agent-graph.mjs, examples/workflow-dsl.mjs, examples/mission-api.mjs

Legacy WorkflowEngine

The original WorkflowEngine (src/orchestration/workflows/) continues to work for existing consumers. The new orchestration layer is opt-in and runs alongside it.

const definitions = agent.listWorkflowDefinitions();
const instance = await agent.startWorkflow('data-pipeline-v1', input);
const status = await agent.getWorkflow(instance.workflowId);

Multi-Agent Coordination

Location: src/agents/agency/

Enables teams of agents to collaborate on shared goals.

AgencyRegistry -- Register and manage agent teams with role assignments
AgentCommunicationBus -- Inter-agent message passing with typed events and handoffs
AgencyMemoryManager -- Shared memory space with vector search for agency-wide knowledge

graph TD
    AR["AgencyRegistry"]
    AR --> A1["Agent (GMI)<br/><b>Researcher</b>"]
    AR --> A2["Agent (GMI)<br/><b>Writer</b>"]
    AR --> A3["Agent (GMI)<br/><b>Reviewer</b>"]
    AR --> A4["Agent (GMI)<br/><b>Deployer</b>"]
    A1 & A2 & A3 & A4 --> Bus["Communication Bus<br/><i>Events · Handoffs · Sync</i>"]
    A1 & A2 & A3 & A4 --> Mem["Agency Memory Manager<br/><i>Shared vector memory</i>"]
    style AR fill:#1c1c28,stroke:#c9a227,color:#f2f2fa
    style Bus fill:#151520,stroke:#00f5ff,color:#f2f2fa
    style Mem fill:#151520,stroke:#8b5cf6,color:#f2f2fa

See docs/architecture/AGENT_COMMUNICATION.md for the full multi-agent specification.

Observability

Location: src/evaluation/observability/

OpenTelemetry-native observability for tracing, metrics, and cost tracking.

ITracer / Tracer -- Span creation and propagation for distributed tracing
otel.ts -- configureAgentOSObservability() sets up the OpenTelemetry SDK with custom exporters
Metrics -- Token usage, latency percentiles, tool execution counts, error rates
Cost tracking -- Per-request and aggregate cost computation across providers

import { configureAgentOSObservability } from '@framers/agentos';

configureAgentOSObservability({
  serviceName: 'my-agent',
  traceExporter: myOTLPExporter,
  metricExporter: myMetricsExporter,
});

See docs/observability/OBSERVABILITY.md and docs/safety/COST_OPTIMIZATION.md for setup guides.

Skills

Location: src/skills/

Skills are portable, self-describing agent capabilities defined in SKILL.md files.

SkillRegistry -- Discovers and registers available skills
SkillLoader -- Parses SKILL.md format (YAML frontmatter + markdown body)
SKILL.md format -- Declarative skill definition with name, description, required tools, and behavioral instructions

See docs/extensions/SKILLS.md for the skill authoring guide.

Structured Output

Location: src/structured/output/

Extract typed, validated data from unstructured text using JSON Schema.

StructuredOutputManager -- Coordinates schema-constrained generation with validation
JSON Schema validation -- Input/output validation via ajv with format support
Parallel function calls -- Multiple tool invocations in a single LLM turn
Entity extraction -- Named entity recognition with schema constraints

See docs/orchestration/STRUCTURED_OUTPUT.md for usage patterns.

Emergent Capabilities

Location: src/emergent/

Agents with emergent: true create new tools at runtime — compose existing tools via a step DSL or write sandboxed JavaScript. An LLM-as-judge evaluates safety and correctness. Tools earn trust through tiered promotion: session (in-memory) → agent (persisted, auto-promoted after 5+ uses with >0.8 confidence) → shared (human-approved HITL gate).

See docs/architecture/EMERGENT_CAPABILITIES.md for details.

Configuration

Development (Quick Start)

createTestAgentOSConfig() provides sensible defaults for local development:

import { AgentOS } from '@framers/agentos';
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';

const agent = new AgentOS();
await agent.initialize(await createTestAgentOSConfig());

Production

createAgentOSConfig() reads from environment variables:

import { AgentOS } from '@framers/agentos';
import { createAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';

const agent = new AgentOS();
await agent.initialize(await createAgentOSConfig());

Multiple Providers

High-level helpers (recommended): Just set env vars. No config object needed.

# Set as many provider keys as you like -- AgentOS uses them on demand
export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...
export GEMINI_API_KEY=AIza...
export OLLAMA_BASE_URL=http://localhost:11434

import { generateText } from '@framers/agentos';

// Each call picks the right credentials from env vars automatically
await generateText({ provider: 'anthropic', prompt: 'Hello from Claude' });
await generateText({ provider: 'openai',    prompt: 'Hello from GPT' });
await generateText({ provider: 'gemini',    prompt: 'Hello from Gemini' });
await generateText({ provider: 'ollama',    prompt: 'Hello from Llama' });

// Or omit provider entirely -- auto-detects the first configured one
await generateText({ prompt: 'Hello from whichever provider is available' });

Full runtime — configure multiple providers with explicit fallback:

const agent = new AgentOS();
const config = await createTestAgentOSConfig();

await agent.initialize({
  ...config,
  modelProviderManagerConfig: {
    providers: [
      { providerId: 'anthropic', enabled: true, isDefault: true,
        config: { apiKey: process.env.ANTHROPIC_API_KEY } },
      { providerId: 'openai', enabled: true,
        config: { apiKey: process.env.OPENAI_API_KEY } },
      { providerId: 'gemini', enabled: true,
        config: { apiKey: process.env.GEMINI_API_KEY } },
      { providerId: 'ollama', enabled: true,
        config: { baseUrl: process.env.OLLAMA_BASE_URL || 'http://localhost:11434' } },
    ],
  },
  gmiManagerConfig: {
    ...config.gmiManagerConfig,
    defaultGMIBaseConfigDefaults: {
      ...(config.gmiManagerConfig.defaultGMIBaseConfigDefaults ?? {}),
      defaultLlmProviderId: 'anthropic',
      defaultLlmModelId: 'claude-sonnet-4-20250514',
    },
  },
});

// Override per request:
for await (const chunk of agent.processRequest({
  userId: 'user-1',
  sessionId: 'session-1',
  textInput: 'Hello',
  options: {
    preferredProviderId: 'gemini',
    preferredModelId: 'gemini-2.5-flash',
  },
})) { /* ... */ }

Environment Variables

# LLM providers (set at least one)
OPENAI_API_KEY=sk-...                      # OpenAI (GPT-4o, GPT-4o-mini, o1, o3)
ANTHROPIC_API_KEY=sk-ant-...               # Anthropic (Claude Sonnet 4, Claude Haiku)
GEMINI_API_KEY=AIza...                     # Google Gemini (2.5 Flash, 2.0)
OPENROUTER_API_KEY=sk-or-...              # OpenRouter (200+ models, auto-routing)
GROQ_API_KEY=gsk_...                       # Groq (fast inference: Llama 3.3 70B)
TOGETHER_API_KEY=...                       # Together AI (Llama, Mixtral)
MISTRAL_API_KEY=...                        # Mistral (Mistral Large, Small)
XAI_API_KEY=xai-...                        # xAI (Grok-2)
OLLAMA_BASE_URL=http://localhost:11434     # Ollama (local models, no API key needed)

# Database
DATABASE_URL=file:./data/agentos.db

# Observability (optional)
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
OTEL_SERVICE_NAME=my-agent

# Voice/Telephony (optional)
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=...
TELNYX_API_KEY=KEY...

API Reference

AgentOS Class

The main service facade. Implements IAgentOS.

class AgentOS implements IAgentOS {
  // Lifecycle
  initialize(config: AgentOSConfig): Promise<void>;
  shutdown(): Promise<void>;

  // Core interaction (streaming-first)
  processRequest(input: AgentOSInput): AsyncGenerator<AgentOSResponse>;
  handleToolResult(streamId, toolCallId, toolName, toolOutput, isSuccess, errorMessage?):
    AsyncGenerator<AgentOSResponse>;

  // Personas
  listPersonas(): IPersonaDefinition[];
  setActivePersona(personaId: string): Promise<void>;

  // Conversation
  getConversationHistory(sessionId: string): Promise<ConversationContext>;

  // Workflows
  listWorkflowDefinitions(): WorkflowDefinition[];
  startWorkflow(definitionId, input, options?): Promise<WorkflowInstance>;
  getWorkflow(workflowId): Promise<WorkflowInstance | null>;
  updateWorkflowTask(workflowId, taskId, update): Promise<void>;
  queryWorkflows(options?): Promise<WorkflowInstance[]>;

  // Feedback
  submitFeedback(feedback: UserFeedbackPayload): Promise<void>;

  // Exposed managers (for advanced usage)
  readonly llmProviderManager: AIModelProviderManager;
  readonly extensionManager: ExtensionManager;
  readonly conversationManager: ConversationManager;
}

IAgentOS Interface

The core contract. See src/api/interfaces/IAgentOS.ts for the full interface definition.

AgentOSInput

interface AgentOSInput {
  userId: string;
  organizationId?: string;        // Multi-tenant routing
  sessionId: string;
  textInput: string | null;
  visionInput?: VisionInputData;  // Image/video input
  audioInput?: AudioInputData;    // Audio input
  preferredPersonaId?: string;    // Request specific persona
  userApiKeys?: Record<string, string>; // User-provided API keys
  feedback?: UserFeedbackPayload; // Inline feedback
  workflowInvocation?: WorkflowInvocationRequest;
  agencyInvocation?: AgencyInvocationRequest;
  memoryControl?: AgentOSMemoryControl;
  options?: ProcessingOptions;    // Model override, temperature, etc.
}

AgentOSResponse Streaming

All core methods return AsyncGenerator<AgentOSResponse>. Each yielded chunk has a type discriminant:

// Handle all chunk types:
for await (const chunk of agent.processRequest(input)) {
  switch (chunk.type) {
    case AgentOSResponseChunkType.TEXT_DELTA:
      process.stdout.write(chunk.textDelta);
      break;
    case AgentOSResponseChunkType.TOOL_CALL_REQUEST:
      console.log('Tools requested:', chunk.toolCalls);
      break;
    case AgentOSResponseChunkType.TOOL_RESULT_EMISSION:
      console.log('Tool result:', chunk.toolName, chunk.toolResult);
      break;
    case AgentOSResponseChunkType.SYSTEM_PROGRESS:
      console.log('Progress:', chunk.message, chunk.progressPercentage);
      break;
    case AgentOSResponseChunkType.WORKFLOW_UPDATE:
      console.log('Workflow:', chunk.workflowProgress);
      break;
    case AgentOSResponseChunkType.AGENCY_UPDATE:
      console.log('Agency event:', chunk.agencyEvent);
      break;
    case AgentOSResponseChunkType.ERROR:
      console.error('Error:', chunk.error);
      break;
    case AgentOSResponseChunkType.FINAL_RESPONSE:
      console.log('Complete:', chunk.finalText);
      break;
  }
}

ITool Interface

See the Tool System section above for the full interface. Tools are registered via extension packs:

const descriptor: ExtensionDescriptor<ITool> = {
  id: 'my-tool',
  kind: EXTENSION_KIND_TOOL,
  payload: myToolImplementation,
};

ExtensionDescriptor

See the Extension System section above for the full type definition and all 12 extension kinds.

IGuardrailService

interface IGuardrailService {
  evaluateInput?(
    input: AgentOSInput,
    context: GuardrailContext,
  ): Promise<GuardrailEvaluationResult>;

  evaluateOutput?(
    output: string,
    context: GuardrailContext,
  ): Promise<GuardrailEvaluationResult>;
}

interface GuardrailEvaluationResult {
  action: GuardrailAction;        // ALLOW | FLAG | SANITIZE | BLOCK
  reason?: string;                // Human-readable explanation
  reasonCode?: string;            // Machine-readable code
  modifiedText?: string;          // Required when action is SANITIZE
  metadata?: Record<string, any>; // Additional context for logging
}

IHumanInteractionManager

interface IHumanInteractionManager {
  requestApproval(action: PendingAction): Promise<ApprovalDecision>;
  requestClarification(request: ClarificationRequest): Promise<ClarificationResponse>;
  escalateToHuman(context: EscalationContext): Promise<EscalationResult>;
  getPendingActions(): Promise<PendingAction[]>;
}

Usage Examples

Orchestration Patterns

Use the new orchestration layer based on how much control you need:

workflow() for deterministic sequential/parallel DAGs
AgentGraph for explicit cycles, retries, and custom routing
mission() when you know the goal but want the planner to decide the steps

import {
  AgentGraph,
  END,
  START,
  gmiNode,
  mission,
  toolNode,
  workflow,
} from '@framers/agentos/orchestration';
import { z } from 'zod';

// 1. Deterministic DAG: sequential steps with a parallel fan-out/join
const onboarding = workflow('user-onboarding')
  .input(z.object({ email: z.string().email() }))
  .returns(z.object({ userId: z.string() }))
  .step('validate', { tool: 'email_validator' })
  .then('create-account', { tool: 'user_service' })
  .parallel(
    [
      { tool: 'send_welcome_email' },
      { tool: 'provision_default_workspace' },
    ],
    {
      strategy: 'all',
      merge: { 'scratch.completedTasks': 'concat' },
    },
  )
  .compile();

// 2. Explicit graph: fixed publish pipeline with explicit nodes and edges
const reviewGraph = new AgentGraph({
  input: z.object({ topic: z.string() }),
  scratch: z.object({ draft: z.string().optional() }),
  artifacts: z.object({ status: z.string().optional(), summary: z.string().optional() }),
})
  .addNode('draft', gmiNode({ instructions: 'Draft the release note.', executionMode: 'single_turn' }))
  .addNode('publish', toolNode('publish_report'))
  .addEdge(START, 'draft')
  .addEdge('draft', 'publish')
  .addEdge('publish', END)
  .compile();

// 3. Goal-first mission: preview the generated plan before running it
const researcher = mission('deep-research')
  .input(z.object({ topic: z.string() }))
  .goal('Research {{topic}} thoroughly and produce a cited summary')
  .returns(z.object({ summary: z.string() }))
  .planner({ strategy: 'plan_and_execute', maxSteps: 8 })
  .compile();

const preview = await researcher.explain({ topic: 'AI safety' });
console.log(preview.steps.map((step) => step.id));

For deeper examples, see docs/architecture/AGENT_GRAPH.md, docs/orchestration/WORKFLOW_DSL.md, docs/orchestration/MISSION_API.md, the runnable examples examples/agent-graph.mjs, examples/workflow-dsl.mjs, examples/mission-api.mjs, and the legacy dependency-ordered example examples/multi-agent-workflow.mjs.

Streaming Chat

import { AgentOS, AgentOSResponseChunkType } from '@framers/agentos';
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';

const agent = new AgentOS();
await agent.initialize(await createTestAgentOSConfig());

for await (const chunk of agent.processRequest({
  userId: 'user-1',
  sessionId: 'session-1',
  textInput: 'Explain how TCP handshakes work',
})) {
  if (chunk.type === AgentOSResponseChunkType.TEXT_DELTA) {
    process.stdout.write(chunk.textDelta);
  }
}

Adding Tools

Tools are registered via extension packs and called automatically by the model:

import {
  AgentOS,
  AgentOSResponseChunkType,
  EXTENSION_KIND_TOOL,
  type ExtensionManifest,
  type ExtensionPack,
  type ITool,
} from '@framers/agentos';
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';

const weatherTool: ITool = {
  id: 'get-weather',
  name: 'get_weather',
  displayName: 'Get Weather',
  description: 'Returns current weather for a city.',
  category: 'utility',
  hasSideEffects: false,
  inputSchema: {
    type: 'object',
    properties: { city: { type: 'string', description: 'City name' } },
    required: ['city'],
  },
  execute: async (args) => ({
    success: true,
    output: { text: `Weather in ${args.city}: 22 C, partly cloudy` },
  }),
};

const manifest: ExtensionManifest = {
  packs: [{
    factory: async () => ({
      name: 'my-tools',
      descriptors: [{ id: weatherTool.id, kind: EXTENSION_KIND_TOOL, payload: weatherTool }],
    } satisfies ExtensionPack),
  }],
};

const agent = new AgentOS();
const config = await createTestAgentOSConfig();
await agent.initialize({ ...config, extensionManifest: manifest });

for await (const chunk of agent.processRequest({
  userId: 'user-1',
  sessionId: 'session-1',
  textInput: 'What is the weather in Tokyo?',
})) {
  switch (chunk.type) {
    case AgentOSResponseChunkType.TEXT_DELTA:
      process.stdout.write(chunk.textDelta);
      break;
    case AgentOSResponseChunkType.TOOL_CALL_REQUEST:
      console.log('Tool calls:', chunk.toolCalls);
      break;
    case AgentOSResponseChunkType.TOOL_RESULT_EMISSION:
      console.log('Tool result:', chunk.toolResult);
      break;
  }
}

Multi-Agent Collaboration

import { AgentCommunicationBus } from '@framers/agentos';

const bus = new AgentCommunicationBus({
  routingConfig: { enableRoleRouting: true, enableLoadBalancing: true },
});

bus.registerAgent('coordinator-gmi', 'agency-docs', 'coordinator');
bus.registerAgent('researcher-gmi', 'agency-docs', 'researcher');
bus.registerAgent('writer-gmi', 'agency-docs', 'writer');

bus.subscribe(
  'researcher-gmi',
  async (message) => {
    if (message.type !== 'question') return;
    await bus.sendToAgent(message.fromAgentId, {
      type: 'answer',
      fromAgentId: 'researcher-gmi',
      content: { findings: ['auth edge cases', 'missing audit trail', 'weak retry policy'] },
      inReplyTo: message.messageId,
      priority: 'normal',
    });
  },
  { messageTypes: ['question'] },
);

bus.subscribe(
  'writer-gmi',
  async (message) => {
    if (message.type !== 'task_delegation') return;
    await bus.sendToAgent(message.fromAgentId, {
      type: 'answer',
      fromAgentId: 'writer-gmi',
      content: { accepted: true },
      inReplyTo: message.messageId,
      priority: 'normal',
    });
  },
  { messageTypes: ['task_delegation'] },
);

await bus.sendToRole('agency-docs', 'researcher', {
  type: 'task_delegation',
  fromAgentId: 'coordinator-gmi',
  content: { topic: 'auth module', instructions: 'Find the risky edge cases.' },
  priority: 'high',
});

const review = await bus.requestResponse('researcher-gmi', {
  type: 'question',
  fromAgentId: 'coordinator-gmi',
  content: 'What are the top three findings?',
  priority: 'high',
  timeoutMs: 30_000,
});

const handoff = await bus.handoff('researcher-gmi', 'writer-gmi', {
  taskId: 'auth-audit',
  taskDescription: 'Turn the findings into a release note draft',
  progress: 0.8,
  completedWork: [review.content],
  remainingWork: ['Write polished summary'],
  context: { audience: 'engineering' },
  reason: 'completion',
  instructions: 'Summarize the findings in concise release-note style.',
});

console.log(handoff.accepted);

Runnable example: examples/agent-communication-bus.mjs

Human-in-the-Loop Approvals

import { HumanInteractionManager } from '@framers/agentos';

const hitl = new HumanInteractionManager({ defaultTimeoutMs: 300_000 });

const decision = await hitl.requestApproval({
  actionId: 'archive-inactive',
  description: 'Archive 50K inactive accounts older than 2 years',
  severity: 'high',
  category: 'data_modification',
  agentId: 'data-cleanup-agent',
  context: { affectedRows: 50_000, table: 'users' },
  reversible: true,
  potentialConsequences: ['Users will lose access to archived data'],
  alternatives: [
    { alternativeId: 'soft_delete', description: 'Mark as inactive instead' },
    { alternativeId: 'export_first', description: 'Export to CSV before archiving' },
  ],
});

if (decision.approved) {
  await executeArchive();
} else if (decision.selectedAlternative) {
  await executeAlternative(decision.selectedAlternative);
}

Structured Data Extraction

import { AgentOS, StructuredOutputManager } from '@framers/agentos';
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';

const agent = new AgentOS();
await agent.initialize(await createTestAgentOSConfig());

const structured = new StructuredOutputManager({
  llmProviderManager: agent.llmProviderManager,
});

const contact = await structured.generate({
  prompt: 'Extract: "Meeting with Sarah Chen (sarah@startup.io) on Jan 15 re: Series A"',
  schema: {
    type: 'object',
    properties: {
      name: { type: 'string' },
      email: { type: 'string', format: 'email' },
      date: { type: 'string' },
      topic: { type: 'string' },
    },
    required: ['name', 'email'],
  },
  schemaName: 'ContactInfo',
});
// Result: { name: 'Sarah Chen', email: 'sarah@startup.io', date: 'Jan 15', topic: 'Series A' }

RAG Memory

import { AgentOS } from '@framers/agentos';
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';

const agent = new AgentOS();
const config = await createTestAgentOSConfig();

await agent.initialize({
  ...config,
  ragConfig: {
    embeddingManagerConfig: {
      embeddingModels: [
        { modelId: 'text-embedding-3-small', providerId: 'openai',
          dimension: 1536, isDefault: true },
      ],
    },
    vectorStoreManagerConfig: {
      managerId: 'rag-vsm',
      providers: [
        { id: 'sql-store', type: 'sql', storage: { filePath: './data/vectors.db' } },
      ],
      defaultProviderId: 'sql-store',
      defaultEmbeddingDimension: 1536,
    },
    dataSourceConfigs: [{
      dataSourceId: 'conversations',
      displayName: 'Conversation Memory',
      vectorStoreProviderId: 'sql-store',
      actualNameInProvider: 'conversations',
      embeddingDimension: 1536,
    }],
    retrievalAugmentorConfig: {
      defaultDataSourceId: 'conversations',
      categoryBehaviors: [],
    },
  },
});

// Agent now retrieves relevant context from vector memory before responding

Custom Guardrails

import {
  AgentOS,
  type IGuardrailService,
  GuardrailAction,
  type GuardrailInputPayload,
  type GuardrailOutputPayload,
} from '@framers/agentos';
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';

const piiGuardrail: IGuardrailService = {
  config: {
    evaluateStreamingChunks: true,
    canSanitize: true,
  },

  async evaluateInput({ input }: GuardrailInputPayload) {
    // Check for SSN patterns in user input
    const ssnPattern = /\b\d{3}-\d{2}-\d{4}\b/g;
    if (input.textInput && ssnPattern.test(input.textInput)) {
      return {
        action: GuardrailAction.SANITIZE,
        modifiedText: input.textInput.replace(ssnPattern, '[SSN REDACTED]'),
        reason: 'PII detected in user input',
        reasonCode: 'PII_SSN',
      };
    }
    return { action: GuardrailAction.ALLOW };
  },

  async evaluateOutput({ chunk }: GuardrailOutputPayload) {
    const text =
      chunk.type === 'TEXT_DELTA'
        ? chunk.textDelta ?? ''
        : chunk.type === 'FINAL_RESPONSE'
          ? chunk.finalResponseText ?? ''
          : '';

    if (text.toLowerCase().includes('password')) {
      return {
        action: GuardrailAction.BLOCK,
        reason: 'Output contains potentially sensitive credential information',
        reasonCode: 'CREDENTIAL_LEAK',
      };
    }
    return { action: GuardrailAction.ALLOW };
  },
};

const agent = new AgentOS();
const config = await createTestAgentOSConfig();
await agent.initialize({ ...config, guardrailService: piiGuardrail });

For production PII redaction, use the built-in createPiiRedactionGuardrail() extension instead of hand-rolled regex. It provides four-tier detection (regex + NLP + BERT NER + LLM-as-judge), streaming support, and configurable redaction styles. See GUARDRAILS_USAGE.md for full examples.

Channel Adapters

import {
  EXTENSION_KIND_MESSAGING_CHANNEL,
  type ExtensionManifest,
  type IChannelAdapter,
  type ChannelPlatform,
} from '@framers/agentos';

// Implement a custom channel adapter
const myAdapter: IChannelAdapter = {
  platform: 'webchat' as ChannelPlatform,
  capabilities: new Set(['text', 'rich_text', 'images', 'typing_indicator']),

  async connect() { /* establish connection */ },
  async disconnect() { /* clean up */ },
  async sendMessage(channelId, message) { /* send outbound */ },
  onMessage(handler) { /* register inbound handler */ },
  getConnectionInfo() {
    return { status: 'connected', connectedSince: new Date().toISOString() };
  },
};

// Register via extension manifest
const manifest: ExtensionManifest = {
  packs: [{
    factory: async () => ({
      name: 'my-channels',
      descriptors: [{
        id: 'webchat-adapter',
        kind: EXTENSION_KIND_MESSAGING_CHANNEL,
        payload: myAdapter,
      }],
    }),
  }],
};

Voice Calls

import { CallManager, type IVoiceCallProvider } from '@framers/agentos';

const callManager = new CallManager();

// Initiate an outbound call
const call = await callManager.initiateCall({
  provider: 'twilio',
  to: '+1234567890',
  from: '+0987654321',
  agentId: 'support-agent',
});

// Monitor call state transitions
call.on('stateChange', (newState) => {
  console.log(`Call ${call.id}: ${newState}`);
  // initiated -> ringing -> answered -> active -> speaking <-> listening -> completed
});

// Handle call completion
call.on('completed', (summary) => {
  console.log('Duration:', summary.durationMs);
  console.log('Transcript:', summary.transcript);
});

Package Exports

AgentOS provides 112 export paths for fine-grained imports. Key entry points:

// Main entry -- all public types and classes
import { AgentOS, AgentOSResponseChunkType, /* ... */ } from '@framers/agentos';

// Configuration
import { createAgentOSConfig, createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';

// Safety primitives
import { CircuitBreaker, CostGuard, StuckDetector } from '@framers/agentos/safety/runtime';

// Guardrails
import { GuardrailAction } from '@framers/agentos/safety/guardrails';
import { ParallelGuardrailDispatcher } from '@framers/agentos/safety/guardrails';

// Tools
import type { ITool, ToolExecutionResult } from '@framers/agentos/core/tools';

// HITL
import type { IHumanInteractionManager } from '@framers/agentos/orchestration/hitl';

// RAG
import { VectorStoreManager, EmbeddingManager, RetrievalAugmentor } from '@framers/agentos/rag';
import { GraphRAGEngine } from '@framers/agentos/rag/graphrag';

// Skills
import { SkillRegistry, SkillLoader } from '@framers/agentos/skills';

// Extension runtime helpers and built-in guardrail packs
import { SharedServiceRegistry } from '@framers/agentos';
import { createPiiRedactionGuardrail } from '@framers/agentos-ext-pii-redaction';
import { createMLClassifierGuardrail } from '@framers/agentos-ext-ml-classifiers';
import { createTopicalityGuardrail } from '@framers/agentos-ext-topicality';
import { createCodeSafetyGuardrail } from '@framers/agentos-ext-code-safety';
import { createGroundingGuardrail } from '@framers/agentos-ext-grounding-guard';

// Deep imports (wildcard exports)
import { SomeType } from '@framers/agentos/safety/runtime/CircuitBreaker';
import { SomeConfig } from '@framers/agentos/config/ToolOrchestratorConfig';

Wildcard exports support paths up to 4 levels deep:

./* -- dist/*.js
./*/* -- dist/*/*.js
./*/*/* -- dist/*/*/*.js
./*/*/*/* -- dist/*/*/*/*.js

Internal Documentation

The docs/ directory contains specification and reference documents:

Document	Description
`AGENCY_API.md`	`agency()` reference: all 5 strategies, HITL, guardrails, RAG, voice, nested agencies, full-featured example
`HIGH_LEVEL_API.md`	`generateText()`, `streamText()`, `generateObject()`, `streamObject()`, `embedText()`, `generateImage()`, single `agent()`
`ARCHITECTURE.md`	Complete system architecture with data flow diagrams
`SAFETY_PRIMITIVES.md`	Circuit breaker, cost guard, stuck detection, dedup API reference
`PLANNING_ENGINE.md`	ReAct reasoning, multi-step task planning specification
`HUMAN_IN_THE_LOOP.md`	Approval workflows, clarification, escalation patterns
`GUARDRAILS_USAGE.md`	Input/output guardrail implementation patterns
`RAG_MEMORY_CONFIGURATION.md`	Vector store setup, embedding models, data source config
`MULTIMODAL_RAG.md`	Image, audio, and document RAG pipelines
`STRUCTURED_OUTPUT.md`	JSON schema validation, entity extraction, function calling
`AGENT_COMMUNICATION.md`	Inter-agent messaging, handoffs, shared memory
`TOOL_CALLING_AND_LOADING.md`	Tool registration, discovery, execution pipeline
`OBSERVABILITY.md`	OpenTelemetry setup, custom spans, metrics export
`COST_OPTIMIZATION.md`	Token usage monitoring, caching strategies, model routing
`SKILLS.md`	SKILL.md format specification, skill authoring guide
`PLATFORM_SUPPORT.md`	Channel platform capabilities and adapter configuration
`ECOSYSTEM.md`	Extension ecosystem, official packs, community extensions
`PROVENANCE_IMMUTABILITY.md`	Sealed agents, signed event ledger, external anchoring
`IMMUTABLE_AGENTS.md`	Agent sealing, toolset manifests, revision tracking
`RFC_EXTENSION_STANDARDS.md`	Extension pack authoring standards and conventions
`EVALUATION_FRAMEWORK.md`	Agent evaluation, benchmarking, quality metrics
`RECURSIVE_SELF_BUILDING_AGENTS.md`	Self-modifying agent patterns
`LOGGING.md`	Structured logging configuration with pino
`CLIENT_SIDE_STORAGE.md`	Browser-compatible storage adapters
`SQL_STORAGE_QUICKSTART.md`	SQLite/Postgres setup with `@framers/sql-storage-adapter`
`RELEASING.md`	Release process and semantic versioning

Key Design Patterns

Interface-driven design -- All major components define interface contracts (IAgentOS, ITool, IGuardrailService, IChannelAdapter, IVoiceCallProvider, IVectorStore, etc.). Implementations are swappable.
Streaming-first -- Core interaction methods return AsyncGenerator<AgentOSResponse> for token-level streaming with natural backpressure. Consumers process chunks as they arrive.
Extension system -- Pluggable components via ExtensionDescriptor with 12 kinds, priority stacking, lifecycle hooks, and secret management. Extensions can be loaded from npm packages, local modules, or inline factories.
Multi-provider -- LLM providers, vector stores, voice providers, and channel adapters all support multiple backend implementations with runtime switching.
Safety layering -- Defense-in-depth: input guardrails, output guardrails, circuit breakers, cost guards, tool execution guards, stuck detection, action deduplication, and HITL approval gates.
Observability -- OpenTelemetry integration throughout the stack with distributed tracing, custom metrics, cost tracking, and structured logging via pino.

Contributing

git clone https://github.com/framersai/agentos.git
cd agentos
pnpm install
pnpm run build
pnpm run test

Available scripts:

Script	Purpose
`pnpm run build`	Clean, compile TypeScript, resolve aliases, fix ESM imports
`pnpm run typecheck`	Type-check without emitting
`pnpm run lint`	Strip non-breaking spaces + ESLint
`pnpm run test`	Run vitest test suite
`pnpm run dev:test`	Run vitest in watch mode
`pnpm run docs`	Generate TypeDoc API documentation

We use Conventional Commits: feat: (minor), fix: (patch), BREAKING CHANGE: (major).

See the Contributing Guide for details.

License

Apache 2.0 -- Frame.dev

Built by Frame.dev · @framersai

Name		Name	Last commit message	Last commit date
Latest commit History 906 Commits
.github		.github
assets		assets
db_data		db_data
docs		docs
examples		examples
knowledge		knowledge
scripts		scripts
src		src
tests		tests
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
codecov.yml		codecov.yml
eslint.config.js		eslint.config.js
package.json		package.json
release.config.js		release.config.js
tmp-parity.sqlite		tmp-parity.sqlite
tsconfig.build.json		tsconfig.build.json
tsconfig.json		tsconfig.json
tsdoc.json		tsdoc.json
typedoc.json		typedoc.json
typedoc.modules.json		typedoc.modules.json
vitest.config.ts		vitest.config.ts

Adapter	Runtime	Use Case
`better-sqlite3`	Node.js	Production (default)
`pg`	Node.js	PostgreSQL for cloud deployments
`sql.js`	Browser/WASM	Client-side storage
`capacitor`	Mobile	iOS/Android via Capacitor
`electron`	Desktop	Electron apps with IPC bridge
`indexeddb`	Browser	Fallback browser storage
`memory`	Any	Testing and development

Folders and files

Latest commit

History

Repository files navigation

AgentOS

Table of Contents

Overview

Ecosystem

Guardrail Extensions

Quick Start

1. Generate Text (Provider-First)

2. Agent with Personality & Memory

3. Multimodal RAG

4. Structured Output (Zod Validation)

5. Workflows & Graphs

6. Multi-Agent Teams

7. Voice Pipeline

8. Guardrails & Security

Default Models Per Provider

System Architecture

Architecture Diagram

Request Lifecycle

Layer Breakdown

Core Modules

API Layer

Cognitive Substrate (GMI)

LLM Provider Management

Tool System

Extension System

Planning Engine

Conversation Management

RAG (Retrieval Augmented Generation)

Bundled Platform Knowledge

Safety and Guardrails

Safety Primitives (safety/runtime/)

Guardrails (safety/guardrails/)

Human-in-the-Loop (HITL)

Channels System

Voice and Telephony

Unified Orchestration Layer

Legacy WorkflowEngine

Multi-Agent Coordination

Observability

Skills

Structured Output

Emergent Capabilities

Configuration

Development (Quick Start)

Production

Multiple Providers

Environment Variables

API Reference

AgentOS Class

IAgentOS Interface

AgentOSInput

AgentOSResponse Streaming

ITool Interface

ExtensionDescriptor

IGuardrailService

IHumanInteractionManager

Usage Examples

Orchestration Patterns

Streaming Chat

Adding Tools

Multi-Agent Collaboration

Human-in-the-Loop Approvals

Structured Data Extraction

RAG Memory

Custom Guardrails

Channel Adapters

Voice Calls

Package Exports

Internal Documentation

Key Design Patterns

Contributing

License

About

Topics

Resources

License

Safety Primitives (`safety/runtime/`)

Guardrails (`safety/guardrails/`)

Packages