Standardize tool invocation contracts and fail-open tooling metadata (Vibe Kanban)

[jbax1899]

Summary

This PR establishes a single Tool Invocation Contract v1 across planner, orchestration, and runtime boundaries so tool request, eligibility, and outcome states are serialized consistently.

What Changed

• Added canonical tool contract types in @footnote/contracts/ethics-core:
  • ToolInvocationIntent (planner intent)
  • ToolInvocationRequest (orchestrator eligibility)
  • ToolExecutionContext (runtime outcome)
  • ToolInvocationReasonCode and expanded ExecutionReasonCode for tool-specific fail-open states
• Re-exported these new types from contract index modules and updated web schemas to validate the expanded reason code set.
• Updated backend orchestration and metadata flow:
  • chatOrchestrator now produces serializable tool intent/request context from planner search decisions.
  • chatService now prioritizes explicit orchestrator/runtime tool outcomes before fallback inference.
  • openaiService metadata normalization now defaults failed tool outcomes to ool_execution_error.
• Updated agent runtime integration:
  • Added optional GenerationResult.toolExecution at the runtime boundary.
  • �oltagentRuntime now emits canonical tool outcomes, including fail-open ool_unavailable when provider tooling is not supported.
• Updated �pi-client exports with a shared tool execution type alias for downstream consumers.
• Added architecture documentation at docs/architecture/tool-invocation-contract-v1.md with mapping rules and success/fail-open metadata examples.
• Added/updated tests to cover runtime tool outcome emission and metadata normalization behavior.

Why

Task context required one stable, framework-agnostic, serializable contract for tool behavior across planner -> orchestrator -> runtime, with predictable fail-open handling and clear operator/UI-visible outcomes (�xecuted / skipped / ailed +
easonCode).

This implementation aligns tool semantics across packages, reduces contract drift risk, and preserves backend authority over provenance, trace metadata, and cost ownership.

Important Implementation Details

• Contract shape is additive and backward-compatible for existing metadata consumers.
• Tool reason codes are now explicit for common fail-open paths ( ool_not_requested, ool_unavailable, ool_execution_error, etc.).
• Backend remains the canonical metadata authority; runtime can report tool outcomes but backend still normalizes final trace metadata.
• Orchestrator-to-runtime search handling remains fail-open: unsupported tooling degrades to generation without blocking response flow.

This PR was written using Vibe Kanban

Summary by CodeRabbit

• New Features

  • Standardized tool execution tracking across the request lifecycle with defined outcome states (executed, skipped, failed).
  • Added reason codes to explain why tools were skipped or failed.
  • Tool execution details now surface in response metadata.

• Documentation

  • Added architecture documentation defining the Tool Invocation Contract v1 with standardized tool behavior specifications.

• Tests

  • Added test coverage for failed tool execution scenarios.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR introduces the "Tool Invocation Contract v1," a standardized framework for capturing tool-related behavior across three lifecycle stages: planner intent (ToolInvocationIntent), orchestrator eligibility (ToolInvocationRequest), and runtime outcome (ToolExecutionContext). New types are defined, integrated into the agent runtime to compute tool execution metadata, and propagated through backend orchestration and response metadata pipelines.

Changes

Cohort / File(s)	Summary
Type Contract Definitions packages/contracts/src/ethics-core/types.ts, packages/contracts/src/ethics-core/index.ts, packages/contracts/src/index.ts, packages/contracts/src/web/schemas.ts	Added ToolInvocationName, ToolInvocationReasonCode, ToolInvocationIntent, ToolInvocationRequest, and ToolExecutionContext types; extended ExecutionReasonCode with tool-specific reason codes (tool_not_requested, tool_unavailable, tool_execution_error); updated schema validation to include new reason codes.
Documentation docs/architecture/tool-invocation-contract-v1.md	New architecture documentation defining the three-stage tool invocation contract with canonical outcome states, mapping rules between stages, and example JSON payloads.
Runtime Integration packages/agent-runtime/src/index.ts, packages/agent-runtime/src/voltagentRuntime.ts, packages/agent-runtime/test/voltagentRuntime.test.ts	Added toolExecution field to GenerationResult; implemented logic in normalizeVoltAgentResult to infer or compute tool execution metadata based on search presence and retrieval artifacts; updated tests to assert tool execution context in search and fallback scenarios.
API Client Exports packages/api-client/src/chat.ts, packages/api-client/src/index.ts	Introduced ChatToolExecutionContext type alias and re-exported it for API client consumers.
Backend Orchestration packages/backend/src/services/chatOrchestrator.ts, packages/backend/src/services/chatService.ts, packages/backend/src/services/openaiService.ts, packages/backend/test/openaiService.metadata.test.ts	Added buildWebSearchToolIntent to derive planner-level tool intents; introduced toolRequestContext for orchestrator eligibility decisions; updated effectiveToolExecutionContext resolution to preserve upstream tool outcomes and runtime-provided execution metadata; refined tool execution normalization in response metadata with status-specific reasonCode fallbacks; added test coverage for failed tool normalization.

Sequence Diagram(s)

sequenceDiagram
    participant Planner
    participant Orchestrator
    participant Runtime as Runtime/Provider
    participant Metadata as Response Metadata

    Planner->>Orchestrator: ToolInvocationIntent<br/>(toolName, requested, input?)
    
    Orchestrator->>Orchestrator: Evaluate eligibility<br/>based on provider support
    
    Orchestrator->>Runtime: ToolInvocationRequest<br/>(toolName, requested,<br/>eligible, reasonCode?)
    
    Runtime->>Runtime: Execute or skip<br/>based on eligibility
    
    Runtime-->>Orchestrator: ToolExecutionContext<br/>(toolName, status,<br/>reasonCode?, durationMs?)
    
    Orchestrator->>Metadata: Record execution<br/>in ResponseMetadata.execution[]<br/>with kind="tool"
    
    Metadata-->>Planner: Final metadata with<br/>tool outcome + reasonCode

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• Implement planner-only search degradation and user signals #201: Modifies chatOrchestrator.ts and execution metadata handling; overlaps with tool invocation pipeline and planner-level context changes.
• Unify chat prompting and isolate VoltAgent logging #194: Updates voltagentRuntime.ts with logging plumbing; shares the same module where tool execution inference and fallback handling are now implemented.
• Unify execution timeline formatting across web and Discord #204: Extends execution metadata schema and reasonCode enumerations; aligns with the new tool-specific reason codes added to ExecutionReasonCode.

Poem

🐰 Tool contracts bloom in three-stage grace,
From planner's intent to runtime's embrace,
Metadata flows like clover through the field,
Each hop a reason, each status revealed,
✨ The rabbit's lore now contracts real!

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title directly summarizes the main objective: establishing standardized tool invocation contracts and implementing fail-open metadata handling across the planner-orchestrator-runtime boundary.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch vk/5539-tool-invocation

Warning

Review ran into problems

🔥 Problems

Timed out fetching pipeline failures after 30000ms

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

packages/backend/src/services/chatOrchestrator.ts (1)

291-305: Clarify the semantic of eligible when requested=false.

When toolIntent.requested=false, the code sets eligible: false with reasonCode: 'tool_not_requested'. While semantically reasonable, consider whether eligible has meaning when the tool was never requested.

This creates a fully-defined state for all paths, which aids analytics consistency. If this is intentional defensive design, a brief inline comment explaining the rationale would help future maintainers.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@packages/backend/src/services/chatOrchestrator.ts` around lines 291 - 305,
The code sets toolRequestContext (type ToolInvocationRequest) with eligible:
false when buildWebSearchToolIntent(...).requested is false; add a short inline
comment next to the branch that explains the semantic choice (i.e., when
requested === false we intentionally set eligible to false and provide
reasonCode 'tool_not_requested' so analytics/consumers always see a
fully-defined invocation state), or alternatively make eligible undefined if you
prefer to express "not applicable" — update the branch around toolRequestContext
(the object literal for web_search when requested is false) to include that
explanatory comment referencing the reasonCode and why eligible is set.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@packages/backend/src/services/chatOrchestrator.ts`:
- Around line 291-305: The code sets toolRequestContext (type
ToolInvocationRequest) with eligible: false when
buildWebSearchToolIntent(...).requested is false; add a short inline comment
next to the branch that explains the semantic choice (i.e., when requested ===
false we intentionally set eligible to false and provide reasonCode
'tool_not_requested' so analytics/consumers always see a fully-defined
invocation state), or alternatively make eligible undefined if you prefer to
express "not applicable" — update the branch around toolRequestContext (the
object literal for web_search when requested is false) to include that
explanatory comment referencing the reasonCode and why eligible is set.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b136d619-9cc7-48ce-8068-2e9b2224908d

📥 Commits

Reviewing files that changed from the base of the PR and between d132733 and dddc5c0.

📒 Files selected for processing (14)

• docs/architecture/tool-invocation-contract-v1.md
• packages/agent-runtime/src/index.ts
• packages/agent-runtime/src/voltagentRuntime.ts
• packages/agent-runtime/test/voltagentRuntime.test.ts
• packages/api-client/src/chat.ts
• packages/api-client/src/index.ts
• packages/backend/src/services/chatOrchestrator.ts
• packages/backend/src/services/chatService.ts
• packages/backend/src/services/openaiService.ts
• packages/backend/test/openaiService.metadata.test.ts
• packages/contracts/src/ethics-core/index.ts
• packages/contracts/src/ethics-core/types.ts
• packages/contracts/src/index.ts
• packages/contracts/src/web/schemas.ts