feat(backend): weather.gov pilot via backend tool registry

[jbax1899]

• add backend-owned weather.gov forecast tool pilot with fail-open behavior
• normalize weather request/output and include provenance metadata
• refactor tool integration into backend tool policy + registry + weather adapter seams
• harden planner/orchestrator handling for weather/search interactions and compact weather tool context

Summary by CodeRabbit

• New Features

  • Added weather forecast integration (weather.gov) with planner support, structured weather inputs, single-tool policy favoring weather over web search, and fail-open behavior on tool errors.

• Documentation

  • Added pilot evaluation notes and a backend tools contract describing selection, telemetry, and adapter expectations.

• Tests

  • Added end-to-end and unit tests for planner normalization, orchestration/tool selection, adapter integration, and fail-open scenarios.

[coderabbitai]

📝 Walkthrough

Walkthrough

Integrated a backend-owned, read-only Weather.gov forecast tool into chat orchestration: planner-level weather types, single-tool policy preferring weather over web search, tool registry/adapters with fail-open execution and telemetry, server/handler wiring, prompts/schema updates, and comprehensive tests and docs.

Changes

Cohort / File(s)	Summary
Documentation & Prompts docs/status/2026-03-27-weathergov-pilot-eval.md, packages/backend/src/services/tools/README.md, packages/prompts/src/defaults.yaml	Added pilot evaluation doc and backend tools contract; extended planner prompt/schema to accept optional generation.weather with constrained location/horizon semantics and conflict guidance (weather preferred over search).
Handler & Server Wiring packages/backend/src/handlers/chat.ts, packages/backend/src/server.ts	Threaded optional weatherForecastTool dependency through server init into chat handler/orchestrator creation and instantiated the Weather.gov tool at startup.
Generation & Contracts Types packages/backend/src/services/chatGenerationTypes.ts, packages/contracts/src/ethics-core/types.ts	Added weather input types (lat_lon/gridpoint) and ChatGenerationWeatherRequest; extended plan/contract types to include weather shapes and new tool reason codes and weather_forecast tool name.
Planner Normalization packages/backend/src/services/chatPlanner.ts	Added normalizeWeatherLocation/normalizeWeatherRequest, clamped horizonPeriods, accumulated reasoning suffixes for disabled tool requests, and ensured action branches clear generation.weather when irrelevant.
Orchestration & Service packages/backend/src/services/chatOrchestrator.ts, packages/backend/src/services/chatService.ts	Orchestrator accepts optional weatherForecastTool, applies applySingleToolPolicy, centralizes tool selection via resolveToolSelection, executes selected tool, injects tool result as a system message when present, and tightens effective toolExecutionContext resolution.
Tool Infrastructure (types & policy) packages/backend/src/services/tools/toolTypes.ts, packages/backend/src/services/tools/toolPolicy.ts	Added BackendToolSelection and ToolPolicyLogEvent types and applySingleToolPolicy which deterministically drops web_search when weather is requested.
Tool Registry & Adapter packages/backend/src/services/tools/toolRegistry.ts, packages/backend/src/services/tools/weatherForecastToolAdapter.ts	Registry builds intents/requests, resolves selection (including eligibility/reasonCode), and executes selected tool; adapter calls weatherForecastTool.fetchForecast, formats delimiter-wrapped tool-result message, returns toolExecutionContext with mapped reason codes and duration, and fails open on thrown errors.
Weather.gov Adapter packages/backend/src/services/weatherGovForecastTool.ts	Implemented createWeatherGovForecastTool with /points→forecast discovery, GET/timeout/abort handling, transport-to-reasonCode mapping, forecast period normalization/truncation, provenance metadata, and fully serializable structured result types.
OpenAI / Reason Code Normalization packages/backend/src/services/openaiService.ts	Extended normalizeToolReasonCode passthrough list to include tool_timeout, tool_http_error, tool_network_error, and tool_invalid_response.
Tests — Orchestrator / Planner / Service / Tools / Adapter packages/backend/test/chatOrchestrator.test.ts, packages/backend/test/chatPlanner.test.ts, packages/backend/test/chatService.test.ts, packages/backend/test/toolRegistry.test.ts, packages/backend/test/weatherGovForecastTool.test.ts	Added tests for weather normalization, single-tool policy, injection of backend tool message, fails-open behavior, registry/policy semantics, and weather.gov adapter endpoint resolution + timeout handling; updated existing orchestrator/prompt assertions and fixtures.
Tests — Auth packages/backend/test/chatAuth.test.ts	Reworked Turnstile tests to use runtimeConfig.turnstile, added serialTest() and withTurnstileConfig() helpers, and simplified env mutation/restore logic.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Client
    participant Handler as Chat Handler
    participant Orch as Chat Orchestrator
    participant Policy as Tool Policy
    participant Registry as Tool Registry
    participant Adapter as Weather Adapter
    participant NWS as weather.gov API

    Client->>Handler: POST /chat (generation plan)
    Handler->>Orch: invoke orchestrator with plan
    Orch->>Policy: applySingleToolPolicy(generation)
    Policy-->>Orch: (maybe-modified) generation + logEvent?
    Orch->>Registry: resolveToolSelection(generation, weatherForecastTool?)
    Registry-->>Orch: BackendToolSelection (toolIntent, toolRequest, toolExecution?)
    alt weather eligible & tool available
        Orch->>Registry: executeSelectedTool(selection)
        Registry->>Adapter: executeWeatherForecastTool(request)
        Adapter->>NWS: GET /points or /gridpoints/.../forecast
        NWS-->>Adapter: Forecast JSON
        Adapter-->>Registry: toolResultMessage + toolExecutionContext (executed)
        Registry-->>Orch: execution result
    else unavailable or failed
        Registry-->>Orch: toolExecutionContext (skipped/failed), no toolResultMessage
    end
    Orch->>Orch: inject toolResultMessage as system message (if present)
    Orch-->>Handler: planner/generation payload (with tool metadata)
    Handler-->>Client: response + metadata

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• Deterministic policy-ranked tool fallback routing with auditable reason codes (Vibe Kanban) #216 — Overlaps backend tool-selection and orchestration logic; likely touches registry/policy behavior shared here.
• Enhance model profile handling in backend configuration and runtime. #197 — Modifies orchestrator/profile resolution and provider/capabilities plumbing that intersect with the orchestrator changes.
• Normalize execution timeline semantics and contract validation (Vibe Kanban) #217 — Updates execution-reason/tool-invocation normalization surfaces related to the added reason codes and normalization logic.

Poem

🐰 I hopped through endpoints, timeouts, and skies,
I wrapped forecasts in JSON where the backend lies,
Weather first, then search if the plan allows,
I fail open softly, logging reasons and vows,
A rabbit’s small cheer for telemetry and ties.

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(backend): weather.gov pilot via backend tool registry' clearly and specifically describes the main change: introduction of a weather.gov pilot feature integrated through a new backend tool registry infrastructure.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/weathergov-tool-pilot-squashed

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}