Skip to content

fix: Preserve tool calls for LLMRails tool rails#2073

Draft
JashG wants to merge 2 commits into
NVIDIA-NeMo:developfrom
JashG:fix/llmrails-tool-call-rails
Draft

fix: Preserve tool calls for LLMRails tool rails#2073
JashG wants to merge 2 commits into
NVIDIA-NeMo:developfrom
JashG:fix/llmrails-tool-call-rails

Conversation

@JashG

@JashG JashG commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Description

This PR enables tool call guardrails to work when checking a tool-call message directly (as opposed to when the library itself generates the tool call).

Desired use case:
Supporting tool call guardrails from agentic workflows where we don't rely on generate_async to orchestrate the LLMCalls end-to-end. Example flow:

  • An assistant response generates tool_calls
  • Before executing the tools, use the library to inspect the tool call name/args
  • After the agent executes the tools out-of-band, use the library to inspect the results

Current issue:
In LLMRails, assistant messages were treated as plain text output and removed from the message history. If that assistant message contained tool_calls, the tool calls never became BotToolCalls events, so tool output rails could be skipped.

This change preserves assistant tool_calls long enough for the existing tool-rail pipeline to see them, and updates logging so tool input/output rails appear correctly in the generation log.

Related Issue(s)

Verification

AI Assistance

  • No AI tools were used.
  • AI tools were used; a human reviewed and can explain every change (tool: GPT 5.5 via Cursor).

Checklist

  • I've read the CONTRIBUTING guidelines.
  • This PR links to a triaged issue assigned to me.
  • My PR title follows the project commit convention.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • I've noted any verification beyond CI and any checks I couldn't run.
  • I did not update generated changelog files manually.
  • I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
  • @mentions of the person or team responsible for reviewing proposed changes.

@github-actions github-actions Bot added status: needs triage New issues that have not yet been reviewed or categorized. size: S labels Jun 25, 2026
@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions github-actions Bot added size: M and removed size: S labels Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: M status: needs triage New issues that have not yet been reviewed or categorized.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant