Enhance VS Code Chat Provider Bridge to Fix Create-File (and other) Tool Calls#190
Open
danielrobbins wants to merge 2 commits into
Open
Enhance VS Code Chat Provider Bridge to Fix Create-File (and other) Tool Calls#190danielrobbins wants to merge 2 commits into
danielrobbins wants to merge 2 commits into
Conversation
Before this change, llama-vscode flattened VS Code chat messages down to plain text when building OpenAI-compatible chat requests. That discarded structured tool metadata from earlier turns, including prior tool calls and tool results. In practice, this made tool-use sessions much less reliable. The model could lose track of which tools had already been called, what arguments had been passed, and what results had been returned. One visible failure mode was tool-call hallucination: the model could claim that an action had been taken, particularly new file creation, without emitting the corresponding tool call and receiving a tool result first. This change preserves VS Code tool metadata when translating chat history into OpenAI-compatible messages. Assistant turns now carry tool_calls, tool results are emitted as tool messages with tool_call_id, and tool result payloads are normalized into stable string content instead of being silently dropped. The request bridge now also forwards VS Code tool-mode intent as tool_choice and adds a stronger tool-calling instruction when tools are available. That tuning is defensive rather than foundational, but it helps reduce cases where the model responds with prose about tool usage instead of actually calling the tool. Together, these changes make the VS Code native chat/tooling path much more robust and keep the model grounded in the actual tool state instead of a flattened text-only approximation of the conversation. This branch still requires the separate chat-provider shim fix to build cleanly against current @types/vscode. That compile cleanup is handled independently so the behavioral serialization change stays reviewable on its own.
conflicts with current @types/vscode declarations. The original shim was added when llama-vscode needed to supply its own temporary LM chat-provider typings. That is no longer necessary because current @types/vscode releases already include the relevant language model chat APIs. Leaving the old shim in place causes duplicate declaration errors during TypeScript compilation. Properties such as modelOptions, tools, maxInputTokens, maxOutputTokens, and LanguageModelResponsePart are now defined both by @types/vscode and by the local shim, which breaks the build even though the runtime code itself is otherwise valid. This change removes the dead shim file entirely and keeps one small provider typing cleanup in llama-chat-model-provider.ts so the current tool metadata mapping aligns with the upstream VS Code types. After this cleanup, the repo compiles cleanly against the installed @types/vscode package without relying on duplicate local declarations. This cleanup is also required for the separate VS Code tool serialization fix to build cleanly. Without removing the obsolete shim, that branch still fails compilation on current @types/vscode before its behavioral changes can be validated in a normal TypeScript build.
danielrobbins
changed the title
danielrobbins
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes a core serialization bug in the VS Code chat provider bridge.
Before this change, llama-vscode flattened VS Code chat history into plain
text when building OpenAI-compatible chat requests. That discarded
structured tool metadata from earlier turns, including prior tool calls
and tool results.
In practice, this made tool-use sessions much less reliable. The model
could lose track of which tools had already been called, what arguments
had been passed, and what results had been returned. One visible failure
mode was tool-call hallucination: the model could claim that an action
had been taken, particularly file creation, without emitting the
corresponding tool call and receiving a tool result first.
What changed
into OpenAI-compatible messages
tool_callsinstead of flattening prior tool calls intoplain text
role: "tool"messages withtool_call_idtool_choiceWhy this is needed
The previous bridge lost tool state between turns. That left the model
working from a text-only approximation of the conversation instead of the
actual tool-call history. As a result, the model could respond as if a
tool action had already happened without ever issuing the tool call.
This PR keeps the model grounded in the actual VS Code tool state and
makes the native chat/tooling path much more robust.
Scope
This PR is focused on the VS Code native chat provider bridge in
src/llama-chat-model-provider.ts.Note: this branch currently also includes the prerequisite shim cleanup
from #187 so it compiles cleanly against current VS Code typings. That
overlap is not the main behavior change to review here; the main review
surface is the tool metadata preservation and request/response
serialization logic in the chat provider bridge.
Validation
when building OpenAI-compatible messages (specifically resolving a file creation bug I repeatedly hit when using gemma4, and that other tool calls are still working.)
tool_choice