feat: emulate OpenAI tool calling for gemini-web via the webTools prompt shim (#7286)#7727
Merged
Merged
Conversation
… shim (#7286) Level 2 of the staged approach in #7286: wire the existing webTools.ts prompt-emulation shim (already proven across 11 other web-cookie executors) into gemini-web.ts. The client's tools[] array is now serialized into the prompt typed into the Gemini web UI, and <tool>{...}</tool> blocks in the response are parsed back into OpenAI tool_calls -- including for streaming requests, replayed as a single terminal SSE chunk since gemini-web buffers the whole response by construction. Malformed tool JSON degrades to ordinary chat content, never an error, matching the existing behavior of the other 11 executors. The no-tools code path is unchanged (regression guard). Also Level 1: adds a "Tool calling" column (native/emulated/none) to docs/reference/PROVIDER_REFERENCE.md for providers with confirmed ground truth (the 11 already-wired web-cookie executors + gemini-web -> emulated, claude-web -> none pending its own Level 3 decision). Level 3 (claude-web) and Level 4 (supportsTools capability flag) are explicitly out of scope -- claude-web/payload.ts is untouched.
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
4 tasks
Resolved the docs/reference/PROVIDER_REFERENCE.md conflict by regenerating it from the merged registry (npm run gen:provider-reference) rather than hand- resolving the generated file: 268 unique provider IDs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ships Levels 1-2 only from #7286's staged approach:
webTools.tsprompt-emulation shim (already proven across 11 other web-cookie executors:adapta-web,blackbox-web,chatgpt-web,deepseek-web,duckduckgo-web,gitlab,inner-ai,muse-spark-web,perplexity-web,qwen-web,t3-chat-web) intoopen-sse/executors/gemini-web.ts:prepareToolMessages()serializes the client'stools[]into a<tool>contract, prepended as a synthetic system message.gemini-webconcatenates it with the last user message into the single prompt string typed into the Gemini web UI (buildGeminiToolPrompt).buildGeminiToolResponse()wraps it in the standard OpenAI completion shape and delegates to the sharedbuildToolModeResponse()(chatgptWebTools.ts), which parses<tool>{...}</tool>blocks intotool_calls. Malformed JSON degrades to ordinary chat content — never a thrown error or a 500, matching the other 11 executors' behavior.gemini-webbuffers the whole response before returning (no true token-by-token streaming), tool mode replays either buffered JSON or a single terminal SSE chunk (rolechunk +delta.tool_calls/finish_reason: "tool_calls"chunk +[DONE]).messages.filter(m => m.role === "user").pop()prompt derivation andformatChatCompletion/formatStreamChunkcall sites are untouched whentoolsis absent.native/emulated/none) todocs/reference/PROVIDER_REFERENCE.md, populated only for entries with confirmed ground truth in this pass: the 11 already-wired web-cookie executors +gemini-web→emulated,claude-web→none(pending its own Level 3 decision). The column is regenerated vianpm run gen:provider-reference—PROVIDER_REFERENCE.mdis never hand-edited.Explicitly out of scope (per the plan and issue's staged approach):
claude-web) — needs the owner's emulate-vs-native decision;claude-web.ts/claude-web/payload.tsare untouched in this diff.supportsToolscapability flag in the provider registry) — cross-cutting, tracked separately.Validation (Hard Rule #18 — TDD)
New test file
tests/unit/gemini-web-tool-calling-7286.test.ts(8 tests, all passing):tool_calls,content: null,finish_reason: "tool_calls"(non-streaming).<tool>JSON degrades to ordinary chat content — never throws, never a 500.delta.tool_calls, then[DONE].<tool>block + no requested tools → content passthrough.buildGeminiToolPromptunit tests (with/without a synthetic system message).gemini-web.test.tspattern) proving the wiring actually reaches the Playwright-driven request/response code path:tools[]present →tool_calls;toolsabsent → the typed prompt is byte-identical to the old last-user-message-only derivation and the response shape is unchanged (regression guard).Red→green proof: confirmed the pre-change
open-sse/executors/gemini-web.ts(checked out fromorigin/release/v3.8.49into a scratch file) exports neitherbuildGeminiToolResponsenorbuildGeminiToolPrompt— the new test file's exercised surface does not exist on the base branch. After the change, all 8 new tests plus the 17 pre-existinggemini-webtests pass (25/25).Gates run (all green)
node scripts/check/check-test-discovery.mjsnpm run typecheck:core/npm run typecheck:noimplicit:core(no new errors; pre-existing drift in unrelated files unchanged from base)npx eslint --suppressions-location config/quality/eslint-suppressions.json <changed files>— 0 errorsnpm run check:complexity-ratchets— 2059/890, unchanged from baselinenode scripts/check/check-file-size.mjs— clean, no✗npm run check:cycles— no new cyclesnpm run check:provider-consistency— OKnpm run check:docs-sync— PASSnpx tsc --noEmit -p tsconfig.jsonspot-checked: zero errors touch any file in this diff (confirmed the remaining pre-existing errors are on files untouched by this PR viagit diff --statagainst the base ref)Not run:
npm run test:coverage(heaviest gate, shared devbox) — this change is additive-only (new test file + new exported helpers), CI runs it authoritatively.What was deferred
claude-web) — needs the owner's decision between (a) prompt-emulation viawebTools.ts(additive, same pattern as this PR) or (b) merging client tools into the Claude.ai payload + parsingtool_useblocks (closer to native, needs Hard-Rule-fix(ci): add environment for npm token access #18 VPS validation against the live undocumented upstream contract). Tracked as a follow-up once that's settled.supportsToolscapability flag) — cross-cutting change touching the whole provider registry, tracked separately.This PR intentionally does not close #7286 — the issue covers both Level 2 (gemini-web, shipped here) and Level 3 (claude-web, deferred). Refs #7286.