feat(channels): auto-detect TTS models in channel test button#1826
feat(channels): auto-detect TTS models in channel test button#1826survivor998 wants to merge 2 commits into
Conversation
The channel test button previously sent hardcoded chat completion requests that failed for TTS models (e.g. Xiaomi MiMo-TTS). This fix detects TTS models via their model card's modalities.output field and automatically builds the correct audio request format. Changes: - Add ChatCompletionAudioParam to llm.Request (was previously a TODO) - Add Audio field to openai.Request and carry it in RequestFromLLM - Add isTTSModel detection via model card modalities.output lookup - Add buildTestRequest that constructs TTS-shaped requests for audio models - Update response handling to extract audio transcripts from TTS responses - Update streaming handler to accumulate audio transcripts from delta Verified locally: mimo-v2.5-tts tests successfully on Xiaomi channels. Non-TTS models continue to work as before (backward compatible). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Greptile SummaryThis PR fixes the channel test button for TTS models by auto-detecting audio-output models via their model card and constructing a TTS-specific request. It also adds the
Confidence Score: 3/5The TTS test request is built with assumptions that hold for Xiaomi MiMo TTS but will actively fail against any standard OpenAI-compatible audio provider. The two central issues are in internal/server/orchestrator/tester.go — specifically the Important Files Changed
Reviews (1): Last reviewed commit: "feat(channels): auto-detect TTS models i..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| return &llm.Request{ | ||
| Model: testModel, | ||
| Messages: []llm.Message{ | ||
| { | ||
| Role: "assistant", | ||
| Content: llm.MessageContent{ | ||
| Content: lo.ToPtr("Hello, this is a TTS test. Please respond with audio."), | ||
| }, | ||
| }, | ||
| }, | ||
| Modalities: []string{"text", "audio"}, | ||
| Audio: &llm.ChatCompletionAudioParam{}, | ||
| MaxCompletionTokens: lo.ToPtr(int64(256)), | ||
| Stream: lo.ToPtr(useStream), | ||
| } | ||
| } |
There was a problem hiding this comment.
role: "assistant" instead of "user"
Standard OpenAI-compatible audio/TTS APIs (and most OpenAI-compatible providers) reject conversations whose first (and only) message carries role: "assistant" — the model expects input to come from a user turn and then produce an audio response. Sending an assistant-role message as the sole input will cause these providers to return an error (e.g., "messages[0].role must be 'user' or 'system'"), meaning any TTS channel other than Xiaomi MiMo TTS will always fail the channel test even when correctly configured.
| Modalities: []string{"text", "audio"}, | ||
| Audio: &llm.ChatCompletionAudioParam{}, | ||
| MaxCompletionTokens: lo.ToPtr(int64(256)), | ||
| Stream: lo.ToPtr(useStream), |
There was a problem hiding this comment.
audio param will fail for providers that require voice
&llm.ChatCompletionAudioParam{} serialises to "audio": {} with both voice and format omitted. OpenAI's audio API (and any compatible provider) treats voice as a required field and will return an error when it is absent. This means the TTS channel test would fail for any provider that follows the OpenAI spec, even though the channel is otherwise correctly configured.
Consider defaulting to a reasonable voice (e.g. "alloy") and format (e.g. "mp3") so the test works against standard providers.
|
|
||
| if len(sanitized) < 6 { | ||
| sanitized = "user_" + sanitized | ||
| if len(sanitized) < 6 { | ||
| sanitized = sanitized + strings.Repeat("_", 6-len(sanitized)) | ||
| } |
There was a problem hiding this comment.
"user_" is 5 bytes. Prepending it to any non-empty sanitized string always produces a result of length ≥ 6, so the second if len(sanitized) < 6 check can never be true at runtime (the function already returns early on empty input, and the loop always writes at least one byte per input rune).
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
1 participant
Summary
Audiofield tollm.Request(previously a TODO) andopenai.Request, enabling audio parameters to flow through the request pipeline.TestChannelOrchestratornow checks the model card'smodalities.outputto detect TTS models. For TTS models, it sends a request withrole: assistant,modalities: ["text", "audio"], and an emptyaudioparam (letting the provider use its default voice). For non-TTS models, the original chat completion request is preserved.Changes
llm/model.goChatCompletionAudioParamstruct andAudiofield toRequestllm/transformer/openai/model.goChatCompletionAudioParamstruct andAudiofield toRequestllm/transformer/openai/outbound_convert.goAudiofield inRequestFromLLMinternal/server/orchestrator/tester.gomodelServicedependency,isTTSModel,buildTestRequest,extractMessageTexthelpers; updated stream handling for audio responsesinternal/server/gql/resolver.gomodelServicetoNewTestChannelOrchestratorTest plan
go build ./...)🤖 Generated with Claude Code