Skip to content

fix: enable handling video_url in litellm and chat completions models#2614

Merged
seratch merged 7 commits intoopenai:mainfrom
seokhyunan:fix/chatcmpl-video-url-support
Mar 7, 2026
Merged

fix: enable handling video_url in litellm and chat completions models#2614
seratch merged 7 commits intoopenai:mainfrom
seokhyunan:fix/chatcmpl-video-url-support

Conversation

@seokhyunan
Copy link
Contributor

Summary

  • add video_url support to the chat-completions converter in openai/openai-agents-python
  • allow OpenRouter-compatible video inputs to pass through the standard Agents SDK message conversion path

Problem

The SDK currently handles text, image, audio, and file content parts in the chat-completions converter, but it rejects OpenRouter-style video inputs shaped like:

{
    "type": "video_url",
    "video_url": {
        "url": data_url,
    },
}

Change

Updated src/agents/models/chatcmpl_converter.py so that:

  • raw dict multimodal content items can pass through the converter
  • Converter.extract_all_content() recognizes video_url
  • the converter forwards video_url content instead of rejecting it

Verification

  • uv run python -c "from agents.models.chatcmpl_converter import Converter; print(Converter.extract_all_content([{'type': 'input_text', 'text': 'x'}, {'type': 'video_url', 'video_url': {'url': 'data:video/mp4;base64,abc'}}]))"

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a9dee6b7f6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@seratch seratch marked this pull request as draft March 6, 2026 16:03
@seokhyunan seokhyunan marked this pull request as ready for review March 7, 2026 03:18
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fbc09d1a86

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@seratch seratch marked this pull request as draft March 7, 2026 03:30
@seokhyunan seokhyunan marked this pull request as ready for review March 7, 2026 03:44
Copy link
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took another pass on the latest head (4fef52e4) and re-checked it with a live runtime verification.

The previous tool-output regression looks fixed in this revision, but I still see one remaining blocker:

  • src/agents/models/chatcmpl_converter.py now accepts video_url in the user-message path, and OpenAIChatCompletionsModel always sends that converted payload through the default Chat Completions path (src/agents/models/openai_chatcompletions.py). On official OpenAI, that still does not work.
  • I verified this through the SDK/runtime path, not just by reading the converter: a normal image input succeeds, but a video_url input returns a backend 400 invalid_value from Chat Completions. The error says the supported content-part types are text, image_url, input_audio, refusal, audio, and file.
  • That means this PR currently broadens the default official-OpenAI behavior and turns what used to be a local validation failure into a backend failure.

Given that, I do not think this is ready as an unconditional change for the default OpenAI Chat Completions path. If the goal is support for non-OpenAI OpenAI-compatible providers, I think this needs provider/base-URL gating so the official OpenAI path still rejects video_url locally. Also, if LiteLLM adapter layer (or LiteLLM itself) is the right place to enhance for this need, it'd be a more natural way.

@seratch seratch marked this pull request as draft March 7, 2026 03:57
@seratch seratch added the needs-more-info Waiting for a reply/more info from the author label Mar 7, 2026
@seokhyunan
Copy link
Contributor Author

@codex review again

@chatgpt-codex-connector
Copy link

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@seratch seratch changed the title support video_url in chat completions converter fix: enable handling video_url in litellm and chat completions models Mar 7, 2026
@seratch seratch marked this pull request as ready for review March 7, 2026 05:49
@seratch seratch added this to the 0.11.x milestone Mar 7, 2026
Copy link
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. Now I see the value for developers who want to use video input supported models on openrouter. Having some unit tests would be ideal, but it's okay to skip them this time

@seratch seratch added feature:lite-llm and removed needs-more-info Waiting for a reply/more info from the author labels Mar 7, 2026
@seratch seratch merged commit d23d85a into openai:main Mar 7, 2026
10 checks passed
@seokhyunan seokhyunan deleted the fix/chatcmpl-video-url-support branch March 7, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants