Skip to content

[codex] Honor max-connections above HTTP client cap - #3

Open
ywang96 wants to merge 1 commit into
MoonshotAI:mainfrom
ywang96:codex/honor-max-connections
Open

[codex] Honor max-connections above HTTP client cap#3
ywang96 wants to merge 1 commit into
MoonshotAI:mainfrom
ywang96:codex/honor-max-connections

Conversation

@ywang96

@ywang96 ywang96 commented Apr 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • replace the raw httpx.AsyncClient in KimiAPI with OpenAIAsyncHttpxClient
  • preserve the long streaming timeout while removing the hidden active-connection cap from the HTTP pool
  • let inspect_ai's max_connections semaphore remain the effective concurrency control

Root Cause

--max-connections was passed through correctly, but the provider created a raw httpx.AsyncClient with httpx defaults. That client defaults to max_connections=100, so requests above 100 were throttled by the HTTP pool before they could honor the configured inspect-ai concurrency.

Validation

  • uv run python -m py_compile kimi_model.py
  • instantiated _default_http_client() and verified the transport pool no longer caps active connections at 100 while keeping max_keepalive_connections=100

@ywang96

ywang96 commented Apr 18, 2026

Copy link
Copy Markdown
Collaborator Author

Without this change, evaluating with --max-connnections bigger than 100 will slightly fall back to 100 becasue of httpx default max connections.

@ywang96
ywang96 marked this pull request as ready for review April 18, 2026 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant