-
Notifications
You must be signed in to change notification settings - Fork 0
feat: rate limiting and retry with exponential backoff for providers #12
Copy link
Copy link
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Summary
Add automatic rate-limit handling and retry logic to the provider layer so agents gracefully handle API throttling from OpenAI, Anthropic, and other LLM providers.
Context
Currently, if the OpenAI or Anthropic API returns a 429 (rate limit) or 5xx (server error), the agent run fails immediately. Production agents need to automatically retry with exponential backoff.
Proposed approach
- Add a `RetryPolicy` config (max_retries, base_delay, max_delay, jitter)
- Wrap provider calls in a retry decorator that catches rate-limit and transient errors
- Log each retry attempt with the delay and attempt number
- Respect `Retry-After` headers when present
Acceptance criteria
- 429 responses trigger automatic retry with exponential backoff
- 5xx responses trigger retry (up to max_retries)
- 4xx errors (except 429) fail immediately (no retry)
- Configurable via `Agent(retry_policy=RetryPolicy(...))`
- Retry events appear in the agent's event log
- Works with all three providers (OpenAI, Anthropic, Ollama)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request