Skip to content

feat: rate limiting and retry with exponential backoff for providers #12

@sukethrp

Description

@sukethrp

Summary

Add automatic rate-limit handling and retry logic to the provider layer so agents gracefully handle API throttling from OpenAI, Anthropic, and other LLM providers.

Context

Currently, if the OpenAI or Anthropic API returns a 429 (rate limit) or 5xx (server error), the agent run fails immediately. Production agents need to automatically retry with exponential backoff.

Proposed approach

  1. Add a `RetryPolicy` config (max_retries, base_delay, max_delay, jitter)
  2. Wrap provider calls in a retry decorator that catches rate-limit and transient errors
  3. Log each retry attempt with the delay and attempt number
  4. Respect `Retry-After` headers when present

Acceptance criteria

  • 429 responses trigger automatic retry with exponential backoff
  • 5xx responses trigger retry (up to max_retries)
  • 4xx errors (except 429) fail immediately (no retry)
  • Configurable via `Agent(retry_policy=RetryPolicy(...))`
  • Retry events appear in the agent's event log
  • Works with all three providers (OpenAI, Anthropic, Ollama)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions