-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Description
Summary
This PR introduces asyncio support to aisuite, enabling developers to use the library in modern asynchronous Python applications (FastAPI, async agents, event-driven LLM workflows, chat servers, etc.).
This feature was frequently requested across community discussions and greatly improves the usability of AISuite in production-grade AI systems.
Motivation
AISuite currently exposes synchronous methods only.
However, most real-world AI apps — including chat APIs, RAG servers, and multi-tool agents — run on async frameworks where blocking calls limit concurrency and performance.
Adding async support:
- Enables parallel LLM calls without blocking the event loop
- Makes AISuite compatible with FastAPI, async workers, and async agents
- Improves throughput for multi-provider or multi-step pipelines
- Aligns AISuite with modern LLM SDKs which already expose async methods
This PR adds async support without breaking any existing synchronous APIs.
What This PR Adds
✔️ AsyncClient wrapper class
- Provides async equivalents of existing sync methods
- Uses
asyncio.to_thread()to maintain compatibility with all current provider implementations
✔️ async generate() and async chat() methods
Example:
from aisuite import AsyncClient
client = AsyncClient("openai:gpt-4o")
async def main():
response = await client.generate("Write a haiku about async LLMs.")
print(response.text)✔️ Backward-compatible design
- No changes required for existing users
- Sync and async clients can coexist
- Providers do not need to be modified
✔️ Basic async tests
- Confirms event-loop behavior
- Ensures output matches synchronous results
Implementation Details
- Introduced
AsyncClientclass inaisuite/async_client.py - Forwarded all calls to async wrappers around existing provider implementations
- Added minimal async examples in docs
- Added lightweight async unit tests under
tests/async/
This design keeps AISuite simple while giving developers full async capabilities.
Checklist
- Async client implemented
- Tests added
- Documentation updated
- Backward compatible
- Linted and formatted
Thank you!