Skip to content

Add Asyncio Support to AISuite #255

@janvis11

Description

@janvis11

Summary

This PR introduces asyncio support to aisuite, enabling developers to use the library in modern asynchronous Python applications (FastAPI, async agents, event-driven LLM workflows, chat servers, etc.).

This feature was frequently requested across community discussions and greatly improves the usability of AISuite in production-grade AI systems.


Motivation

AISuite currently exposes synchronous methods only.
However, most real-world AI apps — including chat APIs, RAG servers, and multi-tool agents — run on async frameworks where blocking calls limit concurrency and performance.

Adding async support:

  • Enables parallel LLM calls without blocking the event loop
  • Makes AISuite compatible with FastAPI, async workers, and async agents
  • Improves throughput for multi-provider or multi-step pipelines
  • Aligns AISuite with modern LLM SDKs which already expose async methods

This PR adds async support without breaking any existing synchronous APIs.


What This PR Adds

✔️ AsyncClient wrapper class

  • Provides async equivalents of existing sync methods
  • Uses asyncio.to_thread() to maintain compatibility with all current provider implementations

✔️ async generate() and async chat() methods

Example:

from aisuite import AsyncClient

client = AsyncClient("openai:gpt-4o")

async def main():
    response = await client.generate("Write a haiku about async LLMs.")
    print(response.text)

✔️ Backward-compatible design

  • No changes required for existing users
  • Sync and async clients can coexist
  • Providers do not need to be modified

✔️ Basic async tests

  • Confirms event-loop behavior
  • Ensures output matches synchronous results

Implementation Details

  • Introduced AsyncClient class in aisuite/async_client.py
  • Forwarded all calls to async wrappers around existing provider implementations
  • Added minimal async examples in docs
  • Added lightweight async unit tests under tests/async/

This design keeps AISuite simple while giving developers full async capabilities.


Checklist

  • Async client implemented
  • Tests added
  • Documentation updated
  • Backward compatible
  • Linted and formatted

Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions