Skip to content

Feat/add litellm provider#74

Open
RheagalFire wants to merge 5 commits into
BAI-LAB:mainfrom
RheagalFire:feat/add-litellm-provider
Open

Feat/add litellm provider#74
RheagalFire wants to merge 5 commits into
BAI-LAB:mainfrom
RheagalFire:feat/add-litellm-provider

Conversation

@RheagalFire

Copy link
Copy Markdown

Summary

Adds LiteLLM as an alternative LLM provider for MemoryOS, enabling access to 100+ LLM providers (OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Mistral, Cohere, and more) through a unified interface.

Changes

  • memoryos-pypi/utils.py - New LiteLLMClient class with chat_completion() method, specific exception handling for 7 error types (AuthenticationError, RateLimitError, APIConnectionError, Timeout, NotFoundError, BadRequestError, ContextWindowExceededError), drop_params=True for cross-provider compatibility
  • memoryos-pypi/memoryos.py - Provider dispatch: provider="litellm" routes to LiteLLMClient instead of OpenAIClient
  • memoryos-pypi/__init__.py - Export LiteLLMClient
  • memoryos-pypi/requirements.txt - Added litellm>=1.80.0
  • memoryos-pypi/tests/test_litellm.py - 12 comprehensive unit tests
  • pytest.ini - Test configuration

Tests

Unit tests (12/12 passing):

test_returns_content PASSED
test_authentication_error PASSED
test_not_found_error PASSED
test_rate_limit_error PASSED
test_context_window_exceeded PASSED
test_none_content_returns_error PASSED
test_timeout_error PASSED
test_connection_error PASSED
test_bad_request_error PASSED
test_drop_params_true PASSED
test_api_key_forwarded PASSED
test_api_key_not_in_kwargs_when_none PASSED
12 passed

Risk / Compatibility

  • Additive only. Existing OpenAIClient path completely untouched.
  • litellm is an optional dependency - existing installs unaffected.
  • Provider selection is explicit via provider="litellm" parameter.

Example usage

from memoryos import Memoryos

# Use LiteLLM with Anthropic Claude
memory = Memoryos(
    user_id="user_123",
    assistant_id="assistant_456",
    llm_model="anthropic/claude-sonnet-4-20250514",
    provider="litellm",
    api_key="sk-your-anthropic-key",
    storage_path="./memory_data"
)

# Use LiteLLM with AWS Bedrock
memory = Memoryos(
    user_id="user_123",
    assistant_id="assistant_456",
    llm_model="bedrock/anthropic.claude-3-haiku-20240307-v1:0",
    provider="litellm",
    storage_path="./memory_data"
    # Uses AWS credentials from environment (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
)

# Use LiteLLM with Google Gemini
memory = Memoryos(
    user_id="user_123",
    assistant_id="assistant_456",
    llm_model="gemini/gemini-2.5-flash",
    provider="litellm",
    api_key="your-google-api-key",
    storage_path="./memory_data"
)

# Then use normally - all memory operations work the same regardless of provider
memory.add_interaction(
    user_input="I prefer dark mode and use vim",
    assistant_response="Got it! I'll remember your preferences."
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant