Add support for Cerebras to litellm_caller #110
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds support for Cerebras AI as a new LLM provider to the application's LiteLLM integration. Cerebras is integrated following the same pattern as existing providers (OpenAI, Anthropic, Google), with changes to the model configuration, provider detection logic, API key handling, and environment variable documentation.
Key Changes:
- Added Cerebras provider detection and model name mapping to LiteLLMCaller
- Configured a new Cerebras model in the LLM configuration
- Added CEREBRAS_API_KEY to environment variable examples
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| config/overrides/llmconfig.yml | Adds cerebras-gpt-oss-120b model configuration with Cerebras API endpoint and credentials |
| backend/modules/llm/litellm_caller.py | Implements Cerebras provider support with URL detection, model name mapping, API key handling, and api_base exclusion logic |
| .env.example | Adds CEREBRAS_API_KEY environment variable placeholder for developer reference |
| compliance_level: "External" | ||
| compliance_level: "External" | ||
| cerebras-gpt-oss-120b: | ||
| model_url: "https://api.cerebras.ai/v1" |
There was a problem hiding this comment.
The model_url for Cerebras appears incomplete compared to other providers. OpenAI uses the full endpoint path https://api.openai.com/v1/chat/completions, but Cerebras only specifies https://api.cerebras.ai/v1.
If LiteLLM's Cerebras provider requires the full endpoint path (like /v1/chat/completions), this configuration may not work correctly. Please verify that https://api.cerebras.ai/v1 is the correct format for LiteLLM's Cerebras integration, or update it to include the complete endpoint path if needed.
| model_url: "https://api.cerebras.ai/v1" | |
| model_url: "https://api.cerebras.ai/v1/chat/completions" |
| compliance_level: "External" | ||
| cerebras-gpt-oss-120b: | ||
| model_url: "https://api.cerebras.ai/v1" | ||
| model_name: "gpt-oss-120b" |
There was a problem hiding this comment.
The model name gpt-oss-120b appears to be incorrect for a Cerebras model. This same model name is used for OpenRouter configurations (see line 14 in this file), which suggests it might be a copy-paste error.
Cerebras models typically have names like llama3.1-8b or llama3.1-70b. Please verify and update the model name to match an actual Cerebras model identifier that will be recognized by LiteLLM's Cerebras provider.
| model_name: "gpt-oss-120b" | |
| model_name: "llama3.1-70b" |
2 participants
No description provided.