[Bugfix] Fix getRecommendedModel to return real model names from config #933
+223
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…odel Previously, getRecommendedModel() returned hardcoded invalid model names like 'math-specialized-model' instead of actual model names from the configuration. This caused incorrect model recommendations. Changes: - Updated getRecommendedModel() to read model names from config - Added support for LoRA adapter names (prefer LoRA over base model) - Added case-insensitive category matching - Added fallback to default model when no decision found - Added comprehensive tests for all scenarios Fixes: vllm-project#929 Tested: - All existing tests pass - New tests cover config fallback, LoRA support, and edge cases - golangci-lint: 0 issues - go vet: no errors Signed-off-by: Hen Schwartz <[email protected]>
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Previously,
getRecommendedModel()returned hardcoded invalid model names like'math-specialized-model'instead of actual model names from the configuration. This caused incorrect model recommendations when the router tried to select models based on classified categories.Solution
Updated
getRecommendedModel()to:IntelligentRouting.DecisionsChanges
Updated
getRecommendedModel()method inclassification.go:fmt.Sprintf("%s-specialized-model", category)logicModelRef.LoRANamefirst)strings.EqualFold()Added comprehensive tests in
classification_test.go:TestGetRecommendedModel_WithConfig- Tests real model names from configTestGetRecommendedModel_NoConfig- Tests nil config fallbackTestGetRecommendedModel_EmptyConfig- Tests empty config fallbackTestGetRecommendedModel_NoDecisionFound- Tests default model fallbackTestGetRecommendedModel_EmptyModelRefs- Tests empty ModelRefs handlingTesting
All existing tests pass
New tests cover all scenarios (config fallback, LoRA support, edge cases)
golangci-lint: 0 issuesgo vet: no errorsgo test: all tests pass (including 5 new test functions)Pre-commit hooks: passed
Fixes #929