feat: 完善模型能力标注 #1474
Merged
feat: 完善模型能力标注 #1474
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Introduced a new ModelConfig class to encapsulate model configuration details, including ID, type, and ability. - Updated LLMBackendConfig to use a list of ModelConfig objects for supported models, allowing for richer model information. - Implemented model validation and migration logic to convert legacy model formats to the new ModelConfig structure. - Enhanced auto-detection methods in various LLM adapters to return ModelConfig instances, improving model handling and integration. - Updated related methods and tests to ensure compatibility with the new model configuration structure.
- Included the LLMAbility import in llm_registry.py to ensure proper functionality of LLM backend registration. - Updated the LLMBackendRegistry class to accept additional arguments in the registration method for enhanced flexibility.
Contributor
按能力查询模型的序列图sequenceDiagram
participant Requester as Requesting Code
participant LLMManager
participant ModelInfo as model_info Cache
Requester->>LLMManager: get_supported_models(type, ability)
activate LLMManager
LLMManager->>ModelInfo: Iterate over cached ModelConfig
activate ModelInfo
loop For each ModelConfig
ModelInfo->>ModelInfo: Check if config.type == type
ModelInfo->>ModelInfo: Check if ability.is_capable(config.ability)
alt Matches type and ability
ModelInfo->>LLMManager: Add model_id to result list
end
end
ModelInfo-->>LLMManager:
deactivate ModelInfo
LLMManager-->>Requester: Return List~str~ (matching model IDs)
deactivate LLMManager
模型能力重构的类图classDiagram
direction LR
class ModelType {
<<Enumeration>>
LLM
Embedding
ImageGeneration
Audio
+from_str(str) ModelType
}
class ModelAbility {
<<Interface>>
+is_capable(int ability) bool
}
class LLMAbility {
<<Enumeration>>
+Unknown: 0
+Chat: 1 << 1
+TextInput: 1 << 2
+TextOutput: 1 << 3
+ImageInput: 1 << 4
+ImageOutput: 1 << 5
+AudioInput: 1 << 6
+AudioOutput: 1 << 7
+FunctionCalling: 1 << 8
+TextChat
+is_capable(int ability) bool
}
ModelAbility <|.. LLMAbility
note for LLMAbility "Other ability enums (Embedding, Image, Audio) follow similar pattern"
class ModelConfig {
+id: str
+type: str
+ability: int
}
class LLMBackendConfig {
+adapter: str
+config: Dict
+enable: bool
+models: List~ModelConfig~
+migrate_models_format() validator
}
LLMBackendConfig o-- "*" ModelConfig : contains
note for LLMBackendConfig "models field changed from List<str> to List<ModelConfig>.
Migration validator added."
class LLMManager {
-model_info: Dict~str, ModelConfig~
+load_backend(str)
+unload_backend(str)
+get_supported_models(ModelType, ModelAbility) List~str~
+get_models_by_type(ModelType) List~str~
+get_models_by_ability(ModelType, ModelAbility) str
+get_llm_id_by_ability(ModelAbility) str
}
LLMManager o-- "*" ModelConfig : stores
class LLMBackendRegistry {
- _adapters: Dict
- _configs: Dict
- _ability_registry: Dict # REMOVED
+register(str, Type, Type) # REMOVED ability param
- get_adapter_by_ability() # REMOVED
- search_adapter_by_ability() # REMOVED
}
note for LLMBackendRegistry "Ability registration and lookup removed."
class LLMBackendAdapter {
<<Abstract>>
+config: BaseModel
+auto_detect_models() List~ModelConfig~
}
note for LLMBackendAdapter "auto_detect_models now returns List<ModelConfig>"
class OpenAIAdapter {
+config: OpenAIConfig
+auto_detect_models() List~ModelConfig~
+get_models() list~str~
}
LLMBackendAdapter <|-- OpenAIAdapter
class Utils {
<<Module>>
+guess_openai_model(str) Tuple
+guess_qwen_model(str) Tuple
}
OpenAIAdapter ..> Utils : uses
AlibabaCloudAdapter ..> Utils : uses
文件级别变更
提示和命令与 Sourcery 互动
自定义您的体验访问您的 仪表板 以：
获得帮助Original review guide in EnglishReviewer's GuideThis pull request refactors the model capability representation by introducing Sequence Diagram for Model Auto-Detection and LoadingsequenceDiagram
participant Client
participant Adapter as LLMBackendAdapter
participant ExtAPI as External Model API
participant Utils
participant LLMManager
Client->>Adapter: auto_detect_models()
activate Adapter
Adapter->>ExtAPI: Request model list (e.g., GET /models)
activate ExtAPI
ExtAPI-->>Adapter: Raw model data
deactivate ExtAPI
loop For each raw model
Adapter->>Utils: guess_..._model(model_id)
activate Utils
Utils-->>Adapter: (ModelType, ability_bitmask)
deactivate Utils
Adapter->>Adapter: Create ModelConfig(id, type, ability)
end
Adapter-->>Client: List~ModelConfig~
deactivate Adapter
Client->>LLMManager: load_backend(backend_config)
activate LLMManager
Note right of LLMManager: Backend config contains List<ModelConfig>
LLMManager->>LLMManager: Store models in model_info cache
deactivate LLMManager
Sequence Diagram for Querying Models by CapabilitysequenceDiagram
participant Requester as Requesting Code
participant LLMManager
participant ModelInfo as model_info Cache
Requester->>LLMManager: get_supported_models(type, ability)
activate LLMManager
LLMManager->>ModelInfo: Iterate over cached ModelConfig
activate ModelInfo
loop For each ModelConfig
ModelInfo->>ModelInfo: Check if config.type == type
ModelInfo->>ModelInfo: Check if ability.is_capable(config.ability)
alt Matches type and ability
ModelInfo->>LLMManager: Add model_id to result list
end
end
ModelInfo-->>LLMManager:
deactivate ModelInfo
LLMManager-->>Requester: Return List~str~ (matching model IDs)
deactivate LLMManager
Class Diagram for Model Capability RefactoringclassDiagram
direction LR
class ModelType {
<<Enumeration>>
LLM
Embedding
ImageGeneration
Audio
+from_str(str) ModelType
}
class ModelAbility {
<<Interface>>
+is_capable(int ability) bool
}
class LLMAbility {
<<Enumeration>>
+Unknown: 0
+Chat: 1 << 1
+TextInput: 1 << 2
+TextOutput: 1 << 3
+ImageInput: 1 << 4
+ImageOutput: 1 << 5
+AudioInput: 1 << 6
+AudioOutput: 1 << 7
+FunctionCalling: 1 << 8
+TextChat
+is_capable(int ability) bool
}
ModelAbility <|.. LLMAbility
note for LLMAbility "Other ability enums (Embedding, Image, Audio) follow similar pattern"
class ModelConfig {
+id: str
+type: str
+ability: int
}
class LLMBackendConfig {
+adapter: str
+config: Dict
+enable: bool
+models: List~ModelConfig~
+migrate_models_format() validator
}
LLMBackendConfig o-- "*" ModelConfig : contains
note for LLMBackendConfig "models field changed from List<str> to List<ModelConfig>.
Migration validator added."
class LLMManager {
-model_info: Dict~str, ModelConfig~
+load_backend(str)
+unload_backend(str)
+get_supported_models(ModelType, ModelAbility) List~str~
+get_models_by_type(ModelType) List~str~
+get_models_by_ability(ModelType, ModelAbility) str
}
LLMManager o-- "*" ModelConfig : stores
class LLMBackendRegistry {
- _adapters: Dict
- _configs: Dict
- _ability_registry: Dict # REMOVED
+register(str, Type, Type) # REMOVED ability param
- get_adapter_by_ability() # REMOVED
- search_adapter_by_ability() # REMOVED
}
note for LLMBackendRegistry "Ability registration and lookup removed."
class LLMBackendAdapter {
<<Abstract>>
+config: BaseModel
+auto_detect_models() List~ModelConfig~
}
note for LLMBackendAdapter "auto_detect_models now returns List<ModelConfig>"
class OpenAIAdapter {
+config: OpenAIConfig
+auto_detect_models() List~ModelConfig~
+get_models() list~str~
}
LLMBackendAdapter <|-- OpenAIAdapter
class Utils {
<<Module>>
+guess_openai_model(str) Tuple
+guess_qwen_model(str) Tuple
}
OpenAIAdapter ..> Utils : uses
AlibabaCloudAdapter ..> Utils : uses
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
MyPy 类型检查通过 ✅PR 修改的代码行通过了类型检查。 |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
嘿 @lss233 - 我已经审查了你的更改，发现了一些需要解决的问题。
阻塞性问题:
-
OpenAI 适配器配置包含 API 密钥。(link)
-
基于模型 ID 中的字符串匹配的模型能力猜测逻辑（
guess_openai_model，guess_qwen_model）可能很脆弱；考虑直接从提供商 API 获取能力，或者更多地依赖显式配置。 -
auto_detect_models中用于确定模型能力的方法在不同适配器之间差异很大；尽可能争取更一致的方法。
以下是我在审查期间查看的内容
- 🟡 一般问题：发现 1 个问题
- 🔴 安全：1 个阻塞性问题
- 🟡 测试：发现 1 个问题
- 🟡 复杂性：发现 1 个问题
- 🟢 文档：一切看起来都很好
帮助我更有用！请点击每个评论上的 👍 或 👎，我将使用反馈来改进你的审查。
Original comment in English
Hey @lss233 - I've reviewed your changes and found some issues that need to be addressed.
Blocking issues:
-
The OpenAI adapter configuration contains an API key. (link)
-
The model capability guessing logic based on string matching in model IDs (
guess_openai_model,guess_qwen_model) could be brittle; consider fetching capabilities directly from provider APIs where available or relying more on explicit configuration. -
The methods for determining model capabilities in
auto_detect_modelsvary significantly across adapters; strive for a more consistent approach where possible.
Here's what I looked at during the review
- 🟡 General issues: 1 issue found
- 🔴 Security: 1 blocking issue
- 🟡 Testing: 1 issue found
- 🟡 Complexity: 1 issue found
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| } | ||
| ] | ||
|
|
||
| def test_guess_openai_model(): |
Contributor
There was a problem hiding this comment.
issue (testing): 缺少 guess_qwen_model 函数的测试。
添加一个 test_guess_qwen_model 套件，镜像 test_guess_openai_model，以验证各种 Qwen 模型 ID 和预期类型/能力。
Original comment in English
issue (testing): Missing tests for guess_qwen_model function.
Add a test_guess_qwen_model suite mirroring test_guess_openai_model to validate various Qwen model IDs and expected types/abilities.
| return None | ||
| return None | ||
|
|
||
| def guess_openai_model(model_id: str) -> Tuple[ModelType, int] | None: |
Contributor
There was a problem hiding this comment.
issue (complexity): 考虑将 guess_openai_model 和 guess_qwen_model 中嵌套的 if-else 逻辑重构为声明式规则列表，以提高可读性和可维护性。
考虑将分支逻辑提取到声明式规则列表中，而不是深度嵌套的 if-else 块。这可以降低复杂性，并使每个案例更容易检查和修改。例如，您可以像这样重构 guess_openai_model：
def guess_openai_model(model_id: str) -> Tuple[ModelType, int] | None:
model_id = model_id.lower()
def embedding_rule(m: str):
return "embedding" in m, (ModelType.Embedding, EmbeddingModelAbility.TextEmbedding.value | EmbeddingModelAbility.Batch.value)
def image_rule(m: str):
if "dall-e" in m or "gpt-image" in m:
ability = ImageModelAbility.TextToImage.value
if "dall-e-2" in m or "gpt-image" in m:
ability |= ImageModelAbility.ImageEdit.value | ImageModelAbility.Inpainting.value
return True, (ModelType.ImageGeneration, ability)
return False, None
# Define additional rules in similar fashion ...
rules = [embedding_rule, image_rule]
# Append additional rules as needed.
for rule in rules:
match, result = rule(model_id)
if match:
return result
# Fall back to your original LLM logic as default.
ability = LLMAbility.TextChat.value
# ... remaining conditions applied declaratively if possible.
return (ModelType.LLM, ability)您可以对 guess_qwen_model 应用类似的重构。这种方法：
- 减少深度嵌套： 每个规则都是独立的。
- 提高可维护性： 可以将新规则添加为函数或 lambda 条目。
- 保持功能完整： 逻辑保持不变，只是结构不同。
尝试一次重构几个部分，并通过单元测试验证行为，以确保现有功能得到保留。
Original comment in English
issue (complexity): Consider refactoring the nested if-else logic in guess_openai_model and guess_qwen_model into a declarative list of rules to improve readability and maintainability.
Consider extracting the branching logic into a declarative list of rules instead of deeply nested if-else blocks. This can reduce complexity and make each case easier to inspect and modify. For example, you could refactor guess_openai_model like so:
def guess_openai_model(model_id: str) -> Tuple[ModelType, int] | None:
model_id = model_id.lower()
def embedding_rule(m: str):
return "embedding" in m, (ModelType.Embedding, EmbeddingModelAbility.TextEmbedding.value | EmbeddingModelAbility.Batch.value)
def image_rule(m: str):
if "dall-e" in m or "gpt-image" in m:
ability = ImageModelAbility.TextToImage.value
if "dall-e-2" in m or "gpt-image" in m:
ability |= ImageModelAbility.ImageEdit.value | ImageModelAbility.Inpainting.value
return True, (ModelType.ImageGeneration, ability)
return False, None
# Define additional rules in similar fashion ...
rules = [embedding_rule, image_rule]
# Append additional rules as needed.
for rule in rules:
match, result = rule(model_id)
if match:
return result
# Fall back to your original LLM logic as default.
ability = LLMAbility.TextChat.value
# ... remaining conditions applied declaratively if possible.
return (ModelType.LLM, ability)You can apply a similar restructure for guess_qwen_model. This approach:
- Reduces deep nesting: Each rule is self-contained.
- Improves maintainability: New rules can be added as functions or lambda entries.
- Keeps functionality intact: The logic remains the same, only structured differently.
Try refactoring a few sections at a time and verify behavior with unit tests to ensure existing functionality is preserved.
Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
… TelegramAdapter - Added try-except blocks to handle exceptions during resource cleanup in MCPServer and TelegramAdapter. - Enhanced logging to capture errors that occur during shutdown processes, improving overall reliability and debuggability.
- Changed the expected memory usage percentage in the test from 2.5 to 0.5 to reflect accurate resource utilization. - Ensured that the test remains aligned with the current system status reporting.
lss233
force-pushed
the
feature/llm_ability
branch
from
c8de649 to
4bddfe2
Compare
Codecov ReportAttention: Patch coverage is
✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## master #1474 +/- ##
==========================================
+ Coverage 65.84% 66.17% +0.33%
==========================================
Files 161 162 +1
Lines 8148 8299 +151
==========================================
+ Hits 5365 5492 +127
- Misses 2783 2807 +24 ☔ View full report in Codecov by Sentry. |
- Introduced MediaConfig class to manage media-related settings, including cleanup duration and auto removal of unreferenced files. - Enhanced MediaManager with a setup_cleanup_task method to schedule automatic cleanup of unreferenced media files based on configuration. - Added new API endpoints for retrieving system information and updating media configuration, allowing for dynamic management of media settings. - Implemented tests for the new media management features to ensure functionality and reliability.
… logging - Added logging for errors during adapter stopping and starting processes to improve debuggability. - Enhanced the update_adapter function to handle adapter renaming and type validation more robustly. - Implemented checks for existing adapter names and types, returning appropriate error messages for invalid requests. - Updated the response structure to reflect the new adapter name and running status after updates.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sourcery 总结
通过引入全面的模型类型和能力分类机制，增强模型能力注释和检测系统
新特性:
增强功能:
杂项:
Original summary in English
Summary by Sourcery
Enhance model capability annotation and detection system by introducing a comprehensive model type and ability classification mechanism
New Features:
Enhancements:
Chores: