[Feature Request] Manage LLM model configurations via external files (config)

[Barca0412]

Required prerequisites

• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Motivation

Currently, large language model (LLM) configurations (e.g., model = ModelFactory.create(...)) are hardcoded in the codebase. This approach requires users to modify the source code directly if they want to add new model types or adjust configurations. However, this creates a maintenance burden: when the code is updated, local modifications may conflict with upstream changes.

Solution

Proposed Solution

1. Abstract LLM-related parameters into a configuration class.
2. Manage these configurations via external files (e.g., config.json or config.yaml).

This would allow users to:

• Customize models without altering source code.
• Avoid conflicts during upstream updates.
• Easily share/reuse configurations across environments.

Example Implementation

# Load from config file  
with open("llm_config.yaml") as f:  
    config = load_yaml(f)  

model = ModelFactory.create(**config)  

This approach aligns with common practices for modular and maintainable code. Thanks for considering this suggestion!

Alternatives

No response

Additional context

No response

[lightaime]

Thanks @Barca0412. This is a very good sugggestion. We should support serialization and persistence in the core modules. Using pydantic could be a solution to this. c.c @Wendong-Fan