Add a simple Gradio UI for Open Deep Research #525
Conversation
| @@ -455,7 +454,7 @@ def planning_step(self, task, is_first_step: bool, step: int) -> None: | |||
| """ | |||
| if is_first_step: | |||
| message_prompt_facts = { | |||
| "role": MessageRole.SYSTEM, | |||
| "role": MessageRole.USER, | |||
There was a problem hiding this comment.
Won't this actually break other models?..
There was a problem hiding this comment.
OK I did some searching and it seems like there was a change recently that broke Gemini compatibility:
the above change makes the request to model() not include any USER , as it moves the task description to be a SYSTEM message. This change could have been made for a) increasing accuracy or b) simplyfing the prompting implementation. In any case, the case of models not supporting being sent a single system model was probably unknown and understadably overlooked. I can revert my change, the above change, or wait for instructions.
Regarding breaking compatibility, I know most (if not all) support being sent a single user message. The opposite is not true, as at least Gemini models do not yet support this
src/smolagents/models.py
Outdated
| @@ -686,6 +691,12 @@ def __call__( | |||
| ) -> ChatMessage: | |||
| import litellm | |||
|
|
|||
| # IMPORTANT - Set this to TRUE to add the function to the prompt for Non OpenAI LLMs | |||
| if litellm.supports_function_calling(model=self.model_id) == True: | |||
There was a problem hiding this comment.
if litellm.supports_function_calling(model=self.model_id):Or
if litellm.supports_function_calling(model=self.model_id) is True:if you want to be very specific.
There was a problem hiding this comment.
I'm new to this project so I'm not sure if some idioms are acceptable or not.
What do you think of:
litellm.add_function_to_prompt = not litellm.supports_function_calling(model=self.model_id)
Hi,
This PR makes a few changes to get open deep researcher working smoothly with Gemini 2.0 Flash through LiteLLM, and sets up a basic Gradio demo to show it off. I focused on Gemini 2.0 Flash specifically because it's a cheap way to experiment with the library.
Here's a breakdown of what I did:
Gemini 2.0 Flash has a couple of quirks that needed addressing:
planning_stepmethod insrc/smolagents/agents.pyto send theinitial_factsprompt as a user message instead of a system message. This resolves an error where Gemini would complain aboutcontentsbeing empty.litellm.add_function_to_promptset to True or False depending on the selected model.model_outputbecause sometimes returns None, causing error. Returned early in this case.getattrTo avoid errors whenlast_input_token_countisNoneTool Calling Fixes: The changes make sure Litellm can create a valid tool calling request for Gemini, avoiding an empty parameters dict.
Gradio Demo App: I've included a simple Gradio application.
Peek.2025-02-06.18-50.mp4
code_agent.yamlandtoolcalling_agent.yamlto improve accuracy when calling managed agents.Blocked by this PR on LiteLLMBut one can do:export LITELLM_LOCAL_MODEL_COST_MAP=Trueand set the config locally.