Skip to content

Commit 9494ac9

Browse files
authored
AGS Improvements (Add Test Button in Team Builder View + Others) (#5416)
<!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? - Add ability to test teams in Team Builder view - Update Gallery (add deep research default team, fix bug with gallery serialization) - UI fixes - improve drag drop component experience - improve new session experience (single click rather than 3 clicks to create a session) - fix bug with stop reason not being shown in some cases <img width="1738" alt="Image" src="https://github.com/user-attachments/assets/4b895df2-3bad-474e-bec6-4fbcbf1c4346" /> <img width="1761" alt="Image" src="https://github.com/user-attachments/assets/65f52eb9-e926-4168-88fb-d2496c159474" /> <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number Closes #5392 <!-- For example: "Closes #1234" --> ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 parent 73a7ba5 commit 9494ac9

File tree

20 files changed

+668
-141
lines changed

20 files changed

+668
-141
lines changed

Diff for: python/packages/autogen-core/docs/src/user-guide/autogenstudio-user-guide/faq.md

+66-22
Original file line numberDiff line numberDiff line change
@@ -15,51 +15,95 @@ A: You can specify the directory where files are stored by setting the `--appdir
1515

1616
Yes. AutoGen standardizes on the openai model api format, and you can use any api server that offers an openai compliant endpoint.
1717

18-
AutoGen Studio is based on declaritive specifications which applies to models as well. Agents can include a model_client field which specifies the model endpoint details including `model`, `api_key`, `base_url`, `model type`.
18+
AutoGen Studio is based on declaritive specifications which applies to models as well. Agents can include a model_client field which specifies the model endpoint details including `model`, `api_key`, `base_url`, `model type`. Note, you can define your [model client](https://microsoft.github.io/autogen/dev/user-guide/core-user-guide/components/model-clients.html) in python and dump it to a json file for use in AutoGen Studio.
1919

20-
An example of the openai model client is shown below:
20+
In the following sample, we will define an OpenAI, AzureOpenAI and a local model client in python and dump them to a json file.
21+
22+
```python
23+
from autogen_ext.models.openai import AzureOpenAIChatCompletionClient, OpenAIChatCompletionClient
24+
from autogen_core.models import ModelInfo
25+
26+
model_client=OpenAIChatCompletionClient(
27+
model="gpt-4o-mini",
28+
)
29+
print(model_client.dump_component().model_dump_json())
30+
31+
az_model_client = AzureOpenAIChatCompletionClient(
32+
azure_deployment="{your-azure-deployment}",
33+
model="gpt-4o",
34+
api_version="2024-06-01",
35+
azure_endpoint="https://{your-custom-endpoint}.openai.azure.com/",
36+
api_key="sk-...",
37+
)
38+
print(az_model_client.dump_component().model_dump_json())
39+
40+
mistral_vllm_model = OpenAIChatCompletionClient(
41+
model="TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
42+
base_url="http://localhost:1234/v1",
43+
model_info=ModelInfo(vision=False, function_calling=True, json_output=False, family="unknown"),
44+
)
45+
print(mistral_vllm_model.dump_component().model_dump_json())
46+
```
47+
48+
OpenAI
2149

2250
```json
2351
{
24-
"model": "gpt-4o-mini",
25-
"model_type": "OpenAIChatCompletionClient",
26-
"api_key": "your-api-key"
52+
"provider": "autogen_ext.models.openai.OpenAIChatCompletionClient",
53+
"component_type": "model",
54+
"version": 1,
55+
"component_version": 1,
56+
"description": "Chat completion client for OpenAI hosted models.",
57+
"label": "OpenAIChatCompletionClient",
58+
"config": { "model": "gpt-4o-mini" }
2759
}
2860
```
2961

30-
An example of the azure openai model client is shown below:
62+
Azure OpenAI
3163

3264
```json
3365
{
34-
"model": "gpt-4o-mini",
35-
"model_type": "AzureOpenAIChatCompletionClient",
36-
"azure_deployment": "gpt-4o-mini",
37-
"api_version": "2024-02-15-preview",
38-
"azure_endpoint": "https://your-endpoint.openai.azure.com/",
39-
"api_key": "your-api-key",
40-
"component_type": "model"
66+
"provider": "autogen_ext.models.openai.AzureOpenAIChatCompletionClient",
67+
"component_type": "model",
68+
"version": 1,
69+
"component_version": 1,
70+
"description": "Chat completion client for Azure OpenAI hosted models.",
71+
"label": "AzureOpenAIChatCompletionClient",
72+
"config": {
73+
"model": "gpt-4o",
74+
"api_key": "sk-...",
75+
"azure_endpoint": "https://{your-custom-endpoint}.openai.azure.com/",
76+
"azure_deployment": "{your-azure-deployment}",
77+
"api_version": "2024-06-01"
78+
}
4179
}
4280
```
4381

4482
Have a local model server like Ollama, vLLM or LMStudio that provide an OpenAI compliant endpoint? You can use that as well.
4583

4684
```json
4785
{
48-
"model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
49-
"model_type": "OpenAIChatCompletionClient",
50-
"base_url": "http://localhost:1234/v1",
51-
"api_version": "1.0",
86+
"provider": "autogen_ext.models.openai.OpenAIChatCompletionClient",
5287
"component_type": "model",
53-
"model_capabilities": {
54-
"vision": false,
55-
"function_calling": true,
56-
"json_output": false
88+
"version": 1,
89+
"component_version": 1,
90+
"description": "Chat completion client for OpenAI hosted models.",
91+
"label": "OpenAIChatCompletionClient",
92+
"config": {
93+
"model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
94+
"model_info": {
95+
"vision": false,
96+
"function_calling": true,
97+
"json_output": false,
98+
"family": "unknown"
99+
},
100+
"base_url": "http://localhost:1234/v1"
57101
}
58102
}
59103
```
60104

61105
```{caution}
62-
It is important that you add the `model_capabilities` field to the model client specification for custom models. This is used by the framework instantiate and use the model correctly. Also, the `AssistantAgent` and many other agents in AgentChat require the model to have the `function_calling` capability.
106+
It is important that you add the `model_info` field to the model client specification for custom models. This is used by the framework instantiate and use the model correctly. Also, the `AssistantAgent` and many other agents in AgentChat require the model to have the `function_calling` capability.
63107
```
64108

65109
## Q: The server starts but I can't access the UI

Diff for: python/packages/autogen-core/docs/src/user-guide/autogenstudio-user-guide/index.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@ myst:
1212

1313
AutoGen Studio is a low-code interface built to help you rapidly prototype AI agents, enhance them with tools, compose them into teams and interact with them to accomplish tasks. It is built on [AutoGen AgentChat](https://microsoft.github.io/autogen) - a high-level API for building multi-agent applications.
1414

15-
![AutoGen Studio](https://media.githubusercontent.com/media/microsoft/autogen/refs/heads/main/python/packages/autogen-studio/docs/ags_screen.png)
15+
> See a video tutorial on AutoGen Studio v0.4 (02/25) - [https://youtu.be/oum6EI7wohM](https://youtu.be/oum6EI7wohM)
16+
17+
[![A Friendly Introduction to AutoGen Studio v0.4](https://img.youtube.com/vi/oum6EI7wohM/maxresdefault.jpg)](https://www.youtube.com/watch?v=oum6EI7wohM)
1618

1719
Code for AutoGen Studio is on GitHub at [microsoft/autogen](https://github.com/microsoft/autogen/tree/main/python/packages/autogen-studio)
1820

Diff for: python/packages/autogen-core/docs/src/user-guide/autogenstudio-user-guide/usage.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,9 @@ myst:
1010
AutoGen Studio provides a Team Builder interface where developers can define multiple components and behaviors. Users can create teams, add agents to teams, attach tools and models to agents, and define team termination conditions.
1111
After defining a team, users can test it in the Playground view to accomplish various tasks through direct interaction.
1212

13-
![AutoGen Studio](https://media.githubusercontent.com/media/microsoft/autogen/refs/heads/main/python/packages/autogen-studio/docs/ags_screen.png)
13+
> See a video tutorial on AutoGen Studio v0.4 (02/25) - [https://youtu.be/oum6EI7wohM](https://youtu.be/oum6EI7wohM)
14+
15+
[![A Friendly Introduction to AutoGen Studio v0.4](https://img.youtube.com/vi/oum6EI7wohM/maxresdefault.jpg)](https://www.youtube.com/watch?v=oum6EI7wohM)
1416

1517
## Declarative Specification of Componenents
1618

@@ -100,8 +102,6 @@ This example shows a team with a single agent, using the `RoundRobinGroupChat` t
100102

101103
## Building an Agent Team
102104

103-
<div style="padding:58.13% 0 0 0;position:relative; border-radius:5px; border-bottom:10px"><iframe src="https://player.vimeo.com/video/1043133833?badge=0&amp;autopause=0&amp;player_id=0&amp;app_id=58479" frameborder="0" allow="autoplay; fullscreen; picture-in-picture; clipboard-write; encrypted-media" style="position:absolute;top:0;left:0;width:100%;height:100%;" title="AutoGen Studio v0.4x - Drag and Drop Interface"></iframe></div><script src="https://player.vimeo.com/api/player.js"></script>
104-
105105
<br/>
106106

107107
AutoGen Studio integrates closely with all component abstractions provided by AutoGen AgentChat, including {py:class}`~autogen_agentchat.teams`, {py:class}`~autogen_agentchat.agents`, {py:class}`~autogen_core.models`, {py:class}`~autogen_core.tools`, and termination {py:class}`~autogen_agentchat.conditions`.
@@ -117,6 +117,8 @@ Team Builder Operations:
117117
- Agents: Add models and tools
118118
- Save team configurations
119119

120+
Note: For each node in the visual builder, you can click on the edit icon (top right) to view and edit the JSON configuration.
121+
120122
## Gallery - Sharing and Reusing Components
121123

122124
A Gallery is a collection of components - teams, agents, models, tools, and terminations - that can be shared and reused across projects.

Diff for: python/packages/autogen-studio/autogenstudio/gallery/builder.py

+81-3
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ def create_default_gallery() -> Gallery:
148148
mistral_vllm_model = OpenAIChatCompletionClient(
149149
model="TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
150150
base_url="http://localhost:1234/v1",
151-
model_info=ModelInfo(vision=False, function_calling=True, json_output=False),
151+
model_info=ModelInfo(vision=False, function_calling=True, json_output=False, family="unknown"),
152152
)
153153
builder.add_model(
154154
mistral_vllm_model.dump_component(),
@@ -236,6 +236,7 @@ def create_default_gallery() -> Gallery:
236236
model_client=base_model,
237237
termination_condition=web_termination,
238238
)
239+
239240
builder.add_team(
240241
websurfer_team.dump_component(),
241242
label="Web Agent Team (Operator)",
@@ -256,8 +257,8 @@ def create_default_gallery() -> Gallery:
256257

257258
builder.add_tool(
258259
tools.fetch_webpage_tool.dump_component(),
259-
label="Webpage Generation Tool",
260-
description="A tool that generates a webpage from a list of images. Requires beautifulsoup4 html2text library to function.",
260+
label="Fetch Webpage Tool",
261+
description="A tool that fetches the content of a webpage and converts it to markdown. Requires the requests and beautifulsoup4 library to function.",
261262
)
262263

263264
builder.add_tool(
@@ -272,6 +273,83 @@ def create_default_gallery() -> Gallery:
272273
description="A tool that performs Google searches using the Google Custom Search API. Requires the requests library, [GOOGLE_API_KEY, GOOGLE_CSE_ID] to be set, env variable to function.",
273274
)
274275

276+
# Create deep research agent
277+
model_client = OpenAIChatCompletionClient(model="gpt-4o", temperature=0.7)
278+
279+
research_assistant = AssistantAgent(
280+
name="research_assistant",
281+
description="A research assistant that performs web searches and analyzes information",
282+
model_client=model_client,
283+
tools=[tools.google_search_tool, tools.fetch_webpage_tool],
284+
system_message="""You are a research assistant focused on finding accurate information.
285+
Use the google_search tool to find relevant information.
286+
Break down complex queries into specific search terms.
287+
Always verify information across multiple sources when possible.
288+
When you find relevant information, explain why it's relevant and how it connects to the query. When you get feedback from the a verifier agent, use your tools to act on the feedback and make progress.""",
289+
)
290+
291+
verifier = AssistantAgent(
292+
name="verifier",
293+
description="A verification specialist who ensures research quality and completeness",
294+
model_client=model_client,
295+
system_message="""You are a research verification specialist.
296+
Your role is to:
297+
1. Verify that search queries are effective and suggest improvements if needed
298+
2. Explore drill downs where needed e.g, if the answer is likely in a link in the returned search results, suggest clicking on the link
299+
3. Suggest additional angles or perspectives to explore. Be judicious in suggesting new paths to avoid scope creep or wasting resources, if the task appears to be addressed and we can provide a report, do this and respond with "TERMINATE".
300+
4. Track progress toward answering the original question
301+
5. When the research is complete, provide a detailed summary in markdown format. For incomplete research, end your message with "CONTINUE RESEARCH". For complete research, end your message with APPROVED.
302+
Your responses should be structured as:
303+
- Progress Assessment
304+
- Gaps/Issues (if any)
305+
- Suggestions (if needed)
306+
- Next Steps or Final Summary""",
307+
)
308+
309+
summary_agent = AssistantAgent(
310+
name="summary_agent",
311+
description="A summary agent that provides a detailed markdown summary of the research as a report to the user.",
312+
model_client=model_client,
313+
system_message="""You are a summary agent. Your role is to provide a detailed markdown summary of the research as a report to the user. Your report should have a reasonable title that matches the research question and should summarize the key details in the results found in natural an actionable manner. The main results/answer should be in the first paragraph.
314+
Your report should end with the word "TERMINATE" to signal the end of the conversation.""",
315+
)
316+
317+
termination = TextMentionTermination("TERMINATE") | MaxMessageTermination(max_messages=30)
318+
319+
selector_prompt = """You are coordinating a research team by selecting the team member to speak/act next. The following team member roles are available:
320+
{roles}.
321+
The research_assistant performs searches and analyzes information.
322+
The verifier evaluates progress and ensures completeness.
323+
The summary_agent provides a detailed markdown summary of the research as a report to the user.
324+
325+
Given the current context, select the most appropriate next speaker.
326+
The research_assistant should search and analyze.
327+
The verifier should evaluate progress and guide the research (select this role is there is a need to verify/evaluate progress). You should ONLY select the summary_agent role if the research is complete and it is time to generate a report.
328+
329+
Base your selection on:
330+
1. Current stage of research
331+
2. Last speaker's findings or suggestions
332+
3. Need for verification vs need for new information
333+
Read the following conversation. Then select the next role from {participants} to play. Only return the role.
334+
335+
{history}
336+
337+
Read the above conversation. Then select the next role from {participants} to play. ONLY RETURN THE ROLE."""
338+
339+
deep_research_team = SelectorGroupChat(
340+
participants=[research_assistant, verifier, summary_agent],
341+
model_client=model_client,
342+
termination_condition=termination,
343+
selector_prompt=selector_prompt,
344+
allow_repeated_speaker=True,
345+
)
346+
347+
builder.add_team(
348+
deep_research_team.dump_component(),
349+
label="Deep Research Team",
350+
description="A team that performs deep research using web searches, verification, and summarization.",
351+
)
352+
275353
return builder.build()
276354

277355

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
VERSION = "0.4.0"
1+
VERSION = "0.4.1"
22
__version__ = VERSION
33
APP_NAME = "autogenstudio"

Diff for: python/packages/autogen-studio/frontend/src/components/sidebar.tsx

+1-1
Original file line numberDiff line numberDiff line change
@@ -220,7 +220,7 @@ const Sidebar = ({ link, meta, isMobile }: SidebarProps) => {
220220
],
221221
})
222222
}
223-
className="group flex gap-x-3 rounded-md p-2 text-sm font-medium text-primary hover:text-accent hover:bg-secondary justify-center"
223+
className="group hidden flex gap-x-3 rounded-md p-2 text-sm font-medium text-primary hover:text-accent hover:bg-secondary justify-center"
224224
>
225225
<Settings className="h-6 w-6 shrink-0 text-secondary group-hover:text-accent" />
226226
</Link>

0 commit comments

Comments
 (0)