feat: ReAct Implementation #1515

GitHoobar · 2025-01-27T19:42:59Z

Description

Implementing ReAct : Synergizing Reasoning and Acting in Language Models: https://arxiv.org/pdf/2210.03629 with structured output

Motivation and Context

close #1495

I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds core functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation (update in the documentation)
Example (update in the folder of example)

Implemented Tasks

ReAct Agent and examples

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

I have read the CONTRIBUTION guide. (required)
My change requires a change to the documentation.
I have updated the tests accordingly. (required for a bug fix or a new feature)
I have updated the documentation accordingly.

GitHoobar · 2025-01-27T19:44:38Z

will be adding tests in some time!

yiyiyi0817

Thanks for your code. Leave some comments.

I also have a question and would be grateful for your answer:
Maybe it's because I haven't carefully read the ReAct paper. I noticed that the code implements step using "Thought," "Action," and "Observation." If "Thought" is just the response content of one step in ChatAgent, "Action" is function calling, and "Observation" is the agent's memory, then isn't ReActAgent very similar to our existing ChatAgent? What might be the possible differences between them?

camel/agents/react_agent.py

yiyiyi0817 · 2025-02-06T09:34:23Z

camel/agents/react_agent.py

+            },
+        )
+
+    def _parse_react_components(


I see that more LLM structured output is extracted by generating JSON or XML, which may have better results than using colons and strings.
And since camel-ai already has a module for structured output, I think it would be more robust to use camel's existing structured input and output. What do you think?
reference: https://github.com/camel-ai/camel/tree/master/examples/structured_response

yiyiyi0817 · 2025-02-06T09:38:00Z

camel/agents/react_agent.py

+            components["Observation"],
+        )
+
+    def _execute_action(self, action: str) -> str:


If we use more structured output, maybe we can also input the action type (such as ReActActionSpace) and parameters of actions here.

examples/agents/react_agent_example.py

yiyiyi0817 · 2025-02-06T11:49:51Z

camel/agents/react_agent.py

+                if hasattr(tool, 'can_handle') and tool.can_handle(action):
+                    logger.debug("Found tool to handle action")
+                    if hasattr(tool, 'execute'):
+                        return tool.execute(action)


Maybe I didn't quite understand this part. I noticed that the functions can_handle and execute are used here. But the example provided is MathToolkit, it seems have no execute method. And also the action space includes search, lookup, and finish. Is there some inconsistency here?

GitHoobar · 2025-02-06T13:05:33Z

Thanks for your code. Leave some comments.

I also have a question and would be grateful for your answer: Maybe it's because I haven't carefully read the ReAct paper. I noticed that the code implements step using "Thought," "Action," and "Observation." If "Thought" is just the response content of one step in ChatAgent, "Action" is function calling, and "Observation" is the agent's memory, then isn't ReActAgent very similar to our existing ChatAgent? What might be the possible differences between them?

Ahh nice observation, While ReActAgent and ChatAgent share some similarities in their step-by-step processing, there are differences like:

ReActAgent enforces a strict "Thought-Action-Observation" format in each step
ReActAgent has a predefined action space (Search, Lookup, Finish)

class ReActActionSpace(Enum):
    SEARCH = "Search"
    LOOKUP = "Lookup"
    FINISH = "Finish"

It maintains a memory of previous thoughts/action/observations
It terminates an action only if "Finish" is seen (or max steps reached)
It encourages to use tools before getting to answer, which is not the case with ChatAgent

Wendong-Fan

Thanks @GitHoobar for the contribution! Left some comments below, and I also wanted to highlight some points that @yiyiyi0817 mentioned

camel/agents/react_agent.py

examples/agents/react_agent_example.py

…ReAct-implementation

yiyiyi0817 · 2025-02-11T07:41:29Z

Thanks for your code. Leave some comments.
I also have a question and would be grateful for your answer: Maybe it's because I haven't carefully read the ReAct paper. I noticed that the code implements step using "Thought," "Action," and "Observation." If "Thought" is just the response content of one step in ChatAgent, "Action" is function calling, and "Observation" is the agent's memory, then isn't ReActAgent very similar to our existing ChatAgent? What might be the possible differences between them?

Ahh nice observation, While ReActAgent and ChatAgent share some similarities in their step-by-step processing, there are differences like:

ReActAgent enforces a strict "Thought-Action-Observation" format in each step

ReActAgent has a predefined action space (Search, Lookup, Finish)
class ReActActionSpace(Enum):
    SEARCH = "Search"
    LOOKUP = "Lookup"
    FINISH = "Finish"
It maintains a memory of previous thoughts/action/observations

It terminates an action only if "Finish" is seen (or max steps reached)

It encourages to use tools before getting to answer, which is not the case with ChatAgent

Thanks for your explaination.
Are actions in ReAct the same as using tools? Is ReAct designed for general tools for common scenarios, or for some specific search tasks? ReActActionSpace seems to be designed for some web tasks. I'm not sure if duckduckgo's action space is inconsistent with this, because there is only one search function.
And ChatAgent can also use tools extensively and proceed to the result based on the tool's return.

yiyiyi0817

Still a little confused about some places, I left some references for discussion.

yiyiyi0817 · 2025-02-11T07:46:21Z

examples/agents/react_agent_example.py

+
+
+if __name__ == "__main__":
+    main()


I pull this branch and run the scripts, the response is

JSON response: {'thought': "I need to find the most recent population data for Paris, as I don't have access to future data directly. The latest available population figure can be used to estimate the 2024 population based on growth trends.", 'action': 'Lookup', 'observation': [{'result_id': 1, 'title': 'LOOKUP function - Microsoft Support', 'description': 'Learn how to use LOOKUP function to search for a value in a single row or column and return a value from the same position in another row or column. Compare LOOKUP with VLOOKUP and XLOOKUP and see examples and syntax.', 'url': 'https://support.microsoft.com/en-us/office/lookup-function-446d94af-663b-451d-8251-369d5e3864cb'}, {'result_id': 2, 'title': 'Excel LOOKUP function | Exceljet', 'description': 'Learn how to use the Excel LOOKUP function to perform an approximate match lookup in a one-column or one-row range, and return the corresponding value from another range. See examples, syntax, and tips for the vector and array forms of LOOKUP.', 'url': 'https://exceljet.net/functions/lookup-function'}, {'result_id': 3, 'title': 'Different Types of Lookup to Apply in Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUP, VLOOKUP, XLOOKUP, OFFSET, MATCH, INDEX, XMATCH and more.', 'url': 'https://www.exceldemy.com/types-of-lookup-in-excel/'}, {'result_id': 4, 'title': 'Excel LOOKUP function with formula examples - Ablebits', 'description': 'Learn how to use the Excel LOOKUP function to search and retrieve values in one-column or one-row ranges, or in arrays. See formula examples, tips and alternatives for different scenarios of lookup.', 'url': 'https://www.ablebits.com/office-addins-blog/excel-lookup-function/'}, {'result_id': 5, 'title': '6 Formulas to Lookup in Excel', 'description': 'Learn how to use different Excel functions and formulas to lookup values in tables, from left to right or right to left, horizontally or vertically. Compare the advantages and disadvantages of VLOOKUP, HLOOKUP, INDEX-MATCH, OFFSET-MATCH, LOOKUP and LOOKUP (multiple values).', 'url': 'https://www.exceltip.com/lookup-formulas/6-formulas-to-lookup-in-excel.html'}]} -------------------------------------------------- JSON response: {'thought': 'I need to find the current population of Paris, which is the capital of France. The previous lookup did not yield the population data directly, so I will search for the latest population statistics for Paris.', 'action': 'Lookup', 'observation': [{'result_id': 1, 'title': 'LOOKUP function - Microsoft Support', 'description': 'Learn how to use LOOKUP function to search for a value in a single row or column and return a value from the same position in another row or column. Compare LOOKUP with VLOOKUP and XLOOKUP and see examples and syntax.', 'url': 'https://support.microsoft.com/en-us/office/lookup-function-446d94af-663b-451d-8251-369d5e3864cb'}, {'result_id': 2, 'title': 'Excel LOOKUP function | Exceljet', 'description': 'Learn how to use the Excel LOOKUP function to perform an approximate match lookup in a one-column or one-row range, and return the corresponding value from another range. See examples, syntax, and tips for the vector and array forms of LOOKUP.', 'url': 'https://exceljet.net/functions/lookup-function'}, {'result_id': 3, 'title': 'Different Types of Lookup to Apply in Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUP, VLOOKUP, XLOOKUP, OFFSET, MATCH, INDEX, XMATCH and more.', 'url': 'https://www.exceldemy.com/types-of-lookup-in-excel/'}, {'result_id': 4, 'title': 'Excel LOOKUP function with formula eption': 'Learn how to use the Excel LOOKUP function to perform an approximate match lookup in a one-column or one-row range, and return the corresponding value from another range. See examples, syntax, and tips for the vector and array forms of LOOKUP.', 'url': 'https://exceljet.net/functions/lookup-function'}, {'result_id': 3, 'title': 'Different Types of Lookup to Apply in Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUP, VLOOKUP, XLOOKUP, OFFSET, MATCH, INDEX, XMATCH and more.', 'url': 'https://www.exceldemy.com/types-of-lookup-in-excel/'}, {'result_id': 4, 'title': 'Excel LOOKUP function with formula eption': 'Learn how to use the Excel LOOKUP function to perform an approximate match lookup in a one-column or one-row range, and return the corresponding value from another range. See examples, syntax, and tips for the vector and array forms of LOOKUP.', 'url': 'https://exceljet.net/functions/lookup-function'}, {'result_id': 3, 'title': 'Different Types of Lookup to Apply iption': 'Learn how to use the Excel LOOKUP function to perform an approximate match lookup in a one-column or one-row range, and return the corresponding value from another range. See examption': 'Learn how to use the Excel LOOKUP function to perform an approximate match lookup in a one-column or one-row range, and return the corresponding value from another range. See examption': 'Learn how to use the Excel LOOKUP function to perform an approximate match lookup in a one-column or one-row range, and return the corresponding value from another range. See examption': 'Learn how to use the Excel LOOKUP function to perform an approximate match lookup in a one-column or one-row range, and return the corresponding value from another range. See examples, syntax, and tips for the vector and array forms of LOOKUP.', 'url': 'https://exceljet.net/functions/lookup-function'}, {'result_id': 3, 'title': 'Different Types of Lookup to Apply in Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUP, VLOOKUP, XLOOKUP, OFFSET, MATCH, INDEX, XMATCH and more.', 'url': 'https://www.exceldemy.com/types-of-lookup-in-excel/'}, {'result_id': 4, 'title': 'Excel LOOKUP function with formula examples - Ablebits', 'description': 'Learn how to use the Excel LOOKUP function to search and retrieve values in one-column or one-row ranges, or in arrays. See formula examples, tips and alternatives for different scenarios of lookup.', 'url': 'https://www.ablebits.com/office-addins-blog/excel-lookup-function/'}, {'result_id': 5, 'title': '6 Formulas to Lookup in Excel', 'n Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUP, VLOOKUP, XLOOKUP, OFFSET, MATCH, INDEX, XMATCH and more.', 'url': 'https://www.exceldemy.com/types-of-lookup-in-excel/'}, {'result_id': 4, 'title': 'Excel LOOKUP function with formula examples - Ablebits', 'description': 'Learn how to use the Excel LOOKUP function to search and retrieve values in one-column or one-row ranges, or in arrays. See formula examples, tips and n Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUP, VLOOKUP, XLOOKUP, OFFSET, MATCH, INDEX, XMATCH and more.', 'url': 'https://www.exceldemy.com/types-of-lookup-in-excel/'}, {'result_id': 4, 'title': 'Excel LOOKUP function with formula en Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUP, VLOOKUP, XLOOKUP, OFFSET, MATCH, INDEX, XMATCH and more.', 'url': 'https://www.exceldemy.com/types-of-lookup-in-excel/'}, {'result_id': 4, 'title': 'Excel LOOKUP function with formula en Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUP, VLOOKUP, XLOOKUP, OFFSET, MATCH, INDEX, XMATCH and more.', 'url': 'https://www.exceldemy.com/types-of-lookup-in-excel/'}, {'result_id': 4, 'title': 'Excel LOOKUP function with formula en Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUn Excel (8 Types)', 'description': 'Learn how to use different types of lookup functions and formulas in Excel to search for values in a table or array. Compare and contrast LOOKUP, HLOOKUP, VLOOKUP, XLOOKUP, OFFSET, MATCH, INDEX, XMATCH and more.', 'url': 'https://www.exceldemy.com/types-of-lookup-in-excel/'}, {'result_id': 4, 'title': 'Excel LOOKUP function with formula examples - Ablebits', 'description': 'Learn how to use the Excel LOOKUP function to search and retrieve values in one-column or one-row ranges, or in arrays. See formula examples, tips and alternatives for different scenarios of lookup.', 'url': 'https://www.ablebits.com/office-addins-blog/excel-lookup-function/'}, {'result_id': 5, 'title': '6 Formulas to Lookup in Excel', 'description': 'Learn how to use different Excel functions and formulas to lookup values in tables, from left to right or right to left, horizontally or vertically. Compare the advantages and disadvantages of VLOOKUP, HLOOKUP, INDEX-MATCH, OFFSET-MATCH, LOOKUP and LOOKUP (multiple values).', 'url': 'https://www.exceltip.com/lookup-formulas/6-formulas-to-lookup-in-excel.html'}]}

It seems not get information from duckduckgo.

camel/agents/react_agent.py

GitHoobar · 2025-02-11T16:02:23Z

Thanks for your code. Leave some comments.
I also have a question and would be grateful for your answer: Maybe it's because I haven't carefully read the ReAct paper. I noticed that the code implements step using "Thought," "Action," and "Observation." If "Thought" is just the response content of one step in ChatAgent, "Action" is function calling, and "Observation" is the agent's memory, then isn't ReActAgent very similar to our existing ChatAgent? What might be the possible differences between them?

Ahh nice observation, While ReActAgent and ChatAgent share some similarities in their step-by-step processing, there are differences like:

ReActAgent enforces a strict "Thought-Action-Observation" format in each step

ReActAgent has a predefined action space (Search, Lookup, Finish)
class ReActActionSpace(Enum):
    SEARCH = "Search"
    LOOKUP = "Lookup"
    FINISH = "Finish"
It maintains a memory of previous thoughts/action/observations

It terminates an action only if "Finish" is seen (or max steps reached)

It encourages to use tools before getting to answer, which is not the case with ChatAgent
Thanks for your explaination. Are actions in ReAct the same as using tools? Is ReAct designed for general tools for common scenarios, or for some specific search tasks? ReActActionSpace seems to be designed for some web tasks. I'm not sure if duckduckgo's action space is inconsistent with this, because there is only one search function. And ChatAgent can also use tools extensively and proceed to the result based on the tool's return.

The ReAct as described in paper is used for search tools, and the Lookup is used to find relevant sources of information from DB or queries.
We can update and include more functionalities if needed, but the paper demonstrates it use in web search but can be extended.
ChatAgent is more general and directly uses tools without the structured reasoning steps

Wendong-Fan

Hey @GitHoobar , I’ve noticed that the ReActAgent sometimes runs into the Finish tag even when the task isn’t actually finished. There are two things we can look into:

For the _set_react_prompt prompt, we could clarify when the agent should trigger this tag.
Consider using a more specific tag wording, like CAMEL_TASK_DONE, to avoid misinterpretation.

Let me know what you think!

GitHoobar · 2025-02-14T04:20:58Z

Hey @GitHoobar , I’ve noticed that the ReActAgent sometimes runs into the Finish tag even when the task isn’t actually finished. There are two things we can look into:

For the _set_react_prompt prompt, we could clarify when the agent should trigger this tag.

Consider using a more specific tag wording, like CAMEL_TASK_DONE, to avoid misinterpretation.

Let me know what you think!

I don't see any improvement after changing _set_react_prompt or adding any validation have tried a couple of variations

GitHoobar added 2 commits January 28, 2025 00:38

implementing ReAct Agent

9fd4c77

example for react agent

92161a5

GitHoobar linked an issue Jan 27, 2025 that may be closed by this pull request

[Feature Request] Implement ReAct #1495

Open

2 tasks

GitHoobar requested review from Wendong-Fan, lightaime, ZIYU-DEEP and Asher-hss January 27, 2025 19:43

added react agent tests

644bd86

GitHoobar marked this pull request as ready for review January 28, 2025 10:35

Wendong-Fan assigned GitHoobar Jan 28, 2025

Wendong-Fan added the New Feature label Jan 28, 2025

Wendong-Fan modified the milestones: Sprint 5, Sprint 22 Jan 28, 2025

Wendong-Fan and others added 2 commits January 30, 2025 02:10

Merge branch 'master' into feat/ReAct-implementation

60612a3

Merge branch 'master' into feat/ReAct-implementation

78d1312

Wendong-Fan mentioned this pull request Feb 5, 2025

[Feature Request] Implement Learn-by-interact #1488

Open

2 tasks

Wendong-Fan requested review from hallerite and yiyiyi0817 and removed request for Asher-hss February 5, 2025 20:14

yiyiyi0817 reviewed Feb 6, 2025

View reviewed changes

Merge branch 'master' into feat/ReAct-implementation

896fb14

Wendong-Fan reviewed Feb 9, 2025

View reviewed changes

GitHoobar added 5 commits February 10, 2025 04:01

fix

c1b5c89

fix

077fbce

Merge branch 'master' of https://github.com/camel-ai/camel into feat/…

0ffc0c9

…ReAct-implementation

refactor: complete track_agent implementation in commons.py

eeadc2d

Merge branch 'master' of https://github.com/camel-ai/camel into feat/…

80b6756

…ReAct-implementation

changes

e00d094

yiyiyi0817 reviewed Feb 11, 2025

View reviewed changes

fixes

2240f61

Wendong-Fan reviewed Feb 11, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: ReAct Implementation #1515

feat: ReAct Implementation #1515

GitHoobar commented Jan 27, 2025

GitHoobar commented Jan 27, 2025

yiyiyi0817 left a comment •

edited

Loading

yiyiyi0817 Feb 6, 2025

yiyiyi0817 Feb 6, 2025

yiyiyi0817 Feb 6, 2025 •

edited

Loading

GitHoobar commented Feb 6, 2025

Wendong-Fan left a comment

yiyiyi0817 commented Feb 11, 2025 •

edited

Loading

yiyiyi0817 left a comment

yiyiyi0817 Feb 11, 2025

GitHoobar commented Feb 11, 2025

Wendong-Fan left a comment •

edited

Loading

GitHoobar commented Feb 14, 2025

feat: ReAct Implementation #1515

Are you sure you want to change the base?

feat: ReAct Implementation #1515

Conversation

GitHoobar commented Jan 27, 2025

Description

Motivation and Context

Types of changes

Implemented Tasks

Checklist

GitHoobar commented Jan 27, 2025

yiyiyi0817 left a comment • edited Loading

Choose a reason for hiding this comment

yiyiyi0817 Feb 6, 2025

Choose a reason for hiding this comment

yiyiyi0817 Feb 6, 2025

Choose a reason for hiding this comment

yiyiyi0817 Feb 6, 2025 • edited Loading

Choose a reason for hiding this comment

GitHoobar commented Feb 6, 2025

Wendong-Fan left a comment

Choose a reason for hiding this comment

yiyiyi0817 commented Feb 11, 2025 • edited Loading

yiyiyi0817 left a comment

Choose a reason for hiding this comment

yiyiyi0817 Feb 11, 2025

Choose a reason for hiding this comment

GitHoobar commented Feb 11, 2025

Wendong-Fan left a comment • edited Loading

Choose a reason for hiding this comment

GitHoobar commented Feb 14, 2025

yiyiyi0817 left a comment •

edited

Loading

yiyiyi0817 Feb 6, 2025 •

edited

Loading

yiyiyi0817 commented Feb 11, 2025 •

edited

Loading

Wendong-Fan left a comment •

edited

Loading