Skip to content

perf: parallelize MCP server initialization in get_all_tool_definitions#138

Open
JasonOA888 wants to merge 1 commit intoMiroMindAI:mainfrom
JasonOA888:perf/parallel-mcp-init
Open

perf: parallelize MCP server initialization in get_all_tool_definitions#138
JasonOA888 wants to merge 1 commit intoMiroMindAI:mainfrom
JasonOA888:perf/parallel-mcp-init

Conversation

@JasonOA888
Copy link

Partially addresses #137

Problem:
MCP tool servers were being initialized sequentially in a for loop:

  • tool-python (E2B sandbox): ~33s
  • search_and_scrape_webpage (Serper): ~21s
  • jina_scrape_llm_summary: ~17s
  • Total: ~71s per task

With 1266 BC-EN tasks, this adds significant overhead to evaluation runs.

Solution:

  • Refactored server connection logic into _get_server_tools() helper function
  • Used asyncio.gather() to connect to all servers in parallel
  • Expected savings: ~40-50s per task (parallel time = max of individual times, not sum)

Changes:

  • libs/miroflow-tools/src/miroflow_tools/manager.py:
    • New internal async function _get_server_tools(config)
    • Parallel execution via asyncio.gather(..., return_exceptions=True)
    • Graceful handling of exceptions from parallel execution

Error handling preserved:

  • Failed connections still add an error entry to results
  • Exceptions are logged and handled without crashing the entire initialization

Benchmark impact:

  • BC-EN (1266 tasks): ~40-50s × 1266 = ~14-17 hours saved per run
  • BC-ZH (289 tasks): ~4 hours saved per run

Partially addresses MiroMindAI#137

MCP tool servers were being initialized sequentially in a for loop,
causing ~70-80s overhead per task (tool-python ~33s, search ~21s, jina ~17s).

This change:
- Refactors server connection logic into a helper function _get_server_tools()
- Uses asyncio.gather() to connect to all servers in parallel
- Expected savings: ~40-50s per task initialization

The parallel approach maintains the same error handling behavior:
- Failed connections still add an error entry
- Exceptions from asyncio.gather are logged and handled gracefully
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant