perf: parallelize tool server init and reduce LLM retry overhead

[wangbinluo]

Summary

Addresses evaluation pipeline performance bottlenecks identified in #137.

Three changes targeting the slowest parts of BC benchmark evaluation.

Closes #137

Changes

• P0: Parallelize MCP tool server initialization — manager.py get_all_tool_definitions() now uses asyncio.gather() instead of sequential for loop.
• P0: Reduce LLM retry overhead — openai_client.py base_wait_time 30s → 10s, max_retries 10 → 5. Prevents 60-90s wasted on retries.
• P1: httpx connection pooling — search_and_scrape_webpage.py reuses a shared httpx.AsyncClient instead of creating a new one per request (~346 search calls per BC task).

Benchmark Results

Tool server initialization time (3 runs on dev_wbl)

Run	dev_wbl (parallel)
Run 1 (cold start)	29.3s
Run 2	12.1s
Run 3	9.6s
Average	17.0s

Comparison with main branch (extracted from existing evaluation logs)

	main (sequential)	dev_wbl (parallel)	Speedup
Average	234s	17.0s	13.8x
Min	22s	9.6s	2.3x
Max	945s (15.8 min)	29.3s	32.3x

Note: main branch baseline was extracted from BC-ZH evaluation logs (qwen_xxg_negative_r10_new1_step50, 30 tasks). The high average (234s) includes E2B sandbox queueing under high concurrency (MAX_CONCURRENT=60). Per-server breakdown on main: tool-python avg=145s (max=631s), search avg=69s, jina avg=19s.

Test plan

• Run bench_init_time.py on dev_wbl — 17.0s avg vs main baseline 234s avg
• Run BC-ZH evaluation end-to-end and verify no regression in accuracy
• Verify LLM retry behavior works correctly with reduced parameters
• Check Serper API connection pooling doesn't cause stale connection issues

[wangbinluo]

@shawnlimn @xingxuanli Could you review this PR when you get a chance?

This PR addresses the performance bottlenecks identified in #137 with two optimizations:

1. Parallel tool server init (asyncio.gather) — measured 13.8x speedup (234s → 17s avg)
2. httpx connection pooling — reuse TCP connections for Serper API (~346 calls per BC task)

Note: PR #138 by @JasonOA888 covers only item 1; this PR is a superset with additional optimization and benchmark data.