Should agent frameworks route tools before LLM inference instead of exposing the full tool set?

[SAURAV SINGLA (sauravsingla)]

I’m interested in feedback from the AutoGen community on a tool-orchestration design question.

As tool ecosystems grow, should the LLM always receive the complete available tool set, or should a deterministic routing layer first construct a smaller task-relevant action space?

I’ve been working on AgentWeave, where the routing layer sits before function-calling inference:

user task
→ candidate tools
→ pre-inference routing
→ selected tool subset
→ function-calling model

The underlying model itself is unchanged.

In a frozen BFCL-derived routing-pressure experiment with Hammer2.1-1.5B, the routed configuration achieved 6/48 native task successes while all-tools, random top-k, and semantic top-k baselines each achieved 0/48. The routed setup also exposed substantially fewer tool schemas and input tokens.

Important caveat: these are not official BFCL leaderboard scores; the experiment intentionally added deterministic routing pressure. We have separately opened a BFCL-compatible upstream handler so the same system can be evaluated under standard BFCL data.

The part I find most interesting is that semantic retrieval had higher candidate-tool recall than AgentWeave but still had zero downstream task successes.

That raises a broader question:

Is tool retrieval recall sufficient, or does the structure and size of the model-visible action space materially affect downstream function-calling reliability?

I’d be interested to hear how AutoGen users currently handle:

• large tool catalogs;
• dynamic tool filtering;
• tool routing before inference;
• multi-tool completeness;
• latency/context trade-offs.

AgentWeave repo:
https://github.com/sauravsingla/agentweave

BFCL integration PR:
ShishirPatil/gorilla#1358