Performance of MiroRL-14B-SingleAgent-Preview-v0.1

[yiwan-rl]

First, great work!

I would like to understand how well the performance "40.29%(Avg@8) on the GAIA-text-103 subset" is. You mentioned that this score was obtained by fine-tuning your MiroRL-14B-SingleAgent-Preview-v0.1 model with GRPO. But you didn't mention the original model's performance.

In addition, I found that MiroThinker-14B-SFT-v0.1 obtained a 44.4 score on the same benchmark. I assume that MiroRL-14B-SingleAgent-Preview-v0.1 should achieve a score lower than 40.29. Could you explain the differences between MiroRL-14B-SingleAgent-Preview-v0.1 and MiroThinker-14B-SFT-v0.1?

Thanks

[merlinarer]

same question

[RogerChern]

It's the difference between single agent and multi-agents.

Models released under the name of MiroThinker are trained for the multi-agent setting.

In contrast, current models released under the name of MiroRL are trained for the single agent setting.

We are still working on training e2e multiple agents with RL, but in the mean while, you can already start to train your own Deep Research RL agent with the SFT cold start checkpoint for single agent.

[yiwan-rl]

It's the difference between single agent and multi-agents.

Models released under the name of MiroThinker are trained for the multi-agent setting.

In contrast, current models released under the name of MiroRL are trained for the single agent setting.

We are still working on training e2e multiple agents with RL, but in the mean while, you can already start to train your own Deep Research RL agent with the SFT cold start checkpoint for single agent.

Thanks. So, could you share the performance of the base model (MiroRL-14B-SingleAgent-Preview-v0.1)?

Two other questions to understand the performance you reported:

1. How many training steps did you perform to obtain the performance you reported?
2. I noticed that your batch size is 64 and the number of samples for each prompt is 8, do you think a larger batch size or more samples per prompt would help?