Toolathlon leaderboard
As of November 8, 2026, MiMo-V2.6-Pro (Xiaomi MiMo) leads Toolathlon at 76.9%, ahead of Kimi K3 at 76.5% and DeepSeek V4 Pro at 74.1%. 9 systems ranked. Every score here is published by the model's own lab; each row links to the page it comes from.
What is Toolathlon? Toolathlon (the Tool Decathlon) tests language agents on long workflows across real software through MCP tools: calendars and Notion, but also WooCommerce, Kubernetes and BigQuery. Made by HKUST NLP group and co-authors, October 2025 (Toolathlon-Verified in June 2026).
Top models on Toolathlon, November 2026
| # | Model | Score | Price / 1M | Context | Source |
|---|---|---|---|---|---|
| 1 | MiMo-V2.6-Pro Xiaomi MiMo | as reported | $0.435 / $0.87 | 1M | Vendor-reported Xiaomi |
| 2 | Kimi K3 Moonshot (Kimi) | max effort | $3 / $15 | 1M | Vendor-reported Moonshot AI |
| 3 | DeepSeek V4 Pro DeepSeek · listed as “DeepSeek V4 Pro (0813)” | as reported | $1.32 / $3.96 | 1M | Vendor-reported DeepSeek |
| 4 | MiMo-V2.6-Flash Xiaomi MiMo | as reported | $0.14 / $0.28 | 1M | Vendor-reported Xiaomi |
| 5 | GLM-5.3 Zhipu (Z.ai / GLM) | as reported | $1.4 / $4.4 | 1M | Vendor-reported Z.ai (Zhipu) |
| 6 | Qwen3.8 Max Alibaba (Qwen) | pass@1 | $2 / $6 | 1M | Vendor-reported Alibaba (Qwen) |
| 7 | DeepSeek V4 Flash Deprecated DeepSeek · listed as “DeepSeek V4 Flash (0731)” | as reported | $0.44 / $1.32 | 1M | Vendor-reported DeepSeek |
| 8 | GLM-5.2 Zhipu (Z.ai / GLM) | as reported | $1.4 / $4.4 | 1M | Vendor-reported Z.ai (Zhipu) |
| 9 | Qwen3.7 Max Alibaba (Qwen) | pass@1 | $2.5 / $7.5 | 1M | Vendor-reported Alibaba (Qwen) |
Toolathlon: score against price
Can you trust Toolathlon scores?
No independent leaderboard publishes Toolathlon results for these models yet, so every number on this page is the lab's own report of its own model. Labs choose the harness, the effort level and the subset they report, and they rarely publish failures. Read the setting under each score before comparing two rows.
- Toolathlon-Verified repaired graders, reference answers or infrastructure in 83 of the 108 tasks; the authors say its scores are not comparable with earlier ones.
- The apps are live and stateful, so environment state and latency can move a result.
- Leaderboard entries list the agent scaffold per row; not every submission uses the same one.
How Toolathlon works
The task
Each task is a multi-app workflow that starts from a realistic, pre-filled environment and runs for about 20 turns of tool calls.
The score
Every task has its own execution-based checking script, with no model judge. The leaderboard reports pass@1 averaged over runs.
Test set
108 tasks over 32 applications and 604 tools, as published by the authors.
Made by
HKUST NLP group and co-authors, first published October 2025 (Toolathlon-Verified in June 2026).
Official sources
- toolathlon.xyz
- Paper: arxiv.org/abs/2510.25726
- Definition read on 2026-11-08
Sources and freshness
9 published scores from 5 sources. Each was read on its source page; the oldest reading on this page is from November 8, 2026. Prices beside each score come from our model price index and move when a vendor reprices.
- Vendor-reported Alibaba (Qwen)
- Vendor-reported DeepSeek
- Vendor-reported Xiaomi
- Vendor-reported Z.ai (Zhipu)
- Vendor-reported Moonshot AI
Toolathlon FAQ
Answers derived from the 9 published scores on this page and the current price index.