CharXiv Reasoning leaderboard
As of November 8, 2026, Kimi K3.1 Pro (Moonshot) leads CharXiv Reasoning at 89%, ahead of Gemini 3.8 Flash at 86.2% and Gemini 3.6 Flash at 85.2%. 10 systems ranked. Every score here is published by the model's own lab; each row links to the page it comes from.
What is CharXiv Reasoning? CharXiv tests chart understanding on real figures from arXiv papers; the reasoning questions need a model to compare, estimate and combine information across a chart. Made by Princeton University (Zirui Wang, Danqi Chen and co-authors), June 2024.
Top models on CharXiv Reasoning, November 2026
| # | Model | Score | Price / 1M | Context | Source |
|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro Moonshot (Kimi) | max effort, no tools | $2 / $10 | 1M | Vendor-reported Moonshot AI |
| 2 | Gemini 3.8 Flash Google DeepMind | no tools | $0.75 / $3.75 | 1M | Vendor-reported Google DeepMind |
| 3 | Gemini 3.6 Flash Google | no tools | $0.75 / $3.75 | 1M | Vendor-reported Google DeepMind |
| 4 | Kimi K3 Moonshot (Kimi) | max effort, no tools | $3 / $15 | 1M | Vendor-reported Moonshot AI |
| 5 | Gemini 3.7 Flash Deprecated Google DeepMind | no tools | $0.75 / $3.75 | 1M | Vendor-reported Google DeepMind |
| 6 | Gemini 3.5 Flash Deprecated Google | no tools | $1.5 / $9 | 1M | Vendor-reported Google DeepMind |
| 7 | Gemini 3.1 Pro Preview Preview Google · listed as “Gemini 3.1 Pro” | no tools | $2 / $12 | 1M | Vendor-reported Google DeepMind |
| 8 | Gemini 3.5 Flash-Lite Google | no tools | $0.3 / $2.5 | 1M | Vendor-reported Google DeepMind |
| 9 | Gemini 3.1 Flash-Lite Deprecated Google | no tools | $0.25 / $1.5 | 1M | Vendor-reported Google DeepMind |
CharXiv Reasoning under other settings
Labs report CharXiv Reasoning under different settings, and a score from one setting is not comparable with a score from another. Each setting has its own table, ranked on its own.
With tools
The model may run Python on the image.
| # | Model | Score | Price / 1M | Context | Source |
|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro Moonshot (Kimi) | max effort, with Python | $2 / $10 | 1M | Vendor-reported Moonshot AI |
| 2 | Kimi K3 Moonshot (Kimi) | max effort, with tools | $3 / $15 | 1M | Vendor-reported Moonshot AI |
| 3 | Gemini 3.6 Flash Google | with tools | $0.75 / $3.75 | 1M | Vendor-reported Google DeepMind |
| 4 | Gemini 3.7 Flash Deprecated Google DeepMind | with tools | $0.75 / $3.75 | 1M | Vendor-reported Google DeepMind |
| 5 | Gemini 3.5 Flash Deprecated Google | with tools | $1.5 / $9 | 1M | Vendor-reported Google DeepMind |
| 6 | Gemini 3.1 Pro Preview Preview Google · listed as “Gemini 3.1 Pro” | with tools | $2 / $12 | 1M | Vendor-reported Google DeepMind |
| 7 | Gemini 3.5 Flash-Lite Google | with tools | $0.3 / $2.5 | 1M | Vendor-reported Google DeepMind |
| 8 | Gemini 3.1 Flash-Lite Deprecated Google | with tools | $0.25 / $1.5 | 1M | Vendor-reported Google DeepMind |
Tool use not stated
| # | Model | Score | Price / 1M | Context | Source |
|---|---|---|---|---|---|
| 1 | MiniMax M3.1 Flash MiniMax | tool use not stated | $0.15 / $0.6 | 1M | Vendor-reported MiniMax |
CharXiv Reasoning: score against price
Can you trust CharXiv Reasoning scores?
No independent leaderboard publishes CharXiv Reasoning results for these models yet, so every number on this page is the lab's own report of its own model. Labs choose the harness, the effort level and the subset they report, and they rarely publish failures. Read the setting under each score before comparing two rows.
- Test-split answers are withheld, so outside evaluations use the 1,000-chart validation split; lab numbers may come from either.
- Some labs let the model run code on the chart. The official protocol is zero-shot without tools, so tool runs are a separate table here.
- Grading uses a model, not exact match.
How CharXiv Reasoning works
The task
The model sees a chart from a scientific paper and answers a hand-written question that needs reasoning over several visual elements, giving a short answer.
The score
Accuracy, with a model grader extracting the short answer and marking it right or wrong against the reference.
Test set
2,323 charts (1,000 in the public validation split), as published by the authors.
Made by
Princeton University (Zirui Wang, Danqi Chen and co-authors), first published June 2024.
Official sources
- charxiv.github.io
- Paper: arxiv.org/abs/2406.18521
- Definition read on 2026-11-08
Sources and freshness
18 published scores from 7 sources. Each was read on its source page; the oldest reading on this page is from November 8, 2026. Prices beside each score come from our model price index and move when a vendor reprices.
- Vendor-reported Google DeepMind
- Vendor-reported Google DeepMind
- Vendor-reported Google DeepMind
- Vendor-reported Google DeepMind
- Vendor-reported Moonshot AI
- Vendor-reported Moonshot AI
- Vendor-reported MiniMax
CharXiv Reasoning FAQ
Answers derived from the 18 published scores on this page and the current price index.