DeepSWE leaderboard
As of November 8, 2026, Gemini 4 Argon (Google DeepMind) leads DeepSWE at 77.9%, ahead of DeepSeek V4.1 Flash at 77% and Kimi K3.1 Pro at 75.5%. 27 systems ranked. Every score here is published by the model's own lab; each row links to the page it comes from.
What is DeepSWE? DeepSWE tests coding agents on new features and behaviour changes in real open-source repositories, written as short behaviour-focused requests that usually touch several files. Made by Datacurve, May 2026 (v1.1 in July 2026).
Top models on DeepSWE, November 2026
| # | Model | Score | $ / task | Price / 1M | Context | Source |
|---|---|---|---|---|---|---|
| 1 | Gemini 4 Argon Google DeepMind | as reported | — | $2 / $10 | 1M | Vendor-reported Google DeepMind |
| 2 | DeepSeek V4.1 Flash DeepSeek · listed as “DeepSeek V4.1 Flash (1017)” | as reported | — | $0.3 / $1.2 | 1M | Vendor-reported DeepSeek |
| 3 | Kimi K3.1 Pro Moonshot (Kimi) | max effort | — | $2 / $10 | 1M | Vendor-reported Moonshot AI |
| 4 | GPT-6.1 Sol OpenAI | high effort | $0.65 | $2 / $10 | 1.05M | Vendor-reported OpenAI |
| 5 | GPT-6 Astra OpenAI | xhigh effort | $4.43 | $10 / $50 | 1.05M | Vendor-reported OpenAI |
| 6 | Gemini 3.8 Flash Google DeepMind | as reported | — | $0.75 / $3.75 | 1M | Vendor-reported Google DeepMind |
| 6 | MiniMax M3.1 Flash MiniMax | as reported | — | $0.15 / $0.6 | 1M | Vendor-reported MiniMax |
| 8 | GPT-5.6 Sol OpenAI | max effort | $6.46 | $4 / $20 | 1.05M | Vendor-reported OpenAI |
| 9 | MiMo-V2.6-Pro Xiaomi MiMo | as reported | — | $0.435 / $0.87 | 1M | Vendor-reported Xiaomi |
| 10 | Grok 4.7 SpaceXSI | high effort | — | $2 / $6 | 500K | Vendor-reported SpaceXSI (xAI) |
| 11 | GPT-6 Sol OpenAI | max effort | $2.74 | $2 / $10 | 1.05M | Vendor-reported OpenAI |
| 12 | MiMo-V2.6-Flash Xiaomi MiMo | as reported | — | $0.14 / $0.28 | 1M | Vendor-reported Xiaomi |
| 13 | Kimi K3 Moonshot (Kimi) | max effort | — | $3 / $15 | 1M | Vendor-reported Moonshot AI |
| 14 | GLM-5.3 Zhipu (Z.ai / GLM) | as reported | — | $1.4 / $4.4 | 1M | Vendor-reported Z.ai (Zhipu) |
| 15 | GPT-6 Luna OpenAI | max effort | $0.22 | $0.1 / $0.5 | 1.05M | Vendor-reported OpenAI |
| 16 | Grok 4.6 SpaceXSI | high effort | — | $2 / $6 | 500K | Vendor-reported SpaceXSI (xAI) |
| 17 | Gemini 3.7 Flash Deprecated Google DeepMind | as reported | — | $0.75 / $3.75 | 1M | Vendor-reported Google DeepMind |
| 18 | DeepSeek V4 Pro DeepSeek | resolved rate | — | $1.32 / $3.96 | 1M | Vendor-reported DeepSeek |
| 19 | GPT-5.6 Luna OpenAI | max effort | $0.53 | $0.2 / $1.2 | 1.05M | Vendor-reported OpenAI |
| 20 | Qwen3.8 Max Alibaba (Qwen) | as reported | — | $2 / $6 | 1M | Vendor-reported Alibaba (Qwen) |
| 21 | DeepSeek V4 Flash Deprecated DeepSeek | resolved rate | — | $0.44 / $1.32 | 1M | Vendor-reported DeepSeek |
| 22 | Grok 4.5 SpaceXSI | high effort | — | $2 / $6 | 500K | Vendor-reported SpaceXSI (xAI) |
| 23 | Gemini 3.6 Flash Google | as reported | — | $0.75 / $3.75 | 1M | Vendor-reported Google DeepMind |
| 24 | GLM-5.2 Zhipu (Z.ai / GLM) | as reported | — | $1.4 / $4.4 | 1M | Vendor-reported Z.ai (Zhipu) |
| 25 | Gemini 3.5 Flash Deprecated Google | as reported | — | $1.5 / $9 | 1M | Vendor-reported Google DeepMind |
| 26 | Qwen3.7 Max Alibaba (Qwen) | as reported | — | $2.5 / $7.5 | 1M | Vendor-reported Alibaba (Qwen) |
| 27 | Gemini 3.1 Pro Preview Preview Google · listed as “Gemini 3.1 Pro” | as reported | — | $2 / $12 | 1M | Vendor-reported Google DeepMind |
DeepSWE: score against price
What more reasoning buys
Some publishers ran the same model at several reasoning-effort levels and printed the score and the cost per task for each. Higher effort spends more tokens per task; this is what it bought on DeepSWE.
| Model | Score by effort level ($ per task) | Source |
|---|---|---|
| GPT-6.1 Sol | low 64.4% $0.17 medium 73% $0.42 high 75.2% $0.65 xhigh 71.9% $0.79 max 71.9% $1.57 | OpenAI |
| GPT-6 Astra | low 67% $1.60 medium 72.8% $3.08 high 73.2% $3.92 xhigh 74.1% $4.43 max 73.2% $7.50 | OpenAI |
| GPT-5.6 Sol | low 45.4% $0.82 medium 61.1% $1.42 high 69.4% $2.66 xhigh 70.7% $3.60 max 72.7% $6.46 | OpenAI |
| GPT-6 Sol | low 37.2% $0.16 medium 56.6% $0.38 high 65.3% $0.64 xhigh 66.6% $1.00 max 68.8% $2.74 | OpenAI |
| GPT-6 Luna | low 2.4% $0.0057 medium 44.5% $0.052 high 59.3% $0.084 xhigh 61.3% $0.11 max 66.6% $0.22 | OpenAI |
| GPT-5.6 Luna | low 1.2% $0.011 medium 9.3% $0.031 high 42.4% $0.13 xhigh 56.2% $0.27 max 62.2% $0.53 | OpenAI |
Can you trust DeepSWE scores?
No independent leaderboard publishes DeepSWE results for these models yet, so every number on this page is the lab's own report of its own model. Labs choose the harness, the effort level and the subset they report, and they rarely publish failures. Read the setting under each score before comparing two rows.
- Every model runs in the same minimal agent with one bash tool, which can keep a model below what it scores in its own coding agent.
- Repositories are open-source and mostly TypeScript, Go and Python, so results may not carry over to private codebases.
- v1.1 changed how patches are verified, so v1 and v1.1 scores are kept apart.
How DeepSWE works
The task
The agent gets a short prompt describing a feature or change in one of 91 open-source repositories in five languages and has to work out where and how to implement it.
The score
The agent's committed patch is applied in a fresh container and hand-written behavioural tests decide pass or fail. The score is the share of tasks resolved on one attempt.
Test set
113 tasks across 91 repositories, as published by the authors.
Made by
Datacurve, first published May 2026 (v1.1 in July 2026).
Official sources
- deepswe.datacurve.ai
- Definition read on 2026-11-08
Sources and freshness
28 published scores from 16 sources. Each was read on its source page; the oldest reading on this page is from November 8, 2026. Prices beside each score come from our model price index and move when a vendor reprices.
- Vendor-reported OpenAI
- Vendor-reported DeepSeek
- Vendor-reported Alibaba (Qwen)
- Vendor-reported Google DeepMind
- Vendor-reported Google DeepMind
- Vendor-reported SpaceXSI (xAI)
- Vendor-reported Xiaomi
- Vendor-reported Z.ai (Zhipu)
- Vendor-reported DeepSeek
- Vendor-reported Google DeepMind
- Vendor-reported Google DeepMind
- Vendor-reported MiniMax
- Vendor-reported Moonshot AI
- Vendor-reported Moonshot AI
- Vendor-reported OpenAI
- Vendor-reported SpaceXSI (xAI)
DeepSWE FAQ
Answers derived from the 28 published scores on this page and the current price index.