SWE-Bench Pro leaderboard
As of November 8, 2026, Gemini 3.6 Flash (Google) leads SWE-Bench Pro at 58.7%, ahead of Gemini 3.5 Flash at 55.1% and Gemini 3.1 Pro Preview at 54.2%. 8 systems ranked. Every score here is published by the model's own lab; each row links to the page it comes from.
What is SWE-Bench Pro? SWE-Bench Pro tests coding agents on long, real software tasks in professional repositories, including private and copyleft code chosen to be hard to find in training data. Made by Scale AI, September 2025.
Top models on SWE-Bench Pro, November 2026
| # | Model | Score | Price / 1M | Context | Source |
|---|---|---|---|---|---|
| 1 | Gemini 3.6 Flash Google | public set | $0.75 / $3.75 | 1M | Vendor-reported Google DeepMind |
| 2 | Gemini 3.5 Flash Deprecated Google | public set | $1.5 / $9 | 1M | Vendor-reported Google DeepMind |
| 3 | Gemini 3.1 Pro Preview Preview Google · listed as “Gemini 3.1 Pro” | public set | $2 / $12 | 1M | Vendor-reported Google DeepMind |
| 3 | Gemini 3.5 Flash-Lite Google | public set | $0.3 / $2.5 | 1M | Vendor-reported Google DeepMind |
| 5 | Gemini 3 Pro Preview Retired Google · listed as “Gemini 3 Pro” | single attempt, thinking high | $2 / $12 | 1M | Vendor-reported Google DeepMind |
| 6 | Gemini 3.1 Flash-Lite Deprecated Google | public set | $0.25 / $1.5 | 1M | Vendor-reported Google DeepMind |
SWE-Bench Pro under other settings
Labs report SWE-Bench Pro under different settings, and a score from one setting is not comparable with a score from another. Each setting has its own table, ranked on its own.
Set not stated
| # | Model | Score | Price / 1M | Context | Source |
|---|---|---|---|---|---|
| 1 | Qwen3.8 Max Alibaba (Qwen) | set not stated | $2 / $6 | 1M | Vendor-reported Alibaba (Qwen) |
| 2 | Qwen3.7 Max Alibaba (Qwen) | set not stated | $2.5 / $7.5 | 1M | Vendor-reported Alibaba (Qwen) |
Can you trust SWE-Bench Pro scores?
No independent leaderboard publishes SWE-Bench Pro results for these models yet, so every number on this page is the lab's own report of its own model. Labs choose the harness, the effort level and the subset they report, and they rarely publish failures. Read the setting under each score before comparing two rows.
- Only the 731-task public set is released; the held-out and commercial sets are private, so most of the benchmark cannot be checked from outside.
- Scaffolds and turn limits vary between runs on Scale's own leaderboard, and lab-reported numbers use their own agents.
- Some labs do not say which set they ran; those rows sit in their own table.
How SWE-Bench Pro works
The task
Given a repository, a problem statement, requirements and an interface specification, the agent writes a patch for a feature or a fix; tasks can span many files.
The score
Resolve rate: a task counts only when the patch makes the failing tests pass and every test that passed before still passes.
Test set
1,865 tasks from 41 repositories; 731 in the public set, as published by the authors.
Made by
Scale AI, first published September 2025.
Official sources
- labs.scale.com/leaderboard/swe_bench_pro_public
- Paper: arxiv.org/abs/2509.16941
- Definition read on 2026-11-08
Sources and freshness
8 published scores from 4 sources. Each was read on its source page; the oldest reading on this page is from November 8, 2026. Prices beside each score come from our model price index and move when a vendor reprices.
- Vendor-reported Google DeepMind
- Vendor-reported Alibaba (Qwen)
- Vendor-reported Google DeepMind
- Vendor-reported Google DeepMind
SWE-Bench Pro FAQ
Answers derived from the 8 published scores on this page and the current price index.