ProgramBench leaderboard
As of November 8, 2026, Kimi K3.1 Pro (Moonshot) leads ProgramBench at 31.5%, ahead of DeepSeek V4.1 Flash at 29.1% and MiMo-V2.6-Pro at 26.5%. 7 systems ranked. Every score here is published by the model's own lab; each row links to the page it comes from.
What is ProgramBench? ProgramBench asks whether a model can rebuild a program from scratch, from small command-line tools up to FFmpeg, SQLite and the PHP interpreter, given only the program and its documentation. Made by Meta, Stanford University and Harvard University (John Yang, Kilian Lieret, Ofir Press and co-authors), May 2026.
Top models on ProgramBench, November 2026
| # | Model | Score | Price / 1M | Context | Source |
|---|---|---|---|---|---|
| 1 | Kimi K3.1 Pro Moonshot (Kimi) | max effort | $2 / $10 | 1M | Vendor-reported Moonshot AI |
| 2 | DeepSeek V4.1 Flash DeepSeek · listed as “DeepSeek V4.1 Flash (1017)” | as reported | $0.3 / $1.2 | 1M | Vendor-reported DeepSeek |
| 3 | MiMo-V2.6-Pro Xiaomi MiMo | as reported | $0.435 / $0.87 | 1M | Vendor-reported Xiaomi |
| 4 | MiMo-V2.6-Flash Xiaomi MiMo | as reported | $0.14 / $0.28 | 1M | Vendor-reported Xiaomi |
| 5 | GLM-5.3 Zhipu (Z.ai / GLM) | almost-solved rate | $1.4 / $4.4 | 1M | Vendor-reported Z.ai (Zhipu) |
| 6 | DeepSeek V4 Pro DeepSeek | almost@1 | $1.32 / $3.96 | 1M | Vendor-reported DeepSeek |
| 7 | GLM-5.2 Zhipu (Z.ai / GLM) | almost-solved rate | $1.4 / $4.4 | 1M | Vendor-reported Z.ai (Zhipu) |
ProgramBench: score against price
Can you trust ProgramBench scores?
No independent leaderboard publishes ProgramBench results for these models yet, so every number on this page is the lab's own report of its own model. Labs choose the harness, the effort level and the subset they report, and they rarely publish failures. Read the setting under each score before comparing two rows.
- The tests are a finite, fuzzing-generated set, so passing them is a lower bound on correctness.
- Only input-output behaviour is tested, not speed or memory.
- The official leaderboard runs every model in one minimal agent with step and time limits; lab-reported numbers may use other setups.
How ProgramBench works
The task
The agent gets a reference executable and its usage docs, with no internet, and has to write a full codebase and build script that reproduce the program's observable behaviour.
The score
A task is resolved when every hidden behavioural test passes and the run is not flagged for cheating. A looser 'almost' rate counts tasks with at least 95% of tests passing.
Test set
200 tasks, as published by the authors.
Made by
Meta, Stanford University and Harvard University (John Yang, Kilian Lieret, Ofir Press and co-authors), first published May 2026.
Official sources
- programbench.com
- Paper: arxiv.org/abs/2605.03546
- Definition read on 2026-11-08
Sources and freshness
8 published scores from 5 sources. Each was read on its source page; the oldest reading on this page is from November 8, 2026. Prices beside each score come from our model price index and move when a vendor reprices.
- Vendor-reported DeepSeek
- Vendor-reported Xiaomi
- Vendor-reported Z.ai (Zhipu)
- Vendor-reported DeepSeek
- Vendor-reported Moonshot AI
ProgramBench FAQ
Answers derived from the 8 published scores on this page and the current price index.