Benchmark Almanac

SWE-bench Verified Extended

DeepSeek V4 Pro 0813 83.3% leads since August 2026
Not in model cards

Agents resolve real GitHub issues by editing codebases, scored by repository tests. Mercor adds 50 new expert-written tasks to the 500 public ones and reports both scores.

Held-out extension of swe-bench-verified. Mercor shows the original and extended score for each model. The month is approximate. The Wayback Machine first saved the earliest extended pages on August 15th, 2026.

Made by
Mercor
Released
August 2026
Kind
Fixing real repositories
Level
Expert
Scored
Checked automatically
Answers
Code
Tasks
Public, plus a private test set
Strongest model tested
Claude Opus 5.5 #4 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
45
Entries on its own leaderboard, 44 models we track

DeepSeek V4 Pro 0813 leads with 83.3%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

45 entries. Scroll sideways for the rest.

  1. 83.3 DeepSeek V4 Pro 0813
  2. 82.7 DeepSeek V4.1 Flash
  3. 82.0 Claude Opus 5
  4. 78.7 Claude Fable 5
  5. 78.7 GPT-5.6 Terra
  6. 78.7 Kimi K3
  7. 78.0 Grok 4.6
  8. 74.7 GPT-5.6 Luna
  9. 70.7 Grok 4.5
  10. 68.0 GLM-5.3
  11. 67.3 Claude Sonnet 5
  12. 66.0 GLM-5.3-Flash
  13. 66.0 Claude Opus 5.5
  14. 63.3 Claude Opus 4.8
  15. 63.3 GPT-5.6 Sol
  16. 61.3 Claude Fable 5.1
  17. 61.3 MiMo-V2.6 Pro
  18. 59.0 Muse Wonder
  19. 58.7 Qwen 3.8 Max
  20. 58.0 Claude Opus 4.7
  21. 57.3 Claude Sonnet 5.5
  22. 57.0 Muse Spark Open
  23. 55.3 Grok 4.7
  24. 54.0 Gemini 3.8 Flash
  25. 54.0 Muse Spark 1.3
  26. 53.3 Claude Haiku 5.5
  27. 47.3 GLM-5.2
  28. 45.3 GPT-6.1 Sol
  29. 40.0 GPT-5.4
  30. 39.3 Beam
  31. 39.3 GPT-5.5
  32. 39.3 GPT-6 Luna
  33. 39.3 GPT-6 Sol
  34. 33.3 Gemini 3.6 Flash
  35. 30.0 Gemini 3.7 Flash
  36. 28.7 MiniMax-M3
  37. 27.3 Claude Sonnet 4.6
  38. 26.7 Kimi K2.7 Code
  39. 26.7 Inkling
  40. 22.7 Gemini 3.5 Flash
  41. 22.0 DeepSeek-V4-Pro
  42. 21.3 Gemini 3.1 Pro
  43. 10.0 DeepSeek-V3.2
  44. 8.70 Nemotron 3 Ultra
  45. 6.70 MiniMax-M2.7
  1. 1 DeepSeek V4 Pro 0813 max 83.3%
  2. 2 DeepSeek V4.1 Flash max 82.7%
  3. 3 Claude Opus 5 max 82%
  4. 4 Claude Fable 5 max 78.7%
  5. 5 GPT-5.6 Terra max 78.7%
  6. 6 Kimi K3 max 78.7%
  7. 7 Grok 4.6 xhigh 78%
  8. 8 GPT-5.6 Luna max 74.7%
  9. 9 Grok 4.5 high 70.7%
  10. 10 GLM-5.3 max 68%
  11. 11 Claude Sonnet 5 max 67.3%
  12. 12 GLM-5.3-Flash max 66%
  13. 13 Claude Opus 5.5 max 66%
  14. 14 Claude Opus 4.8 max 63.3%
  15. 15 GPT-5.6 Sol max 63.3%
  16. 16 Claude Fable 5.1 max 61.3%
  17. 17 MiMo-V2.6 Pro auto 61.3%
  18. 18 Muse Wonder max 59%
  19. 19 Qwen 3.8 Max xhigh 58.7%
  20. 20 Claude Opus 4.7 max 58%
  21. 21 Claude Sonnet 5.5 max 57.3%
  22. 22 Muse Spark Open xhigh 57%
  23. 23 Grok 4.7 xhigh 55.3%
  24. 24 Gemini 3.8 Flash high 54%
  25. 25 Muse Spark 1.3 xhigh 54%
  26. 26 Claude Haiku 5.5 max 53.3%
  27. 27 GLM-5.2 max 47.3%
  28. 28 GPT-6.1 Sol max 45.3%
  29. 29 GPT-5.4 xhigh 40%
  30. 30 Beam max 39.3%
  31. 31 GPT-5.5 xhigh 39.3%
  32. 32 GPT-6 Luna max 39.3%
  33. 33 GPT-6 Sol max 39.3%
  34. 34 Gemini 3.6 Flash high 33.3%
  35. 35 Gemini 3.7 Flash high 30%
  36. 36 MiniMax-M3 high 28.7%
  37. 37 Claude Sonnet 4.6 high 27.3%
  38. 38 Kimi K2.7 Code high 26.7%
  39. 39 Inkling high 26.7%
  40. 40 Gemini 3.5 Flash high 22.7%
  41. 41 DeepSeek-V4-Pro max 22%
  42. 42 Gemini 3.1 Pro high 21.3%
  43. 43 DeepSeek-V3.2 10%
  44. 44 Nemotron 3 Ultra high 8.70%
  45. 45 MiniMax-M2.7 high 6.70%

Source: Official SWE-bench Verified Extended leaderboard

#EntryScoreSettingDateSource
1 DeepSeek V4 Pro 0813 83.3% max Aug 2026 Official leaderboard
2 DeepSeek V4.1 Flash 82.7% max Sep 2026 Official leaderboard
3 Claude Opus 5 82% max Jul 2026 Official leaderboard
4 Claude Fable 5 78.7% max Jun 2026 Official leaderboard
5 GPT-5.6 Terra 78.7% max Jul 2026 Official leaderboard
6 Kimi K3 78.7% max Jul 2026 Official leaderboard
7 Grok 4.6 78% xhigh Aug 2026 Official leaderboard
8 GPT-5.6 Luna 74.7% max Jul 2026 Official leaderboard
9 Grok 4.5 70.7% high Jul 2026 Official leaderboard
10 GLM-5.3 68% max Aug 2026 Official leaderboard
11 Claude Sonnet 5 67.3% max Jun 2026 Official leaderboard
12 GLM-5.3-Flash 66% max Aug 2026 Official leaderboard
13 Claude Opus 5.5 66% max Sep 2026 Official leaderboard
14 Claude Opus 4.8 63.3% max May 2026 Official leaderboard
15 GPT-5.6 Sol 63.3% max Jul 2026 Official leaderboard
16 Claude Fable 5.1 61.3% max Sep 2026 Official leaderboard
17 MiMo-V2.6 Pro 61.3% auto Sep 2026 Official leaderboard
18 Muse Wonder 59% max Oct 2026 Official leaderboard
19 Qwen 3.8 Max 58.7% xhigh Aug 2026 Official leaderboard
20 Claude Opus 4.7 58% max Apr 2026 Official leaderboard
21 Claude Sonnet 5.5 57.3% max Sep 2026 Official leaderboard
22 Muse Spark Open 57% xhigh Oct 2026 Official leaderboard
23 Grok 4.7 55.3% xhigh Sep 2026 Official leaderboard
24 Gemini 3.8 Flash 54% high Sep 2026 Official leaderboard
25 Muse Spark 1.3 54% xhigh Sep 2026 Official leaderboard
26 Claude Haiku 5.5 53.3% max Oct 2026 Official leaderboard
27 GLM-5.2 47.3% max Jun 2026 Official leaderboard
28 GPT-6.1 Sol 45.3% max Sep 2026 Official leaderboard
29 GPT-5.4 40% xhigh Mar 2026 Official leaderboard
30 Beam 39.3% max Official leaderboard
31 GPT-5.5 39.3% xhigh Apr 2026 Official leaderboard
32 GPT-6 Luna 39.3% max Sep 2026 Official leaderboard
33 GPT-6 Sol 39.3% max Sep 2026 Official leaderboard
34 Gemini 3.6 Flash 33.3% high Jul 2026 Official leaderboard
35 Gemini 3.7 Flash 30% high Aug 2026 Official leaderboard
36 MiniMax-M3 28.7% high Jun 2026 Official leaderboard
37 Claude Sonnet 4.6 27.3% high Feb 2026 Official leaderboard
38 Kimi K2.7 Code 26.7% high Jun 2026 Official leaderboard
39 Inkling 26.7% high Jul 2026 Official leaderboard
40 Gemini 3.5 Flash 22.7% high May 2026 Official leaderboard
41 DeepSeek-V4-Pro 22% max Apr 2026 Official leaderboard
42 Gemini 3.1 Pro 21.3% high Feb 2026 Official leaderboard
43 DeepSeek-V3.2 10% Dec 2025 Official leaderboard
44 Nemotron 3 Ultra 8.70% high Jun 2026 Official leaderboard
45 MiniMax-M2.7 6.70% high Mar 2026 Official leaderboard

2 models have held number one on the official leaderboard since its release in August 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic DeepSeek
Own leaderboard 2 leaders
DeepSeek V4 Pro 0813 83.3%

Source: Official SWE-bench Verified Extended leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) DeepSeek V4 Pro 0813 Claude Fable 5 Claude Opus 4.8 Claude Opus 4.7 GPT-5.4 Claude Sonnet 4.6 DeepSeek-V3.2 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) DeepSeek V4 Pro 0813 Claude Opus 4.8

Source: Official SWE-bench Verified Extended leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Coding

All 282 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471