Benchmark Almanac

SWE-Bench Pro

Muse Spark 1.1 61.5% leads since July 2026
Claude Fable 5.2 93% leads since November 2026
In use

1,865 long-horizon issue-resolution tasks from 41 repos in four languages. Agent patch must pass hidden tests. Pass@1.

731 public, 858 held-out and 276 commercial tasks. Copyleft repos reduce contamination. Vendor-reported scores differ from Scale runs. In 2026-06 Cursor found that 63% of successful Opus 4.8 Max solves looked up the fix. With git history sealed and no internet, that score fell from 87.1% to 73.0%.

Made by
Scale AI
Released
September 2025
Kind
Fixing real repositories
Level
Expert
Scored
Checked automatically
Answers
Code
Tasks
Public, plus a private test set
Strongest model tested
Claude Fable 5.2 #1 on the capability index
Last in a launch post
November 2026
Last model added
November 2026
Scores refreshed
November 12th, 2026

In the authors’ words

We introduce SWE-Bench Pro, a substantially more challenging benchmark that builds upon the best practices of SWE-BENCH [25], but is explicitly designed to capture realistic, complex, enterprise-level problems beyond the scope of SWE-BENCH. SWE-BENCH PRO contains 1,865 problems sourced from a diverse set of 41 actively maintained repositories spanning business applications, B2B services, and developer tools. The benchmark is partitioned into a public set with open access to problems sourced from 11 repositories, a held-out set of 12 repositories and a commercial set of 18 proprietary repositories where we have formal partnership agreements with early-stage startups. Problems in the held-out and the commercial set are not publicly accessible, but we release results on the commercial set. Our benchmark features long-horizon tasks that may require hours to days for a professional software engineer to complete, often involving patches across multiple files and substantial code modifications. All tasks are human-verified and augmented with sufficient context to ensure resolvability. To better understand these limitations, we cluster the failure modes observed in the collected agent trajectories for a clearer characterization of the error patterns exhibited by current models. Overall, SWE-BENCH PRO provides a contamination-resistant testbed that more faithfully captures the complexity and diversity of real-world software development, advancing the pursuit of truly autonomous software engineering agents at a professional level.

Abstract of SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? (arXiv:2509.16941, CC0)
25
Entries on its own leaderboard, 24 models we track
77
Launch posts list it, from 27 labs
37
System cards report it, from 15 labs

Muse Spark 1.1 leads with 61.5% among models released by July 2026

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

Claude Fable 5.2 leads with 93%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in July 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

25 entries. Scroll sideways for the rest.

  1. 61.5 Muse Spark 1.1
  2. 59.1 GPT-5.4
  3. 55.0 Muse Spark
  4. 51.9 Claude Opus 4.6
  5. 46.1 Gemini 3.1 Pro
  6. 45.9 Claude Opus 4.5
  7. 43.6 Claude Sonnet 4.5
  8. 43.3 Gemini 3 Pro
  9. 42.7 Claude Sonnet 4
  10. 41.8 GPT-5
  11. 41.0 GPT-5.2 Codex
  12. 39.5 Claude Haiku 4.5
  13. 38.7 Qwen3-Coder-480B-A35B
  14. 36.8 MiniMax-M2.1
  15. 34.6 Gemini 3 Flash
  16. 29.9 GPT-5.2
  17. 27.7 Kimi K2
  18. 21.4 Qwen3-235B-A22B
  19. 16.2 gpt-oss-120b
  20. 15.6 DeepSeek-V3.2
  21. 11.4 Gemma 3 27B
  22. 11.2 Llama 3.1-405B
  23. 9.67 GLM-4.6
  24. 5.24 Llama 4 Maverick
  25. 1.51 codestral-2405
  1. 1 Muse Spark 1.1 61.5%
  2. 2 GPT-5.4 xhigh 59.1%
  3. 3 Muse Spark 55%
  4. 4 Claude Opus 4.6 thinking 51.9%
  5. 5 Gemini 3.1 Pro thinking 46.1%
  6. 6 Claude Opus 4.5 45.9%
  7. 7 Claude Sonnet 4.5 43.6%
  8. 8 Gemini 3 Pro 43.3%
  9. 9 Claude Sonnet 4 42.7%
  10. 10 GPT-5 high 41.8%
  11. 11 GPT-5.2 Codex 41%
  12. 12 Claude Haiku 4.5 39.5%
  13. 13 Qwen3-Coder-480B-A35B 38.7%
  14. 14 MiniMax-M2.1 36.8%
  15. 15 Gemini 3 Flash 34.6%
  16. 16 GPT-5.2 29.9%
  17. 17 Kimi K2 Jul 2025 27.7%
  18. 18 Qwen3-235B-A22B 21.4%
  19. 19 gpt-oss-120b 16.2%
  20. 20 DeepSeek-V3.2 15.6%
  21. 21 Gemma 3 27B 11.4%
  22. 22 Llama 3.1-405B 11.2%
  23. 23 GLM-4.6 9.67%
  24. 24 Llama 4 Maverick 5.24%
  25. 25 codestral-2405 1.51%

Source: Official SWE-Bench Pro leaderboard

98 models. Scroll sideways for the rest.

  1. 93.0 Claude Fable 5.2
  2. 89.9 Claude Opus 5.5
  3. 81.3 Claude Sonnet 5.5
  4. 81.2 Claude Fable 5.1
  5. 80.3 Claude Fable 5
  6. 79.2 Claude Opus 5
  7. 77.8 Claude Mythos Preview
  8. 69.2 Claude Opus 4.8
  9. 68.7 Ling-3.1-flash
  10. 68.5 Intern-S2-397B
  11. 67.7 Qwen 3.8 Max
  12. 65.7 Tencent Hy4 preview
  13. 65.5 Beam 1.0
  14. 65.2 KAT-Coder-Pro V2.5
  15. 64.8 Claude Haiku 5.5
  16. 64.7 Grok 4.5
  17. 64.6 GPT-5.6 Sol
  18. 64.3 Claude Opus 4.7
  19. 63.4 GPT-5.6 Terra
  20. 63.3 Qwen3.8-Omni-Flash
  21. 63.2 Claude Sonnet 5
  22. 62.7 GPT-5.6 Luna
  23. 62.5 Qwen3.8 Flash
  24. 62.1 GLM-5.2
  25. 61.7 Qwen 3.8 27B
  26. 61.5 Muse Spark 1.1
  27. 60.6 Qwen3.7-Max
  28. 59.5 LongCat-2.0
  29. 59.4 Laguna S 2.1
  30. 59.1 GPT-5.4
  31. 59.0 MiniMax-M3
  32. 58.7 Gemini 3.6 Flash
  33. 58.6 Kimi K2.6
  34. 58.6 GPT-5.5
  35. 58.4 GLM-5.1
  36. 57.9 Tencent Hy3
  37. 57.6 Qwen3.7-Plus
  38. 57.5 Seed2.1 Pro
  39. 57.3 Qwen 3.6 Max (Preview)
  40. 57.2 MiMo-V2.5-Pro
  41. 56.8 GPT-5.3 Codex
  42. 56.6 Qwen 3.6 Plus
  43. 56.4 GPT-5.2 Codex
  44. 56.3 Step 3.7 Flash
  45. 56.2 MiniMax-M2.7
  46. 56.1 MiMo-V2.5
  47. 55.9 Inkling-Small
  48. 55.6 GPT-5.2
  49. 55.4 MiniMax-M2.5
  50. 55.4 DeepSeek-V4-Pro
  51. 55.1 Gemini 3.5 Flash
  52. 55.0 Muse Spark
  53. 54.4 GPT-5.4 Mini
  54. 54.3 Inkling
  55. 54.3 Spark-X2.6-4B
  56. 54.2 Gemini 3.1 Pro
  57. 54.2 Gemini 3.5 Flash-Lite
  58. 53.8 Ring-2.6-1T
  59. 53.5 Qwen3.6 27B
  60. 52.8 MAI-Thinking-1
  61. 52.6 DeepSeek-V4-Flash
  62. 52.4 GPT-5.4 Nano
  63. 51.9 Claude Opus 4.6
  64. 51.8 Nemotron 3.5 Super VL
  65. 51.6 Claude Opus 4.5
  66. 51.5 GPT-5.3-Codex-Spark
  67. 51.2 MAI-Code-1-Flash
  68. 51.2 Muse Glimmer
  69. 50.7 Kimi K2.5
  70. 50.4 SWE-1.6
  71. 49.5 Qwen 3.6 35B-A3B
  72. 49.2 Laguna M.1
  73. 47.6 Laguna XS 2.1
  74. 46.9 Seed2.0 Pro
  75. 46.6 Seed2.0 Lite (0428)
  76. 46.3 Laguna XS.2
  77. 46.0 KAT-Coder-V2.5-Dev
  78. 44.3 Qwen3-Coder-Next
  79. 43.6 Claude Sonnet 4.5
  80. 43.3 Gemini 3 Pro
  81. 42.7 Claude Sonnet 4
  82. 41.8 GPT-5
  83. 40.2 North Mini Code 1.0
  84. 40.1 SWE-1.5
  85. 39.5 Claude Haiku 4.5
  86. 38.7 Qwen3-Coder-480B-A35B
  87. 36.8 MiniMax-M2.1
  88. 34.6 Gemini 3 Flash
  89. 33.3 Granite 4.2 30B
  90. 28.4 Beam 1.0 Base
  91. 27.7 Kimi K2
  92. 21.4 Qwen3-235B-A22B
  93. 16.2 gpt-oss-120b
  94. 15.6 DeepSeek-V3.2
  95. 11.4 Gemma 3 27B
  96. 11.2 Llama 3.1-405B
  97. 9.67 GLM-4.6
  98. 5.24 Llama 4 Maverick
  1. 1 Claude Fable 5.2 93%
  2. 2 Claude Opus 5.5 89.9%
  3. 3 Claude Sonnet 5.5 81.3%
  4. 4 Claude Fable 5.1 81.2%
  5. 5 Claude Fable 5 80.3%
  6. 6 Claude Opus 5 79.2%
  7. 7 Claude Mythos Preview 77.8%
  8. 8 Claude Opus 4.8 69.2%
  9. 9 Ling-3.1-flash 68.7%
  10. 10 Intern-S2-397B 68.5%
  11. 11 Qwen 3.8 Max 67.7%
  12. 12 Tencent Hy4 preview 65.7%
  13. 13 Beam 1.0 65.5%
  14. 14 KAT-Coder-Pro V2.5 65.2%
  15. 15 Claude Haiku 5.5 64.8%
  16. 16 Grok 4.5 64.7%
  17. 17 GPT-5.6 Sol 64.6%
  18. 18 Claude Opus 4.7 64.3%
  19. 19 GPT-5.6 Terra 63.4%
  20. 20 Qwen3.8-Omni-Flash 63.3%
  21. 21 Claude Sonnet 5 63.2%
  22. 22 GPT-5.6 Luna 62.7%
  23. 23 Qwen3.8 Flash 62.5%
  24. 24 GLM-5.2 62.1%
  25. 25 Qwen 3.8 27B 61.7%
  26. 26 Muse Spark 1.1 61.5%
  27. 27 Qwen3.7-Max 60.6%
  28. 28 LongCat-2.0 59.5%
  29. 29 Laguna S 2.1 59.4%
  30. 30 GPT-5.4 xhigh 59.1%
  31. 31 MiniMax-M3 59%
  32. 32 Gemini 3.6 Flash 58.7%
  33. 33 Kimi K2.6 58.6%
  34. 34 GPT-5.5 58.6%
  35. 35 GLM-5.1 58.4%
  36. 36 Tencent Hy3 57.9%
  37. 37 Qwen3.7-Plus 57.6%
  38. 38 Seed2.1 Pro 57.5%
  39. 39 Qwen 3.6 Max (Preview) 57.3%
  40. 40 MiMo-V2.5-Pro 57.2%
  41. 41 GPT-5.3 Codex 56.8%
  42. 42 Qwen 3.6 Plus 56.6%
  43. 43 GPT-5.2 Codex 56.4%
  44. 44 Step 3.7 Flash 56.3%
  45. 45 MiniMax-M2.7 56.2%
  46. 46 MiMo-V2.5 56.1%
  47. 47 Inkling-Small 55.9%
  48. 48 GPT-5.2 55.6%
  49. 49 MiniMax-M2.5 55.4%
  50. 50 DeepSeek-V4-Pro 55.4%
  51. 51 Gemini 3.5 Flash 55.1%
  52. 52 Muse Spark 55%
  53. 53 GPT-5.4 Mini 54.4%
  54. 54 Inkling 54.3%
  55. 55 Spark-X2.6-4B 54.3%
  56. 56 Gemini 3.1 Pro 54.2%
  57. 57 Gemini 3.5 Flash-Lite 54.2%
  58. 58 Ring-2.6-1T 53.8%
  59. 59 Qwen3.6 27B 53.5%
  60. 60 MAI-Thinking-1 52.8%
  61. 61 DeepSeek-V4-Flash 52.6%
  62. 62 GPT-5.4 Nano 52.4%
  63. 63 Claude Opus 4.6 thinking 51.9%
  64. 64 Nemotron 3.5 Super VL 51.8%
  65. 65 Claude Opus 4.5 51.6%
  66. 66 GPT-5.3-Codex-Spark 51.5%
  67. 67 MAI-Code-1-Flash 51.2%
  68. 68 Muse Glimmer 51.2%
  69. 69 Kimi K2.5 50.7%
  70. 70 SWE-1.6 50.4%
  71. 71 Qwen 3.6 35B-A3B 49.5%
  72. 72 Laguna M.1 49.2%
  73. 73 Laguna XS 2.1 47.6%
  74. 74 Seed2.0 Pro 46.9%
  75. 75 Seed2.0 Lite (0428) 46.6%
  76. 76 Laguna XS.2 46.3%
  77. 77 KAT-Coder-V2.5-Dev 46%
  78. 78 Qwen3-Coder-Next 44.3%
  79. 79 Claude Sonnet 4.5 43.6%
  80. 80 Gemini 3 Pro 43.3%
  81. 81 Claude Sonnet 4 42.7%
  82. 82 GPT-5 high 41.8%
  83. 83 North Mini Code 1.0 40.2%
  84. 84 SWE-1.5 40.1%
  85. 85 Claude Haiku 4.5 39.5%
  86. 86 Qwen3-Coder-480B-A35B 38.7%
  87. 87 MiniMax-M2.1 36.8%
  88. 88 Gemini 3 Flash 34.6%
  89. 89 Granite 4.2 30B 33.3%
  90. 90 Beam 1.0 Base 28.4%
  91. 91 Kimi K2 Jul 2025 27.7%
  92. 92 Qwen3-235B-A22B 21.4%
  93. 93 gpt-oss-120b 16.2%
  94. 94 DeepSeek-V3.2 15.6%
  95. 95 Gemma 3 27B 11.4%
  96. 96 Llama 3.1-405B 11.2%
  97. 97 GLM-4.6 9.67%
  98. 98 Llama 4 Maverick 5.24%

Sources: Labs’ model cards and official SWE-Bench Pro leaderboard

#EntryScoreSettingDateSource
1 Muse Spark 1.1 61.5% mini-SWE-agent Jul 2026 Official leaderboard
2 GPT-5.4 59.1% xhigh, mini-SWE-agent Mar 2026 Official leaderboard
3 Muse Spark 55% mini-SWE-agent Apr 2026 Official leaderboard
4 Claude Opus 4.6 51.9% thinking, mini-SWE-agent Feb 2026 Official leaderboard
5 Gemini 3.1 Pro 46.1% thinking, mini-SWE-agent Feb 2026 Official leaderboard
6 Claude Opus 4.5 45.9% SWE-agent Nov 2025 Official leaderboard
7 Claude Sonnet 4.5 43.6% SWE-agent Sep 2025 Official leaderboard
8 Gemini 3 Pro 43.3% SWE-agent Nov 2025 Official leaderboard
9 Claude Sonnet 4 42.7% SWE-agent May 2025 Official leaderboard
10 GPT-5 41.8% high, SWE-agent Aug 2025 Official leaderboard
11 GPT-5.2 Codex 41% SWE-agent Dec 2025 Official leaderboard
12 Claude Haiku 4.5 39.5% SWE-agent Oct 2025 Official leaderboard
13 Qwen3-Coder-480B-A35B 38.7% SWE-agent Jul 2025 Official leaderboard
14 MiniMax-M2.1 36.8% SWE-agent Dec 2025 Official leaderboard
15 Gemini 3 Flash 34.6% SWE-agent Dec 2025 Official leaderboard
16 GPT-5.2 29.9% SWE-agent Dec 2025 Official leaderboard
17 Kimi K2 Jul 2025 27.7% SWE-agent Jul 2025 Official leaderboard
18 Qwen3-235B-A22B 21.4% SWE-agent Apr 2025 Official leaderboard
19 gpt-oss-120b 16.2% SWE-agent Aug 2025 Official leaderboard
20 DeepSeek-V3.2 15.6% SWE-agent Dec 2025 Official leaderboard
21 Gemma 3 27B 11.4% SWE-agent Mar 2025 Official leaderboard
22 Llama 3.1-405B 11.2% SWE-agent Jul 2024 Official leaderboard
23 GLM-4.6 9.67% SWE-agent Sep 2025 Official leaderboard
24 Llama 4 Maverick 5.24% SWE-agent Apr 2025 Official leaderboard
25 codestral-2405 1.51% SWE-agent Official leaderboard
#ModelScoreSettingDateSource
1 Claude Fable 5.2 93% Nov 2026 Lab model card
2 Claude Opus 5.5 89.9% Sep 2026 Lab model card
3 Claude Sonnet 5.5 81.3% Sep 2026 Lab model card
4 Claude Fable 5.1 81.2% Sep 2026 Lab model card
5 Claude Fable 5 80.3% Jun 2026 Lab model card
6 Claude Opus 5 79.2% Jul 2026 Lab model card
7 Claude Mythos Preview 77.8% Apr 2026 Lab model card
8 Claude Opus 4.8 69.2% May 2026 Lab model card
9 Ling-3.1-flash 68.7% Oct 2026 Lab model card
10 Intern-S2-397B 68.5% Sep 2026 Lab model card
11 Qwen 3.8 Max 67.7% Aug 2026 Lab model card
12 Tencent Hy4 preview 65.7% Aug 2026 Lab model card
13 Beam 1.0 65.5% Oct 2026 Lab model card
14 KAT-Coder-Pro V2.5 65.2% Jul 2026 Lab model card
15 Claude Haiku 5.5 64.8% Oct 2026 Lab model card
16 Grok 4.5 64.7% Jul 2026 Lab model card
17 GPT-5.6 Sol 64.6% Jul 2026 Lab model card
18 Claude Opus 4.7 64.3% Apr 2026 Lab model card
19 GPT-5.6 Terra 63.4% Jul 2026 Lab model card
20 Qwen3.8-Omni-Flash 63.3% Sep 2026 Lab model card
21 Claude Sonnet 5 63.2% Jun 2026 Lab model card
22 GPT-5.6 Luna 62.7% Jul 2026 Lab model card
23 Qwen3.8 Flash 62.5% Aug 2026 Lab model card
24 GLM-5.2 62.1% Jun 2026 Lab model card
25 Qwen 3.8 27B 61.7% Aug 2026 Lab model card
26 Muse Spark 1.1 61.5% mini-SWE-agent Jul 2026 Official leaderboard
27 Qwen3.7-Max 60.6% May 2026 Lab model card
28 LongCat-2.0 59.5% Jul 2026 Lab model card
29 Laguna S 2.1 59.4% Jul 2026 Lab model card
30 GPT-5.4 59.1% xhigh, mini-SWE-agent Mar 2026 Official leaderboard
31 MiniMax-M3 59% Jun 2026 Lab model card
32 Gemini 3.6 Flash 58.7% Jul 2026 Lab model card
33 Kimi K2.6 58.6% Apr 2026 Lab model card
34 GPT-5.5 58.6% Apr 2026 Lab model card
35 GLM-5.1 58.4% Apr 2026 Lab model card
36 Tencent Hy3 57.9% Jul 2026 Lab model card
37 Qwen3.7-Plus 57.6% Jun 2026 Lab model card
38 Seed2.1 Pro 57.5% Jun 2026 Lab model card
39 Qwen 3.6 Max (Preview) 57.3% Apr 2026 Lab model card
40 MiMo-V2.5-Pro 57.2% Apr 2026 Lab model card
41 GPT-5.3 Codex 56.8% Feb 2026 Lab model card
42 Qwen 3.6 Plus 56.6% Apr 2026 Lab model card
43 GPT-5.2 Codex 56.4% Dec 2025 Lab model card
44 Step 3.7 Flash 56.3% May 2026 Lab model card
45 MiniMax-M2.7 56.2% Mar 2026 Lab model card
46 MiMo-V2.5 56.1% Apr 2026 Lab model card
47 Inkling-Small 55.9% Jul 2026 Lab model card
48 GPT-5.2 55.6% Dec 2025 Lab model card
49 MiniMax-M2.5 55.4% Feb 2026 Lab model card
50 DeepSeek-V4-Pro 55.4% Apr 2026 Lab model card
51 Gemini 3.5 Flash 55.1% May 2026 Lab model card
52 Muse Spark 55% mini-SWE-agent Apr 2026 Official leaderboard
53 GPT-5.4 Mini 54.4% Mar 2026 Lab model card
54 Inkling 54.3% Jul 2026 Lab model card
55 Spark-X2.6-4B 54.3% Oct 2026 Lab model card
56 Gemini 3.1 Pro 54.2% Feb 2026 Lab model card
57 Gemini 3.5 Flash-Lite 54.2% Jul 2026 Lab model card
58 Ring-2.6-1T 53.8% Jun 2026 Lab model card
59 Qwen3.6 27B 53.5% Apr 2026 Lab model card
60 MAI-Thinking-1 52.8% Jun 2026 Lab model card
61 DeepSeek-V4-Flash 52.6% Apr 2026 Lab model card
62 GPT-5.4 Nano 52.4% Mar 2026 Lab model card
63 Claude Opus 4.6 51.9% thinking, mini-SWE-agent Feb 2026 Official leaderboard
64 Nemotron 3.5 Super VL 51.8% Oct 2026 Lab model card
65 Claude Opus 4.5 51.6% Nov 2025 Lab model card
66 GPT-5.3-Codex-Spark 51.5% Feb 2026 Lab model card
67 MAI-Code-1-Flash 51.2% Jun 2026 Lab model card
68 Muse Glimmer 51.2% Aug 2026 Lab model card
69 Kimi K2.5 50.7% Jan 2026 Lab model card
70 SWE-1.6 50.4% Apr 2026 Lab model card
71 Qwen 3.6 35B-A3B 49.5% Apr 2026 Lab model card
72 Laguna M.1 49.2% May 2026 Lab model card
73 Laguna XS 2.1 47.6% Jul 2026 Lab model card
74 Seed2.0 Pro 46.9% Feb 2026 Lab model card
75 Seed2.0 Lite (0428) 46.6% Apr 2026 Lab model card
76 Laguna XS.2 46.3% May 2026 Lab model card
77 KAT-Coder-V2.5-Dev 46% Jul 2026 Lab model card
78 Qwen3-Coder-Next 44.3% Feb 2026 Lab model card
79 Claude Sonnet 4.5 43.6% SWE-agent Sep 2025 Official leaderboard
80 Gemini 3 Pro 43.3% SWE-agent Nov 2025 Official leaderboard
81 Claude Sonnet 4 42.7% SWE-agent May 2025 Official leaderboard
82 GPT-5 41.8% high, SWE-agent Aug 2025 Official leaderboard
83 North Mini Code 1.0 40.2% Jun 2026 Lab model card
84 SWE-1.5 40.1% Oct 2025 Lab model card
85 Claude Haiku 4.5 39.5% SWE-agent Oct 2025 Official leaderboard
86 Qwen3-Coder-480B-A35B 38.7% SWE-agent Jul 2025 Official leaderboard
87 MiniMax-M2.1 36.8% SWE-agent Dec 2025 Official leaderboard
88 Gemini 3 Flash 34.6% SWE-agent Dec 2025 Official leaderboard
89 Granite 4.2 30B 33.3% Aug 2026 Lab model card
90 Beam 1.0 Base 28.4% Oct 2026 Lab model card
91 Kimi K2 Jul 2025 27.7% SWE-agent Jul 2025 Official leaderboard
92 Qwen3-235B-A22B 21.4% SWE-agent Apr 2025 Official leaderboard
93 gpt-oss-120b 16.2% SWE-agent Aug 2025 Official leaderboard
94 DeepSeek-V3.2 15.6% SWE-agent Dec 2025 Official leaderboard
95 Gemma 3 27B 11.4% SWE-agent Mar 2025 Official leaderboard
96 Llama 3.1-405B 11.2% SWE-agent Jul 2024 Official leaderboard
97 GLM-4.6 9.67% SWE-agent Sep 2025 Official leaderboard
98 Llama 4 Maverick 5.24% SWE-agent Apr 2025 Official leaderboard

Labs report scores for 81 models in their cards. Switch the scores above to rank them together.

6 models have held number one on the official leaderboard since its release in September 2025 12 models have held number one with claims included since its release in September 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic OpenAI Meta Cognition
Own leaderboard Replaced by V2, Sep 2026
Muse Spark 1.1 61.5%
Lab claims Passed 85% in Sep 2026, by Claude Opus 5.5
Claude Fable 5.2 93%

Sources: Official SWE-Bench Pro leaderboard and labs’ model cards

Anthropic’s own number for Claude Opus 5.5 passed 85% in September 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Claude Fable 5.2 Claude Fable 5.1 Claude Mythos Preview GPT-5.4 Claude Opus 4.5 Claude Sonnet 4.5 Qwen3-235B-A22B 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Claude Fable 5.2 Claude Fable 5.1

Sources: Official SWE-Bench Pro leaderboard and labs’ model cards

21 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Cognition
OpenAI
Moonshot AI
Qwen
MiniMax
ByteDance Seed
Google DeepMind
Anthropic
Z.ai
Meta
Xiaomi Corp
DeepSeek
Poolside
StepFun
Microsoft
Cohere
Tencent
Meituan
Kuaishou
Thinking Machines
xAI
IBM
Shanghai AI Lab
Reflection
Ant Group
NVIDIA
iFlytek
DateLaunch postScoreAs the post writes it
Nov 9, 2026 Claude Fable 5.2 93 SWE-bench Pro, adaptive thinking max effort, average of five trials (Table 8.1.A only)
Oct 22, 2026 Spark-X2.6-4B 54.3 SWE Pro
Oct 15, 2026 Nemotron 3.5 Super VL 51.8 SWE-bench Pro (OpenCode harness)
Oct 13, 2026 Ling-3.1-flash 68.7 SWE-bench Pro (LingCode)
Oct 7, 2026 Claude Haiku 5.5 64.8 SWE-Bench Pro
Oct 5, 2026 Beam 1.0 65.5 SWE Bench Pro v1
Sep 28, 2026 Claude Sonnet 5.5 81.3 SWE-Bench Pro
Sep 22, 2026 Claude Opus 5.5 89.9 SWE-bench Pro
Sep 18, 2026 Qwen3.8-Omni-Flash 63.3 SWE-bench Pro (Claude Code, Qwen-corrected tasks)
Sep 13, 2026 Intern-S2-397B 68.5 SWE-bench-Pro (Mini-SWE-Agent, anti-hack configs)
Sep 1, 2026 Claude Fable 5.1 81.2 SWE-bench Pro
Aug 28, 2026 Tencent Hy4 preview 65.7 SWE-bench Pro
Aug 26, 2026 Qwen3.8 Flash 62.5 Agentic coding SWE-bench Pro (Claude Code, Qwen-corrected tasks)
Aug 25, 2026 Granite 4.2 30B 33.3 SWE Bench Pro
Aug 14, 2026 Qwen 3.8 27B 61.7 Agentic coding SWE-bench Pro (Claude Code, Qwen-corrected tasks)
Aug 10, 2026 Muse Glimmer 51.2 SWE-Bench Pro
Aug 3, 2026 Qwen 3.8 Max 67.7 SWE-bench Pro
Jul 30, 2026 Inkling-Small 55.9 SWEBench Pro public
Jul 24, 2026 Claude Opus 5 79.2 SWE-bench Pro
Jul 23, 2026 KAT-Coder-V2.5-Dev 46 SWE-bench Pro
Jul 21, 2026 Laguna S 2.1 59.4 SWE-Bench Pro (Public Dataset)
Jul 21, 2026 Gemini 3.5 Flash-Lite 54.2 SWE-Bench Pro (Public)
Jul 21, 2026 Gemini 3.6 Flash 58.7 SWE-Bench Pro (Public)
Jul 16, 2026 Grok 4.5 64.7 SWE Bench Pro resolve rate
Jul 15, 2026 Inkling 54.3 SWEBench Pro Public
Jul 9, 2026 GPT-5.6 Luna 62.7 SWE-Bench Pro
Jul 9, 2026 GPT-5.6 Terra 63.4 SWE-Bench Pro
Jul 9, 2026 Muse Spark 1.1 61.5 SWE-Bench Pro (Diverse software engineering)
Jul 9, 2026 GPT-5.6 Sol 64.6 SWE-Bench Pro
Jul 6, 2026 KAT-Coder-Pro V2.5 65.2 SWE-Bench Pro
Jul 5, 2026 LongCat-2.0 59.5 SWE-bench Pro
Jul 2, 2026 Laguna XS 2.1 47.6 SWE-Bench Pro
Jul 2, 2026 Tencent Hy3 57.9 SWE-bench Pro
Jun 30, 2026 Claude Sonnet 5 63.2 Agentic coding SWE-bench Pro
Jun 16, 2026 GLM-5.2 62.1 SWE-bench Pro
Jun 9, 2026 North Mini Code 1.0 40.2 SWE-Bench Pro (SWE-agent v1.1.0)
Jun 9, 2026 Claude Fable 5 80.3 Agentic coding SWE-Bench Pro
Jun 2, 2026 MAI-Code-1-Flash 51.2 SWE-Bench Pro
Jun 2, 2026 MAI-Thinking-1 52.8 SWE-Bench Pro
Jun 1, 2026 Qwen3.7-Plus 57.6 SWE-Pro (Qwen-corrected tasks)
Jun 1, 2026 MiniMax-M3 59 SWE-Bench Pro
May 29, 2026 Step 3.7 Flash 56.3 SWE-Bench Pro
May 28, 2026 Claude Opus 4.8 69.2 Agentic coding SWE-Bench Pro
May 20, 2026 Qwen3.7-Max 60.6 SWE-Pro
May 19, 2026 Gemini 3.5 Flash 55.1 SWE-Bench Pro (Public) (Single attempt)
Apr 28, 2026 Laguna XS.2 44.5 SWE-Bench Pro (mean pass@1 over 3 runs)
Apr 28, 2026 Laguna M.1 46.9 SWE-Bench Pro (mean pass@1 over 3 runs)
Apr 28, 2026 Seed2.0 Lite (0428) 46.6 SWE-Bench Pro
Apr 27, 2026 MiMo-V2.5-Pro 57.2 SWE-Bench Pro
Apr 24, 2026 DeepSeek-V4-Flash 52.6 SWE Pro (Resolved), V4-Flash Max
Apr 24, 2026 DeepSeek-V4-Pro 55.4 SWE Pro (Resolved)
Apr 23, 2026 GPT-5.5 58.6 SWE-Bench Pro (Public)
Apr 22, 2026 Qwen3.6 27B 53.5 SWE-bench Pro (Qwen-corrected tasks)
Apr 22, 2026 MiMo-V2.5 56.1 SWE-Bench Pro
Apr 20, 2026 Qwen 3.6 Max (Preview) 57.3 SWE-bench Pro Agentic Coding
Apr 20, 2026 Kimi K2.6 58.6 SWE-Bench Pro
Apr 16, 2026 Claude Opus 4.7 64.3 Agentic coding SWE-bench Pro
Apr 15, 2026 Qwen 3.6 35B-A3B 49.5 SWE-bench Pro (Qwen-corrected tasks)
Apr 8, 2026 Muse Spark 52.4 SWE-Bench Pro (Diverse Agentic Coding)
Apr 7, 2026 SWE-1.6 50.4 SWE-Bench Pro (731 agentic coding tasks across 41 repos)
Apr 7, 2026 GLM-5.1 58.4 SWE-Bench Pro
Apr 7, 2026 Claude Mythos Preview 77.8 SWE-bench Pro
Apr 2, 2026 Qwen 3.6 Plus 56.6 SWE-bench Pro
Mar 18, 2026 MiniMax-M2.7 56.2 SWE Bench Pro (SWE-Pro 56.22%)
Mar 17, 2026 GPT-5.4 Nano 52.4 SWE-Bench Pro (Public)
Mar 17, 2026 GPT-5.4 Mini 54.4 SWE-Bench Pro (Public)
Mar 5, 2026 GPT-5.4 57.7 SWE-Bench Pro (Public)
Feb 19, 2026 Gemini 3.1 Pro 54.2 SWE-Bench Pro (Public) (Single attempt)
Feb 14, 2026 Seed2.0 Pro 46.9 SWE-Bench Pro
Feb 12, 2026 GPT-5.3-Codex-Spark 51.5 SWE-Bench Pro, xhigh
Feb 12, 2026 MiniMax-M2.5 55.4 SWE-Bench Pro
Feb 5, 2026 GPT-5.3 Codex 56.8 SWE-Bench Pro (Public)
Feb 2, 2026 Qwen3-Coder-Next 44.3 SWE-Bench Pro (w/ SWE-Agent)
Jan 27, 2026 Kimi K2.5 50.7 SWE-Bench Pro
Dec 18, 2025 GPT-5.2 Codex 56.4 SWE-Bench Pro
Dec 11, 2025 GPT-5.2 55.6 SWE-Bench Pro (public) Software engineering
Oct 29, 2025 SWE-1.5 40.1 SWE-Bench Pro, 731 tasks (Cascade harness)

37 system cards report it, from 15 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Nov 9, 2026 Claude Fable 5.2 system card 93 SWE-bench Pro, adaptive thinking max effort, average of five trials (Table 8.1.A and Section 8.2) Capabilities
Oct 19, 2026 Muse Glimmer technical report 51.2 SWE-Bench Pro, high, 51.6 xhigh (Tables 2 and 5) Capabilities
Oct 15, 2026 Nemotron 3.5 Super VL technical report 51.8 SWE-bench Pro, OpenCode harness (Table 10). Claude Code harness gives 49.45 (3.10) Capabilities
Oct 7, 2026 Claude Haiku 5.5 system card 64.8 SWE-Bench Pro, average of five trials, max effort (Table 8.1.A) Capabilities
Oct 5, 2026 Beam 1.0 Base technical report 28.4 SWE-Bench Pro, agentic, temperature 0.7, Table 23 Capabilities
Sep 28, 2026 Claude Sonnet 5.5 system card 81.3 SWE-Bench Pro, average of five trials (Table 8.1.A) Capabilities
Sep 22, 2026 Qwen3.8-Omni-Flash technical report 63.3 SWE-bench Pro, Claude Code harness, Qwen-corrected tasks (Table 2) Capabilities
Sep 22, 2026 Claude Opus 5.5 system card 89.9 SWE-bench Pro, average of five trials, max effort (Table 8.1.A, 8.2) Capabilities
Sep 1, 2026 Claude Fable 5.1 system card 81.2 SWE-bench Pro, average of 5 trials, adaptive thinking max effort (Table 8.1.A) Capabilities
Jul 30, 2026 Inkling-Small model card 55.9 SWEBench Pro public (5. Evaluations table) Capabilities
Jul 24, 2026 Claude Opus 5 system card 79.2 SWE-bench Pro, average of five trials, adaptive thinking at max effort (Table 8.1.A, 8.2) Capabilities
Jul 21, 2026 Gemini 3.6 Flash model card 58.7 SWE-Bench Pro (Public) (results table, p.4) Capabilities
Jul 21, 2026 Gemini 3.5 Flash-Lite model card 54.2 SWE-Bench Pro (Public) (results table, p.4) Capabilities
Jul 15, 2026 Inkling model card 54.3 SWEBench Pro Public (5. Evaluations table, effort 0.99) Capabilities
Jul 14, 2026 Grok 4.5 model card 64.7 SWE-Bench Pro, resolve rate, high, with reward-hacking controls (2.4) Capabilities
Jul 9, 2026 Muse Spark 1.1 system card 61.5 SWE-Bench Pro (Figure 44) Capabilities
Jun 30, 2026 Claude Sonnet 5 system card 63.2 SWE-bench Pro, average of five trials, adaptive thinking at max effort (Table 8.1.A, 8.2) Capabilities
Jun 23, 2026 Seed2.1 Pro model card 57.5 SWE-Pro Bench (Table 5) Capabilities
Jun 13, 2026 Ring-2.6-1T technical report 53.8 SWE-bench-Pro (Resolved), Claude Code scaffold, high, no xhigh value (Table 6) Capabilities
Jun 9, 2026 Claude Fable 5 system card 80 SWE-bench Pro, average of 5 trials, production safeguards including fallback to Opus 4.8 (Table 8.1.A, 8.2) Capabilities
Jun 2, 2026 MAI-Thinking-1 technical report 52.8 SWE-Bench Pro, bash and string-replace tools, 256k context (Table 11) Capabilities
Jun 2, 2026 MAI-Code-1-Flash model card 51.2 SWE-Bench Pro, pass rate, Copilot production harness (Coding table) Capabilities
May 28, 2026 Claude Opus 4.8 system card 69.2 SWE-bench Pro, average of five trials, max effort (Table 8.1.A, 8.2) Capabilities
May 26, 2026 MiniMax-M2.7 technical report 56.2 SWE-bench Pro, Claude Code scaffold, avg of 4 trials (Table 4) Capabilities
May 26, 2026 MiniMax-M2.5 technical report 55.4 SWE-bench Pro, Claude Code scaffold, avg of 4 trials (Table 4) Capabilities
May 25, 2026 Laguna XS.2 technical report 46.3 SWE-Bench Pro, pool harness, 500 steps, mean pass@1 over 4 runs, some task images and verifiers patched (Table 6) Capabilities
May 25, 2026 Laguna M.1 technical report 49.2 SWE-Bench Pro, pool harness, 500 steps, mean pass@1 over 4 runs, some task images and verifiers patched (Table 7) Capabilities
May 19, 2026 Gemini 3.5 Flash model card 53.9 SWE-Bench Pro (Public), single attempt (results table, p.4) Capabilities
Apr 26, 2026 DeepSeek-V4-Pro technical report 55.4 SWE Pro (Resolved), DS-V4-Pro Max (Table 6) Capabilities
Apr 26, 2026 DeepSeek-V4-Flash technical report 52.6 SWE Pro (Resolved), DeepSeek-V4-Flash Max (Table 7) Capabilities
Apr 16, 2026 Claude Opus 4.7 system card 64.3 SWE-bench Pro, average of 5 trials (Table 8.1.A, 8.2) Capabilities
Apr 7, 2026 Claude Mythos Preview system card 77.8 SWE-bench Pro, adaptive thinking, max effort, average of 5 trials (Table 6.3.A, 6.4) Capabilities
Feb 28, 2026 Qwen3-Coder-Next technical report 42.7 SWE-Bench-Pro, SWE-Agent. MiniSWE-Agent 38.7 (Table 4) Capabilities
Feb 19, 2026 Gemini 3.1 Pro model card 54.2 SWE-Bench Pro (Public), single attempt (results table, p.4) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 46.9 SWE-Bench Pro (Table 11) Capabilities
Jan 30, 2026 Kimi K2.5 technical report 50.7 SWE-Bench Pro (public) (Table 4) Capabilities
Nov 24, 2025 Claude Opus 4.5 system card 51.6 SWE-bench Pro, 64k thinking, avg of 5 trials (Table 2.4.A; 52.0% with no thinking) Capabilities

The lab's number comes out ahead in 4 of 6 pairs that differ

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
GPT-5.2 55.6% SWE-Bench Pro (public) Software engineering 29.9% Leaderboard, SWE-agent +26 pts
GPT-5.2 Codex 56.4% SWE-Bench Pro 41% Leaderboard, SWE-agent +15 pts
Gemini 3.1 Pro 54.2% SWE-Bench Pro (Public) (Single attempt) 46.1% Leaderboard, thinking, mini-SWE-agent +8.1 pts
Claude Opus 4.5 51.6% SWE-bench Pro, 64k thinking, avg of 5 trials (Table 2.4.A; 52.0% with no thinking) 45.9% Leaderboard, SWE-agent +5.7 pts
Muse Spark 52.4% SWE-Bench Pro (Diverse Agentic Coding) 55% Leaderboard, mini-SWE-agent −2.6 pts
GPT-5.4 57.7% SWE-Bench Pro (Public) 59.1% Leaderboard, xhigh, mini-SWE-agent −1.4 pts
Muse Spark 1.1 61.5% SWE-Bench Pro (Diverse software engineering) 61.5% Leaderboard, mini-SWE-agent Same

2 logged changes that move scores

Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.

  1. Grader changed February 9th, 2026

    The maintainers removed outdated unit tests, including tests requiring the year 2025, and tests included by mistake. Earlier and later verifiers can judge the same patch differently. Source

  2. Harness changed October 3rd, 2025

    The maintainers published results without the previous cap limit. Comparisons with earlier capped runs must account for the different execution budget. Source

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic OpenAI Meta
SWE-Bench Pro V2 Passed 85% in Sep 2026, by Claude Opus 5
Claude Opus 5 99.4%
SWE-Bench Pro Replaced by V2, Sep 2026
Muse Spark 1.1 61.5%
SWE-Bench Pro V2 Passed 85% in Sep 2026, by Claude Opus 5
Claude Opus 5 99.4%

Source: Official leaderboards

Replaces SWE-bench Verified Replaced by SWE-Bench Pro V2

More in Coding

All 282 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471