Benchmark Almanac

SWE-bench Multilingual

Claude Fable 5.2 96.6% leads since November 2026
Gemini 3 Flash 72.7% leads since December 2025
Claude Fable 5.2 96.6% leads since November 2026
In use

300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests.

Released with SWE-smith. Paper link is the SWE-smith paper.

Released
April 2025
Kind
Fixing real repositories
Level
Expert
Scored
Checked automatically
Answers
Code
Tasks
All public
Strongest model tested
Claude Fable 5.2 #1 on the capability index
Last in a launch post
November 2026
Last model added
November 2026
Scores refreshed
November 12th, 2026
73
Models scored in lab cards
66
Launch posts list it, from 25 labs
38
System cards report it, from 15 labs

Claude Fable 5.2 leads with 96.6%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Gemini 3 Flash leads with 72.7% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Claude Fable 5.2 leads with 96.6%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

73 models. Scroll sideways for the rest.

  1. 96.6 Claude Fable 5.2
  2. 93.9 Claude Opus 5.5
  3. 90.3 Claude Sonnet 5.5
  4. 89.5 Claude Opus 5
  5. 89.1 Claude Fable 5.1
  6. 88.5 Ling-3.1-flash
  7. 87.3 Claude Mythos Preview
  8. 84.4 Claude Opus 4.8
  9. 84.0 Intern-S2-397B
  10. 83.7 Claude Haiku 5.5
  11. 82.9 Tencent Hy4 preview
  12. 81.2 Inkling 2 Small
  13. 81.0 Qwen3.8 Flash
  14. 80.5 Claude Opus 4.7
  15. 80.5 Qwen3.8-Omni-Flash
  16. 79.8 Composer 2.5
  17. 78.5 Laguna S 2.1
  18. 78.3 Qwen3.7-Max
  19. 78.3 Claude Sonnet 5
  20. 78.0 Grok 4.5
  21. 78.0 Beam 1.0
  22. 77.8 Claude Opus 4.6
  23. 77.8 SWE-1.7
  24. 77.3 LongCat-2.0
  25. 76.7 Kimi K2.6
  26. 76.5 MiniMax-M2.7
  27. 76.2 Claude Opus 4.5
  28. 76.2 DeepSeek-V4-Pro
  29. 75.9 Claude Sonnet 4.6
  30. 75.8 Qwen3.7-Plus
  31. 75.8 Tencent Hy3
  32. 74.1 MiniMax-M2.5
  33. 73.8 Qwen 3.6 Plus
  34. 73.7 Composer 2
  35. 73.3 GLM-5
  36. 73.3 DeepSeek-V4-Flash
  37. 73.0 Kimi K2.5
  38. 72.5 MiniMax-M2.1
  39. 72.4 Step 3.7 Flash
  40. 71.7 MiMo-V2-Flash
  41. 71.7 Seed2.0 Pro
  42. 71.7 MiMo-V2-Pro
  43. 71.3 Qwen3.6 27B
  44. 70.2 DeepSeek-V3.2
  45. 69.3 Qwen3.5 397B-A17B
  46. 67.7 Nemotron 3.5 Super VL
  47. 67.7 Nemotron 3 Ultra
  48. 67.4 Step 3.5 Flash
  49. 67.3 Laguna M.1
  50. 67.2 Qwen 3.6 35B-A3B
  51. 66.7 GLM-4.7
  52. 66.6 Seed2.0 Lite (0428)
  53. 65.5 MAI-Code-1-Flash
  54. 64.7 Spark-X2.6-4B
  55. 63.1 Laguna XS 2.1
  56. 63.0 KAT-Coder-V2.5-Dev
  57. 62.8 Qwen3-Coder-Next
  58. 62.4 Laguna XS.2
  59. 61.3 Devstral 2
  60. 61.1 Kimi K2 Thinking
  61. 57.9 DeepSeek-V3.2-Exp
  62. 57.8 DeepSeek-V3.1-Terminus
  63. 56.5 MiniMax-M2
  64. 55.9 Kimi K2
  65. 55.7 Devstral Small 2
  66. 54.7 Qwen3-Coder-480B-A35B
  67. 54.5 DeepSeek-V3.1
  68. 47.3 Kimi K2
  69. 45.8 Nemotron 3 Super
  70. 41.9 Granite 4.2 30B
  71. 39.3 Nemotron 3.5 Lightning 30B-A3B
  72. 32.1 Beam 1.0 Base
  73. 23.0 LFM3-1.7B
  1. 1 Claude Fable 5.2 96.6%
  2. 2 Claude Opus 5.5 93.9%
  3. 3 Claude Sonnet 5.5 90.3%
  4. 4 Claude Opus 5 89.5%
  5. 5 Claude Fable 5.1 89.1%
  6. 6 Ling-3.1-flash 88.5%
  7. 7 Claude Mythos Preview 87.3%
  8. 8 Claude Opus 4.8 84.4%
  9. 9 Intern-S2-397B 84%
  10. 10 Claude Haiku 5.5 83.7%
  11. 11 Tencent Hy4 preview 82.9%
  12. 12 Inkling 2 Small 81.2%
  13. 13 Qwen3.8 Flash 81%
  14. 14 Claude Opus 4.7 80.5%
  15. 15 Qwen3.8-Omni-Flash 80.5%
  16. 16 Composer 2.5 79.8%
  17. 17 Laguna S 2.1 78.5%
  18. 18 Qwen3.7-Max 78.3%
  19. 19 Claude Sonnet 5 78.3%
  20. 20 Grok 4.5 78%
  21. 21 Beam 1.0 78%
  22. 22 Claude Opus 4.6 77.8%
  23. 23 SWE-1.7 77.8%
  24. 24 LongCat-2.0 77.3%
  25. 25 Kimi K2.6 76.7%
  26. 26 MiniMax-M2.7 76.5%
  27. 27 Claude Opus 4.5 76.2%
  28. 28 DeepSeek-V4-Pro 76.2%
  29. 29 Claude Sonnet 4.6 75.9%
  30. 30 Qwen3.7-Plus 75.8%
  31. 31 Tencent Hy3 75.8%
  32. 32 MiniMax-M2.5 74.1%
  33. 33 Qwen 3.6 Plus 73.8%
  34. 34 Composer 2 73.7%
  35. 35 GLM-5 73.3%
  36. 36 DeepSeek-V4-Flash 73.3%
  37. 37 Kimi K2.5 73%
  38. 38 MiniMax-M2.1 72.5%
  39. 39 Step 3.7 Flash 72.4%
  40. 40 MiMo-V2-Flash 71.7%
  41. 41 Seed2.0 Pro 71.7%
  42. 42 MiMo-V2-Pro 71.7%
  43. 43 Qwen3.6 27B 71.3%
  44. 44 DeepSeek-V3.2 70.2%
  45. 45 Qwen3.5 397B-A17B 69.3%
  46. 46 Nemotron 3.5 Super VL 67.7%
  47. 47 Nemotron 3 Ultra 67.7%
  48. 48 Step 3.5 Flash 67.4%
  49. 49 Laguna M.1 67.3%
  50. 50 Qwen 3.6 35B-A3B 67.2%
  51. 51 GLM-4.7 66.7%
  52. 52 Seed2.0 Lite (0428) 66.6%
  53. 53 MAI-Code-1-Flash 65.5%
  54. 54 Spark-X2.6-4B 64.7%
  55. 55 Laguna XS 2.1 63.1%
  56. 56 KAT-Coder-V2.5-Dev 63%
  57. 57 Qwen3-Coder-Next 62.8%
  58. 58 Laguna XS.2 62.4%
  59. 59 Devstral 2 61.3%
  60. 60 Kimi K2 Thinking 61.1%
  61. 61 DeepSeek-V3.2-Exp 57.9%
  62. 62 DeepSeek-V3.1-Terminus 57.8%
  63. 63 MiniMax-M2 56.5%
  64. 64 Kimi K2 Sep 2025 55.9%
  65. 65 Devstral Small 2 55.7%
  66. 66 Qwen3-Coder-480B-A35B 54.7%
  67. 67 DeepSeek-V3.1 54.5%
  68. 68 Kimi K2 Jul 2025 47.3%
  69. 69 Nemotron 3 Super 45.8%
  70. 70 Granite 4.2 30B 41.9%
  71. 71 Nemotron 3.5 Lightning 30B-A3B 39.3%
  72. 72 Beam 1.0 Base 32.1%
  73. 73 LFM3-1.7B 23%

Source: Labs’ model cards

Source: Official SWE-bench Multilingual leaderboard

80 models. Scroll sideways for the rest.

  1. 96.6 Claude Fable 5.2
  2. 93.9 Claude Opus 5.5
  3. 90.3 Claude Sonnet 5.5
  4. 89.5 Claude Opus 5
  5. 89.1 Claude Fable 5.1
  6. 88.5 Ling-3.1-flash
  7. 87.3 Claude Mythos Preview
  8. 84.4 Claude Opus 4.8
  9. 84.0 Intern-S2-397B
  10. 83.7 Claude Haiku 5.5
  11. 82.9 Tencent Hy4 preview
  12. 81.2 Inkling 2 Small
  13. 81.0 Qwen3.8 Flash
  14. 80.5 Claude Opus 4.7
  15. 80.5 Qwen3.8-Omni-Flash
  16. 79.8 Composer 2.5
  17. 78.5 Laguna S 2.1
  18. 78.3 Qwen3.7-Max
  19. 78.3 Claude Sonnet 5
  20. 78.0 Grok 4.5
  21. 78.0 Beam 1.0
  22. 77.8 Claude Opus 4.6
  23. 77.8 SWE-1.7
  24. 77.3 LongCat-2.0
  25. 76.7 Kimi K2.6
  26. 76.5 MiniMax-M2.7
  27. 76.2 Claude Opus 4.5
  28. 76.2 DeepSeek-V4-Pro
  29. 75.9 Claude Sonnet 4.6
  30. 75.8 Qwen3.7-Plus
  31. 75.8 Tencent Hy3
  32. 74.1 MiniMax-M2.5
  33. 73.8 Qwen 3.6 Plus
  34. 73.7 Composer 2
  35. 73.3 GLM-5
  36. 73.3 DeepSeek-V4-Flash
  37. 73.0 Kimi K2.5
  38. 72.7 Gemini 3 Flash
  39. 72.5 MiniMax-M2.1
  40. 72.4 Step 3.7 Flash
  41. 71.7 MiMo-V2-Flash
  42. 71.7 Seed2.0 Pro
  43. 71.7 MiMo-V2-Pro
  44. 71.3 Qwen3.6 27B
  45. 70.2 DeepSeek-V3.2
  46. 69.3 Qwen3.5 397B-A17B
  47. 68.7 Gemini 3 Pro
  48. 67.7 Nemotron 3.5 Super VL
  49. 67.7 Nemotron 3 Ultra
  50. 67.4 Step 3.5 Flash
  51. 67.3 Laguna M.1
  52. 67.2 Qwen 3.6 35B-A3B
  53. 67.0 Claude Sonnet 4.5
  54. 66.7 GPT-5.2
  55. 66.7 GLM-4.7
  56. 66.6 Seed2.0 Lite (0428)
  57. 66.3 GPT-5.2 Codex
  58. 65.5 MAI-Code-1-Flash
  59. 64.7 Claude Haiku 4.5
  60. 64.7 Spark-X2.6-4B
  61. 63.1 Laguna XS 2.1
  62. 63.0 KAT-Coder-V2.5-Dev
  63. 62.8 Qwen3-Coder-Next
  64. 62.4 Laguna XS.2
  65. 61.3 Devstral 2
  66. 61.1 Kimi K2 Thinking
  67. 57.9 DeepSeek-V3.2-Exp
  68. 57.8 DeepSeek-V3.1-Terminus
  69. 56.5 MiniMax-M2
  70. 55.9 Kimi K2
  71. 55.7 Devstral Small 2
  72. 54.7 Qwen3-Coder-480B-A35B
  73. 54.5 DeepSeek-V3.1
  74. 47.3 Kimi K2
  75. 45.8 Nemotron 3 Super
  76. 41.9 Granite 4.2 30B
  77. 39.7 GPT-5 mini
  78. 39.3 Nemotron 3.5 Lightning 30B-A3B
  79. 32.1 Beam 1.0 Base
  80. 23.0 LFM3-1.7B
  1. 1 Claude Fable 5.2 96.6%
  2. 2 Claude Opus 5.5 93.9%
  3. 3 Claude Sonnet 5.5 90.3%
  4. 4 Claude Opus 5 89.5%
  5. 5 Claude Fable 5.1 89.1%
  6. 6 Ling-3.1-flash 88.5%
  7. 7 Claude Mythos Preview 87.3%
  8. 8 Claude Opus 4.8 84.4%
  9. 9 Intern-S2-397B 84%
  10. 10 Claude Haiku 5.5 83.7%
  11. 11 Tencent Hy4 preview 82.9%
  12. 12 Inkling 2 Small 81.2%
  13. 13 Qwen3.8 Flash 81%
  14. 14 Claude Opus 4.7 80.5%
  15. 15 Qwen3.8-Omni-Flash 80.5%
  16. 16 Composer 2.5 79.8%
  17. 17 Laguna S 2.1 78.5%
  18. 18 Qwen3.7-Max 78.3%
  19. 19 Claude Sonnet 5 78.3%
  20. 20 Grok 4.5 78%
  21. 21 Beam 1.0 78%
  22. 22 Claude Opus 4.6 77.8%
  23. 23 SWE-1.7 77.8%
  24. 24 LongCat-2.0 77.3%
  25. 25 Kimi K2.6 76.7%
  26. 26 MiniMax-M2.7 76.5%
  27. 27 Claude Opus 4.5 76.2%
  28. 28 DeepSeek-V4-Pro 76.2%
  29. 29 Claude Sonnet 4.6 75.9%
  30. 30 Qwen3.7-Plus 75.8%
  31. 31 Tencent Hy3 75.8%
  32. 32 MiniMax-M2.5 74.1%
  33. 33 Qwen 3.6 Plus 73.8%
  34. 34 Composer 2 73.7%
  35. 35 GLM-5 73.3%
  36. 36 DeepSeek-V4-Flash 73.3%
  37. 37 Kimi K2.5 73%
  38. 38 Gemini 3 Flash 72.7%
  39. 39 MiniMax-M2.1 72.5%
  40. 40 Step 3.7 Flash 72.4%
  41. 41 MiMo-V2-Flash 71.7%
  42. 42 Seed2.0 Pro 71.7%
  43. 43 MiMo-V2-Pro 71.7%
  44. 44 Qwen3.6 27B 71.3%
  45. 45 DeepSeek-V3.2 70.2%
  46. 46 Qwen3.5 397B-A17B 69.3%
  47. 47 Gemini 3 Pro 68.7%
  48. 48 Nemotron 3.5 Super VL 67.7%
  49. 49 Nemotron 3 Ultra 67.7%
  50. 50 Step 3.5 Flash 67.4%
  51. 51 Laguna M.1 67.3%
  52. 52 Qwen 3.6 35B-A3B 67.2%
  53. 53 Claude Sonnet 4.5 67%
  54. 54 GPT-5.2 high 66.7%
  55. 55 GLM-4.7 66.7%
  56. 56 Seed2.0 Lite (0428) 66.6%
  57. 57 GPT-5.2 Codex 66.3%
  58. 58 MAI-Code-1-Flash 65.5%
  59. 59 Claude Haiku 4.5 64.7%
  60. 60 Spark-X2.6-4B 64.7%
  61. 61 Laguna XS 2.1 63.1%
  62. 62 KAT-Coder-V2.5-Dev 63%
  63. 63 Qwen3-Coder-Next 62.8%
  64. 64 Laguna XS.2 62.4%
  65. 65 Devstral 2 61.3%
  66. 66 Kimi K2 Thinking 61.1%
  67. 67 DeepSeek-V3.2-Exp 57.9%
  68. 68 DeepSeek-V3.1-Terminus 57.8%
  69. 69 MiniMax-M2 56.5%
  70. 70 Kimi K2 Sep 2025 55.9%
  71. 71 Devstral Small 2 55.7%
  72. 72 Qwen3-Coder-480B-A35B 54.7%
  73. 73 DeepSeek-V3.1 54.5%
  74. 74 Kimi K2 Jul 2025 47.3%
  75. 75 Nemotron 3 Super 45.8%
  76. 76 Granite 4.2 30B 41.9%
  77. 77 GPT-5 mini 39.7%
  78. 78 Nemotron 3.5 Lightning 30B-A3B 39.3%
  79. 79 Beam 1.0 Base 32.1%
  80. 80 LFM3-1.7B 23%

Sources: Labs’ model cards and official SWE-bench Multilingual leaderboard

#ModelScoreSettingDateSource
1 Claude Fable 5.2 96.6% Nov 2026 Lab model card
2 Claude Opus 5.5 93.9% Sep 2026 Lab model card
3 Claude Sonnet 5.5 90.3% Sep 2026 Lab model card
4 Claude Opus 5 89.5% Jul 2026 Lab model card
5 Claude Fable 5.1 89.1% Sep 2026 Lab model card
6 Ling-3.1-flash 88.5% Oct 2026 Lab model card
7 Claude Mythos Preview 87.3% Apr 2026 Lab model card
8 Claude Opus 4.8 84.4% May 2026 Lab model card
9 Intern-S2-397B 84% Sep 2026 Lab model card
10 Claude Haiku 5.5 83.7% Oct 2026 Lab model card
11 Tencent Hy4 preview 82.9% Aug 2026 Lab model card
12 Inkling 2 Small 81.2% Nov 2026 Lab model card
13 Qwen3.8 Flash 81% Aug 2026 Lab model card
14 Claude Opus 4.7 80.5% Apr 2026 Lab model card
15 Qwen3.8-Omni-Flash 80.5% Sep 2026 Lab model card
16 Composer 2.5 79.8% May 2026 Lab model card
17 Laguna S 2.1 78.5% Jul 2026 Lab model card
18 Qwen3.7-Max 78.3% May 2026 Lab model card
19 Claude Sonnet 5 78.3% Jun 2026 Lab model card
20 Grok 4.5 78% Jul 2026 Lab model card
21 Beam 1.0 78% Oct 2026 Lab model card
22 Claude Opus 4.6 77.8% Feb 2026 Lab model card
23 SWE-1.7 77.8% Jul 2026 Lab model card
24 LongCat-2.0 77.3% Jul 2026 Lab model card
25 Kimi K2.6 76.7% Apr 2026 Lab model card
26 MiniMax-M2.7 76.5% Mar 2026 Lab model card
27 Claude Opus 4.5 76.2% Nov 2025 Lab model card
28 DeepSeek-V4-Pro 76.2% Apr 2026 Lab model card
29 Claude Sonnet 4.6 75.9% Feb 2026 Lab model card
30 Qwen3.7-Plus 75.8% Jun 2026 Lab model card
31 Tencent Hy3 75.8% Jul 2026 Lab model card
32 MiniMax-M2.5 74.1% Feb 2026 Lab model card
33 Qwen 3.6 Plus 73.8% Apr 2026 Lab model card
34 Composer 2 73.7% Mar 2026 Lab model card
35 GLM-5 73.3% Feb 2026 Lab model card
36 DeepSeek-V4-Flash 73.3% Apr 2026 Lab model card
37 Kimi K2.5 73% Jan 2026 Lab model card
38 MiniMax-M2.1 72.5% Dec 2025 Lab model card
39 Step 3.7 Flash 72.4% May 2026 Lab model card
40 MiMo-V2-Flash 71.7% Dec 2025 Lab model card
41 Seed2.0 Pro 71.7% Feb 2026 Lab model card
42 MiMo-V2-Pro 71.7% Mar 2026 Lab model card
43 Qwen3.6 27B 71.3% Apr 2026 Lab model card
44 DeepSeek-V3.2 70.2% Dec 2025 Lab model card
45 Qwen3.5 397B-A17B 69.3% Feb 2026 Lab model card
46 Nemotron 3.5 Super VL 67.7% Oct 2026 Lab model card
47 Nemotron 3 Ultra 67.7% Jun 2026 Lab model card
48 Step 3.5 Flash 67.4% Feb 2026 Lab model card
49 Laguna M.1 67.3% Apr 2026 Lab model card
50 Qwen 3.6 35B-A3B 67.2% Apr 2026 Lab model card
51 GLM-4.7 66.7% Dec 2025 Lab model card
52 Seed2.0 Lite (0428) 66.6% Apr 2026 Lab model card
53 MAI-Code-1-Flash 65.5% Jun 2026 Lab model card
54 Spark-X2.6-4B 64.7% Oct 2026 Lab model card
55 Laguna XS 2.1 63.1% Jul 2026 Lab model card
56 KAT-Coder-V2.5-Dev 63% Jul 2026 Lab model card
57 Qwen3-Coder-Next 62.8% Feb 2026 Lab model card
58 Laguna XS.2 62.4% Apr 2026 Lab model card
59 Devstral 2 61.3% Dec 2025 Lab model card
60 Kimi K2 Thinking 61.1% Nov 2025 Lab model card
61 DeepSeek-V3.2-Exp 57.9% Sep 2025 Lab model card
62 DeepSeek-V3.1-Terminus 57.8% Sep 2025 Lab model card
63 MiniMax-M2 56.5% Oct 2025 Lab model card
64 Kimi K2 Sep 2025 55.9% Sep 2025 Lab model card
65 Devstral Small 2 55.7% Dec 2025 Lab model card
66 Qwen3-Coder-480B-A35B 54.7% Jul 2025 Lab model card
67 DeepSeek-V3.1 54.5% Aug 2025 Lab model card
68 Kimi K2 Jul 2025 47.3% Jul 2025 Lab model card
69 Nemotron 3 Super 45.8% Mar 2026 Lab model card
70 Granite 4.2 30B 41.9% Aug 2026 Lab model card
71 Nemotron 3.5 Lightning 30B-A3B 39.3% Aug 2026 Lab model card
72 Beam 1.0 Base 32.1% Oct 2026 Lab model card
73 LFM3-1.7B 23% Oct 2026 Lab model card
#ModelScoreSettingDateSource
1 Gemini 3 Flash 72.7% mini-SWE-agent Dec 2025 Official leaderboard
2 Claude Opus 4.6 72% mini-SWE-agent Feb 2026 Official leaderboard
3 Claude Opus 4.5 70.7% mini-SWE-agent Nov 2025 Official leaderboard
4 GLM-5 69.7% mini-SWE-agent Feb 2026 Official leaderboard
5 Gemini 3 Pro 68.7% mini-SWE-agent Nov 2025 Official leaderboard
6 MiniMax-M2.5 68.3% mini-SWE-agent Feb 2026 Official leaderboard
7 Kimi K2.5 67.3% mini-SWE-agent Jan 2026 Official leaderboard
8 Claude Sonnet 4.5 67% mini-SWE-agent Sep 2025 Official leaderboard
9 GPT-5.2 66.7% high, mini-SWE-agent Dec 2025 Official leaderboard
10 GPT-5.2 Codex 66.3% mini-SWE-agent Dec 2025 Official leaderboard
11 Claude Haiku 4.5 64.7% mini-SWE-agent Oct 2025 Official leaderboard
12 DeepSeek-V3.2 59% mini-SWE-agent Dec 2025 Official leaderboard
13 GPT-5 mini 39.7% mini-SWE-agent Aug 2025 Official leaderboard
#ModelScoreSettingDateSource
1 Claude Fable 5.2 96.6% Nov 2026 Lab model card
2 Claude Opus 5.5 93.9% Sep 2026 Lab model card
3 Claude Sonnet 5.5 90.3% Sep 2026 Lab model card
4 Claude Opus 5 89.5% Jul 2026 Lab model card
5 Claude Fable 5.1 89.1% Sep 2026 Lab model card
6 Ling-3.1-flash 88.5% Oct 2026 Lab model card
7 Claude Mythos Preview 87.3% Apr 2026 Lab model card
8 Claude Opus 4.8 84.4% May 2026 Lab model card
9 Intern-S2-397B 84% Sep 2026 Lab model card
10 Claude Haiku 5.5 83.7% Oct 2026 Lab model card
11 Tencent Hy4 preview 82.9% Aug 2026 Lab model card
12 Inkling 2 Small 81.2% Nov 2026 Lab model card
13 Qwen3.8 Flash 81% Aug 2026 Lab model card
14 Claude Opus 4.7 80.5% Apr 2026 Lab model card
15 Qwen3.8-Omni-Flash 80.5% Sep 2026 Lab model card
16 Composer 2.5 79.8% May 2026 Lab model card
17 Laguna S 2.1 78.5% Jul 2026 Lab model card
18 Qwen3.7-Max 78.3% May 2026 Lab model card
19 Claude Sonnet 5 78.3% Jun 2026 Lab model card
20 Grok 4.5 78% Jul 2026 Lab model card
21 Beam 1.0 78% Oct 2026 Lab model card
22 Claude Opus 4.6 77.8% Feb 2026 Lab model card
23 SWE-1.7 77.8% Jul 2026 Lab model card
24 LongCat-2.0 77.3% Jul 2026 Lab model card
25 Kimi K2.6 76.7% Apr 2026 Lab model card
26 MiniMax-M2.7 76.5% Mar 2026 Lab model card
27 Claude Opus 4.5 76.2% Nov 2025 Lab model card
28 DeepSeek-V4-Pro 76.2% Apr 2026 Lab model card
29 Claude Sonnet 4.6 75.9% Feb 2026 Lab model card
30 Qwen3.7-Plus 75.8% Jun 2026 Lab model card
31 Tencent Hy3 75.8% Jul 2026 Lab model card
32 MiniMax-M2.5 74.1% Feb 2026 Lab model card
33 Qwen 3.6 Plus 73.8% Apr 2026 Lab model card
34 Composer 2 73.7% Mar 2026 Lab model card
35 GLM-5 73.3% Feb 2026 Lab model card
36 DeepSeek-V4-Flash 73.3% Apr 2026 Lab model card
37 Kimi K2.5 73% Jan 2026 Lab model card
38 Gemini 3 Flash 72.7% mini-SWE-agent Dec 2025 Official leaderboard
39 MiniMax-M2.1 72.5% Dec 2025 Lab model card
40 Step 3.7 Flash 72.4% May 2026 Lab model card
41 MiMo-V2-Flash 71.7% Dec 2025 Lab model card
42 Seed2.0 Pro 71.7% Feb 2026 Lab model card
43 MiMo-V2-Pro 71.7% Mar 2026 Lab model card
44 Qwen3.6 27B 71.3% Apr 2026 Lab model card
45 DeepSeek-V3.2 70.2% Dec 2025 Lab model card
46 Qwen3.5 397B-A17B 69.3% Feb 2026 Lab model card
47 Gemini 3 Pro 68.7% mini-SWE-agent Nov 2025 Official leaderboard
48 Nemotron 3.5 Super VL 67.7% Oct 2026 Lab model card
49 Nemotron 3 Ultra 67.7% Jun 2026 Lab model card
50 Step 3.5 Flash 67.4% Feb 2026 Lab model card
51 Laguna M.1 67.3% Apr 2026 Lab model card
52 Qwen 3.6 35B-A3B 67.2% Apr 2026 Lab model card
53 Claude Sonnet 4.5 67% mini-SWE-agent Sep 2025 Official leaderboard
54 GPT-5.2 66.7% high, mini-SWE-agent Dec 2025 Official leaderboard
55 GLM-4.7 66.7% Dec 2025 Lab model card
56 Seed2.0 Lite (0428) 66.6% Apr 2026 Lab model card
57 GPT-5.2 Codex 66.3% mini-SWE-agent Dec 2025 Official leaderboard
58 MAI-Code-1-Flash 65.5% Jun 2026 Lab model card
59 Claude Haiku 4.5 64.7% mini-SWE-agent Oct 2025 Official leaderboard
60 Spark-X2.6-4B 64.7% Oct 2026 Lab model card
61 Laguna XS 2.1 63.1% Jul 2026 Lab model card
62 KAT-Coder-V2.5-Dev 63% Jul 2026 Lab model card
63 Qwen3-Coder-Next 62.8% Feb 2026 Lab model card
64 Laguna XS.2 62.4% Apr 2026 Lab model card
65 Devstral 2 61.3% Dec 2025 Lab model card
66 Kimi K2 Thinking 61.1% Nov 2025 Lab model card
67 DeepSeek-V3.2-Exp 57.9% Sep 2025 Lab model card
68 DeepSeek-V3.1-Terminus 57.8% Sep 2025 Lab model card
69 MiniMax-M2 56.5% Oct 2025 Lab model card
70 Kimi K2 Sep 2025 55.9% Sep 2025 Lab model card
71 Devstral Small 2 55.7% Dec 2025 Lab model card
72 Qwen3-Coder-480B-A35B 54.7% Jul 2025 Lab model card
73 DeepSeek-V3.1 54.5% Aug 2025 Lab model card
74 Kimi K2 Jul 2025 47.3% Jul 2025 Lab model card
75 Nemotron 3 Super 45.8% Mar 2026 Lab model card
76 Granite 4.2 30B 41.9% Aug 2026 Lab model card
77 GPT-5 mini 39.7% mini-SWE-agent Aug 2025 Official leaderboard
78 Nemotron 3.5 Lightning 30B-A3B 39.3% Aug 2026 Lab model card
79 Beam 1.0 Base 32.1% Oct 2026 Lab model card
80 LFM3-1.7B 23% Oct 2026 Lab model card

The benchmark's own leaderboard has 13 models. Switch the scores above to rank them together.

The best score, 72.7%, costs $0.35 a task

Runs on the cost frontier score higher than any run that's cheaper. MiniMax-M2.5 comes within 4.4 points of the best for $0.10 a task, 3 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Google DeepMind MiniMax Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Gemini 3 Flash MiniMax-M2.5 GPT-5 mini 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Gemini 3 Flash GPT-5 mini

Source: Official SWE-bench Multilingual leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Gemini 3 Flash mini-SWE-agent 72.7% $0.35
Claude Opus 4.6 mini-SWE-agent 72% $0.66
Claude Opus 4.5 mini-SWE-agent 70.7% $0.83
GLM-5 mini-SWE-agent 69.7% $0.64
Gemini 3 Pro mini-SWE-agent 68.7% $1.02
MiniMax-M2.5 mini-SWE-agent 68.3% $0.10
Kimi K2.5 mini-SWE-agent 67.3% $0.69
Claude Sonnet 4.5 mini-SWE-agent 67% $0.67
GPT-5.2 high mini-SWE-agent 66.7% $0.54
GPT-5.2 Codex mini-SWE-agent 66.3% $0.66
Claude Haiku 4.5 mini-SWE-agent 64.7% $0.38
DeepSeek-V3.2 mini-SWE-agent 59% $0.38
GPT-5 mini mini-SWE-agent 39.7% $0.05

12 models have held number one in labs’ own numbers since its release in April 2025 5 models have held number one across all independent runs since its release in April 2025 12 models have held number one with claims included since its release in April 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

OpenAI Anthropic Google DeepMind Moonshot AI Qwen DeepSeek
Own leaderboard Not tested since Feb 2026
Gemini 3 Flash 72.7%
Lab claims Passed 85% in Apr 2026, by Claude Mythos Preview
Claude Fable 5.2 96.6%

Sources: Official SWE-bench Multilingual leaderboard and labs’ model cards

Anthropic’s own number for Claude Mythos Preview passed 85% in April 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Fable 5.2 Claude Opus 5 Claude Mythos Preview Claude Opus 4.6 Gemini 3 Pro DeepSeek-V3.1-Terminus Kimi K2 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Fable 5.2 Claude Opus 5

Sources: Official SWE-bench Multilingual leaderboard and labs’ model cards

16 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Moonshot AI
Qwen
DeepSeek
MiniMax
Mistral AI
Xiaomi Corp
Z.ai
ByteDance Seed
NVIDIA
Cursor
Anthropic
Poolside
StepFun
Microsoft
Tencent
Meituan
Cognition
Kuaishou
IBM
Shanghai AI Lab
Reflection
Ant Group
iFlytek
Liquid AI
Thinking Machines
DateLaunch postScoreAs the post writes it
Nov 9, 2026 Claude Fable 5.2 96.6 SWE-bench Multilingual, adaptive thinking max effort, average of five trials (Table 8.1.A only)
Nov 5, 2026 Inkling 2 Small 81.2 SWE-bench Multilingual, 81.2%
Oct 27, 2026 LFM3-1.7B 23 SWE-bench Multilingual (OpenCode)
Oct 22, 2026 Spark-X2.6-4B 64.7 SWE Multilingual
Oct 15, 2026 Nemotron 3.5 Super VL 67.7 SWE-bench Multilingual (OpenCode harness)
Oct 13, 2026 Ling-3.1-flash 88.5 SWE-bench Multilingual
Oct 7, 2026 Claude Haiku 5.5 83.7 SWE-bench Multilingual
Oct 5, 2026 Beam 1.0 78 SWEBench Multilingual
Sep 28, 2026 Claude Sonnet 5.5 90.3 SWE-Bench Multilingual
Sep 22, 2026 Claude Opus 5.5 93.9 SWE-bench Multilingual
Sep 18, 2026 Qwen3.8-Omni-Flash 80.5 SWE-bench Multilingual (mini-SWE-agent)
Sep 13, 2026 Intern-S2-397B 84 SWE-bench-Multilingual (Mini-SWE-Agent)
Sep 1, 2026 Claude Fable 5.1 89.1 SWE-bench Multilingual
Aug 28, 2026 Tencent Hy4 preview 82.9 SWE-bench Multilingual
Aug 26, 2026 Qwen3.8 Flash 81 SWE-bench Multilingual (mini-SWE-agent)
Aug 25, 2026 Granite 4.2 30B 41.9 SWE Bench Multilingual
Aug 11, 2026 Nemotron 3.5 Lightning 30B-A3B 39.3 SWE-bench Multilingual
Jul 24, 2026 Claude Opus 5 89.5 SWE-bench Multilingual
Jul 23, 2026 KAT-Coder-V2.5-Dev 63 SWE-bench Multilingual
Jul 21, 2026 Laguna S 2.1 78.5 SWE-Bench Multilingual
Jul 8, 2026 SWE-1.7 77.8 SWE-Bench Multilingual (Devin CLI)
Jul 5, 2026 LongCat-2.0 77.3 SWE-bench Multilingual
Jul 2, 2026 Laguna XS 2.1 63.1 SWE-bench Multilingual
Jul 2, 2026 Tencent Hy3 75.8 SWE-bench Multilingual
Jun 4, 2026 Nemotron 3 Ultra 67.7 SWE-Bench Multilingual
Jun 2, 2026 MAI-Code-1-Flash 65.5 SWE-Bench Multilingual
Jun 1, 2026 Qwen3.7-Plus 75.8 SWE-Multilingual
May 29, 2026 Step 3.7 Flash 72.4 SWE-MTLG
May 28, 2026 Claude Opus 4.8 84.4 SWE-bench Multilingual
May 20, 2026 Qwen3.7-Max 78.3 SWE-Multilingual
May 18, 2026 Composer 2.5 79.8 SWE-Bench Multilingual
Apr 28, 2026 Laguna XS.2 62.4 SWE-bench Multilingual (mean pass@1 over 7 runs)
Apr 28, 2026 Laguna M.1 67.3 SWE-bench Multilingual (mean pass@1 over 7 runs)
Apr 28, 2026 Seed2.0 Lite (0428) 66.6 SWE Multilingual
Apr 24, 2026 DeepSeek-V4-Flash 73.3 SWE Multilingual (Resolved), V4-Flash Max
Apr 24, 2026 DeepSeek-V4-Pro 76.2 SWE Multilingual (Resolved)
Apr 22, 2026 Qwen3.6 27B 71.3 SWE-bench Multilingual
Apr 20, 2026 Kimi K2.6 76.7 SWE-Bench Multilingual
Apr 16, 2026 Claude Opus 4.7 80.5 SWE-bench Multilingual
Apr 15, 2026 Qwen 3.6 35B-A3B 67.2 SWE-bench Multilingual
Apr 7, 2026 Claude Mythos Preview 87.3 SWE-bench Multilingual
Apr 2, 2026 Qwen 3.6 Plus 73.8 SWE-bench Multilingual
Mar 19, 2026 Composer 2 73.7 SWE-bench Multilingual
Mar 18, 2026 MiMo-V2-Pro 71.7 SWE-bench Multilingual
Mar 18, 2026 MiniMax-M2.7 76.5 SWE Multilingual (text)
Mar 11, 2026 Nemotron 3 Super 45.8 SWE-Bench Multilingual (OpenHands)
Feb 16, 2026 Qwen3.5 397B-A17B 69.3 SWE-bench Multilingual
Feb 14, 2026 Seed2.0 Pro 71.7 SWE Multilingual
Feb 12, 2026 MiniMax-M2.5 74.1 SWE-Bench Multilingual
Feb 11, 2026 GLM-5 73.3 SWE-bench Multilingual
Feb 2, 2026 Qwen3-Coder-Next 62.8 SWE-Bench Multilingual (w/ SWE-Agent)
Jan 27, 2026 Kimi K2.5 73 SWE-Bench Multilingual
Dec 23, 2025 MiniMax-M2.1 72.5 SWE-bench Multilingual
Dec 22, 2025 GLM-4.7 66.7 SWE-bench Multilingual
Dec 16, 2025 MiMo-V2-Flash 71.7 SWE-Bench Multilingual
Dec 9, 2025 Devstral Small 2 55.7 SWE Bench Multilingual (HF card)
Dec 9, 2025 Devstral 2 61.3 SWE Bench Multilingual (HF card)
Dec 1, 2025 DeepSeek-V3.2 70.2 SWE Multilingual (Resolved)
Nov 6, 2025 Kimi K2 Thinking 61.1 SWE-bench Multilingual, w/ tools
Oct 27, 2025 MiniMax-M2 56.5 SWE-bench Multilingual
Sep 29, 2025 DeepSeek-V3.2-Exp 57.9 SWE-bench Multilingual
Sep 22, 2025 DeepSeek-V3.1-Terminus 57.8 SWE-bench Multilingual
Sep 5, 2025 Kimi K2 55.9 SWE-Bench Multilingual, ACC
Aug 21, 2025 DeepSeek-V3.1 54.5 SWE-bench Multilingual (Agent mode)
Jul 22, 2025 Qwen3-Coder-480B-A35B 54.7 SWE-bench Multilingual
Jul 11, 2025 Kimi K2 47.3 SWE-bench Multilingual (Agentic Coding), Single Attempt (Acc)

38 system cards report it, from 15 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Nov 9, 2026 Claude Fable 5.2 system card 96.6 SWE-bench Multilingual, adaptive thinking max effort, average of five trials (Table 8.1.A and Section 8.2) Capabilities
Oct 15, 2026 Nemotron 3.5 Super VL technical report 67.7 SWE-bench Multilingual, OpenCode harness (Table 10) Capabilities
Oct 15, 2026 Nemotron 3.5 Lightning 30B-A3B technical report 37.1 SWE-Bench Multilingual, BF16 acc. (Table 22) Capabilities
Oct 7, 2026 Claude Haiku 5.5 system card 83.7 SWE-bench Multilingual, average of five trials, max effort (Table 8.1.A) Capabilities
Oct 5, 2026 Beam 1.0 Base technical report 32.1 SWE-bench Multilingual, agentic, temperature 0.7, Table 23 Capabilities
Sep 28, 2026 Claude Sonnet 5.5 system card 90.3 SWE-Bench Multilingual, average of five trials (Table 8.1.A) Capabilities
Sep 22, 2026 Qwen3.8-Omni-Flash technical report 80.5 SWE-bench Multilingual, mini-SWE-agent (Table 2) Capabilities
Sep 22, 2026 Claude Opus 5.5 system card 93.9 SWE-bench Multilingual, average of five trials, max effort (Table 8.1.A, 8.2) Capabilities
Sep 1, 2026 Claude Fable 5.1 system card 89.1 SWE-bench Multilingual (Table 8.1.A) Capabilities
Jul 24, 2026 Claude Opus 5 system card 89.5 SWE-bench Multilingual, average of five trials (Table 8.1.A, 8.2) Capabilities
Jul 14, 2026 Grok 4.5 model card 78 SWE-Bench Multilingual, resolve rate, high, runs reported by Cursor (2.5) Capabilities
Jun 30, 2026 Claude Sonnet 5 system card 78.3 SWE-bench Multilingual, average of five trials (8.2) Capabilities
Jun 9, 2026 Nemotron 3 Ultra technical report 67.7 SWE-Bench Multilingual (Table 10) Capabilities
Jun 2, 2026 MAI-Code-1-Flash model card 65.5 SWE-Bench Multilingual, pass rate, Copilot production harness (Coding table) Capabilities
May 28, 2026 Claude Opus 4.8 system card 84.4 SWE-bench Multilingual, average of five trials, max effort (Table 8.1.A, 8.2) Capabilities
May 26, 2026 MiniMax-M2 technical report 56.5 SWE-bench Multilingual, MiniMax-M2 bar, printed label (Figure 9) Capabilities
May 26, 2026 MiniMax-M2.7 technical report 76.5 SWE-bench Multilingual, Claude Code scaffold, avg of 4 trials (Table 4) Capabilities
May 26, 2026 MiniMax-M2.5 technical report 74.1 SWE-bench Multilingual, Claude Code scaffold, avg of 4 trials (Table 4) Capabilities
May 25, 2026 Laguna XS.2 technical report 57.7 SWE-bench Multilingual, pool harness, 500 steps, mean pass@1 over 4 runs, some task images and verifiers patched (Table 6) Capabilities
May 25, 2026 Laguna M.1 technical report 63.1 SWE-bench Multilingual, pool harness, 500 steps, mean pass@1 over 4 runs, some task images and verifiers patched (Table 7) Capabilities
Apr 26, 2026 DeepSeek-V4-Pro technical report 76.2 SWE Multilingual (Resolved), DS-V4-Pro Max (Table 6) Capabilities
Apr 26, 2026 DeepSeek-V4-Flash technical report 73.3 SWE Multilingual (Resolved), DeepSeek-V4-Flash Max (Table 7) Capabilities
Apr 16, 2026 Claude Opus 4.7 system card 80.5 SWE-bench Multilingual, average of 5 trials (Table 8.1.A, 8.2) Capabilities
Apr 7, 2026 Claude Mythos Preview system card 87.3 SWE-bench Multilingual, adaptive thinking, max effort, average of 5 trials (Table 6.3.A, 6.4) Capabilities
Mar 25, 2026 Composer 2 technical report 73.7 SWE-bench Multi., Cursor's own harness (Table 1 and 7.2) Capabilities
Mar 11, 2026 Nemotron 3 Super technical report 45.8 SWE-Bench Multilingual (OpenHands) (Table 5) Capabilities
Feb 28, 2026 Qwen3-Coder-Next technical report 62.8 SWE-Bench Multilingual, SWE-Agent. MiniSWE-Agent 56.2, OpenHands 64.3 (Table 4) Capabilities
Feb 17, 2026 GLM-5 technical report 73.3 SWE-bench Multilingual, OpenHands (Table 7) Capabilities
Feb 17, 2026 Claude Sonnet 4.6 system card 75.9 SWE-bench Multilingual, adaptive thinking, max effort (2.2) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 71.7 SWE Multilingual (Table 11) Capabilities
Feb 11, 2026 Step 3.5 Flash technical report 67.4 SWE Multilingual (Table 5) Capabilities
Feb 5, 2026 Claude Opus 4.6 system card 77.8 SWE-bench Multilingual, adaptive thinking, max effort, average of 25 trials (2.4) Capabilities
Jan 30, 2026 Kimi K2.5 technical report 73 SWE-Bench Multilingual (Table 4) Capabilities
Jan 6, 2026 MiMo-V2-Flash technical report 71.7 SWE-Bench Multilingual (Table 9) Capabilities
Dec 1, 2025 DeepSeek-V3.2 technical report 70.2 SWE Multilingual (Resolved), DeepSeek-V3.2 Thinking (Table 2) Capabilities
Nov 24, 2025 Claude Opus 4.5 system card 76.2 SWE-bench Multilingual, no thinking and 64k thinking both 76.20% (Table 2.4.A) Capabilities
Sep 29, 2025 DeepSeek-V3.2-Exp technical report 57.9 SWE-bench Multilingual (Agent mode), DeepSeek-V3.2-Exp (Table 1) Capabilities
Jul 21, 2025 Kimi K2 technical report 47.3 SWE-bench Multilingual (Pass@1), single-attempt agentic (Table 3) Capabilities

The lab's number comes out ahead in 6 of 6 pairs that differ

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.

Model Lab says Board says Gap
DeepSeek-V3.2 70.2% SWE Multilingual (Resolved) 59% Leaderboard, mini-SWE-agent +11 pts
Claude Opus 4.6 77.8% SWE-bench Multilingual, adaptive thinking, max effort, average of 25 trials (2.4) 72% Leaderboard, mini-SWE-agent +5.8 pts
MiniMax-M2.5 74.1% SWE-Bench Multilingual 68.3% Leaderboard, mini-SWE-agent +5.8 pts
Kimi K2.5 73% SWE-Bench Multilingual 67.3% Leaderboard, mini-SWE-agent +5.7 pts
Claude Opus 4.5 76.2% SWE-bench Multilingual, no thinking and 64k thinking both 76.20% (Table 2.4.A) 70.7% Leaderboard, mini-SWE-agent +5.5 pts
GLM-5 73.3% SWE-bench Multilingual 69.7% Leaderboard, mini-SWE-agent +3.6 pts

More in Coding

All 282 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471
  • NL2Repo-Bench In use From one natural-language requirements document and an empty workspace, agent builds an installable Python library. Scored by original project's test pass rate. DeepSeek V4.1 Flash 1017 68.2%