Benchmark Almanac

Aider Polyglot

GPT-5 88% leads since August 2025
GPT-5 88% leads since August 2025
GPT-5 88% leads since August 2025
Fading

Models edit code to solve 225 hard Exercism exercises in C++, Go, Java, JavaScript, Python, and Rust. Scored by the share passing unit tests.

Exercises come from public Exercism tracks, so contamination is possible.

Made by
Aider
Released
December 2024
Kind
Function-level problems
Level
University
Scored
Checked automatically
Answers
Code
Tasks
All public
Strongest model tested
GPT-5 #70 on the capability index
Last in a launch post
February 2026, dropped after it saturated
Last model added
February 2026
Scores refreshed
November 12th, 2026
30
Models scored in lab cards
26
Launch posts list it, from 7 labs
17
System cards report it, from 9 labs

GPT-5 leads with 88% among models released by February 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

GPT-5 leads with 88% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

GPT-5 leads with 88%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in February 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

30 models. Scroll sideways for the rest.

  1. 88.0 GPT-5
  2. 82.2 Gemini 2.5 Pro
  3. 81.3 o3
  4. 80.0 Seed2.0 Pro
  5. 76.3 DeepSeek-V3.1
  6. 76.1 DeepSeek-V3.1-Terminus
  7. 74.5 DeepSeek-V3.2-Exp
  8. 74.0 Gemini 2.5 Pro
  9. 71.6 DeepSeek-R1
  10. 71.6 GPT-5 mini
  11. 68.9 o4-mini
  12. 66.2 Qwen3-Coder-Next
  13. 61.9 Gemini 2.5 Flash
  14. 61.8 Qwen3-Coder-480B-A35B
  15. 60.0 Kimi K2
  16. 57.3 gemini-robotics-er-1.5-preview
  17. 54.2 Seed-Thinking-v1.5
  18. 53.3 DeepSeek-R1
  19. 52.9 GPT-4.1
  20. 51.1 Gemini 2.5 Flash
  21. 49.6 DeepSeek-V3
  22. 48.4 GPT-5 nano
  23. 47.1 Magistral Medium 1.0
  24. 44.4 gpt-oss-120b
  25. 35.1 Code World Model (CWM)
  26. 34.7 GPT-4.1 mini
  27. 34.2 gpt-oss-20b
  28. 27.1 Gemini 2.5 Flash-Lite
  29. 14.7 Cohere Command A
  30. 9.80 GPT-4.1 nano
  1. 1 GPT-5 88%
  2. 2 Gemini 2.5 Pro Jun 2025 82.2%
  3. 3 o3 81.3%
  4. 4 Seed2.0 Pro 80%
  5. 5 DeepSeek-V3.1 76.3%
  6. 6 DeepSeek-V3.1-Terminus 76.1%
  7. 7 DeepSeek-V3.2-Exp 74.5%
  8. 8 Gemini 2.5 Pro Mar 2025 74%
  9. 9 DeepSeek-R1 May 2025 71.6%
  10. 10 GPT-5 mini 71.6%
  11. 11 o4-mini 68.9%
  12. 12 Qwen3-Coder-Next 66.2%
  13. 13 Gemini 2.5 Flash Jun 2025 61.9%
  14. 14 Qwen3-Coder-480B-A35B 61.8%
  15. 15 Kimi K2 Jul 2025 60%
  16. 16 gemini-robotics-er-1.5-preview 57.3%
  17. 17 Seed-Thinking-v1.5 54.2%
  18. 18 DeepSeek-R1 53.3%
  19. 19 GPT-4.1 52.9%
  20. 20 Gemini 2.5 Flash Apr 2025 51.1%
  21. 21 DeepSeek-V3 49.6%
  22. 22 GPT-5 nano 48.4%
  23. 23 Magistral Medium 1.0 47.1%
  24. 24 gpt-oss-120b 44.4%
  25. 25 Code World Model (CWM) 35.1%
  26. 26 GPT-4.1 mini 34.7%
  27. 27 gpt-oss-20b 34.2%
  28. 28 Gemini 2.5 Flash-Lite Jun 2025 27.1%
  29. 29 Cohere Command A 14.7%
  30. 30 GPT-4.1 nano 9.80%

Source: Labs’ model cards

46 models. Scroll sideways for the rest.

  1. 88.0 GPT-5
  2. 84.9 o3-pro
  3. 83.1 Gemini 2.5 Pro
  4. 81.3 o3
  5. 79.6 Grok 4
  6. 76.9 Gemini 2.5 Pro
  7. 72.9 Gemini 2.5 Pro
  8. 72.0 o4-mini
  9. 72.0 Claude Opus 4
  10. 71.4 DeepSeek-R1
  11. 64.9 Claude 3.7 Sonnet
  12. 61.7 o1
  13. 61.3 Claude Sonnet 4
  14. 60.4 o3-mini
  15. 59.1 Kimi K2
  16. 56.9 DeepSeek-R1
  17. 55.1 DeepSeek-V3
  18. 55.1 Gemini 2.5 Flash
  19. 53.3 Grok 3
  20. 52.4 GPT-4.1
  21. 51.6 Claude 3.5 Sonnet
  22. 49.3 Grok-3 mini
  23. 47.1 Gemini 2.5 Flash
  24. 45.3 GPT-4o
  25. 44.9 GPT-4.5
  26. 41.8 gpt-oss-120b
  27. 40.0 Qwen3-32B
  28. 35.6 Gemini 2.0 Pro
  29. 32.9 o1-mini
  30. 32.4 GPT-4.1 mini
  31. 28.0 Claude 3.5 Haiku
  32. 27.1 GPT-4o
  33. 23.1 GPT-4o
  34. 22.2 Gemini 2.0 Flash
  35. 20.9 QwQ-32B
  36. 18.2 GPT-4o
  37. 18.2 Gemini 2.0 Flash Thinking
  38. 16.4 Qwen2.5-Coder-32B-Instruct
  39. 15.6 Llama 4 Maverick
  40. 12.9 Yi-Lightning
  41. 12.0 Cohere Command A
  42. 11.1 Codestral
  43. 10.2 openhands-lm-32b-v0.1
  44. 8.90 GPT-4.1 nano
  45. 4.90 Gemma 3 27B
  46. 3.60 GPT-4o mini
  1. 1 GPT-5 high 88%
  2. 2 o3-pro high 84.9%
  3. 3 Gemini 2.5 Pro Jun 2025 83.1%
  4. 4 o3 high 81.3%
  5. 5 Grok 4 high 79.6%
  6. 6 Gemini 2.5 Pro May 2025 76.9%
  7. 7 Gemini 2.5 Pro Mar 2025 72.9%
  8. 8 o4-mini high 72%
  9. 9 Claude Opus 4 32k thinking 72%
  10. 10 DeepSeek-R1 May 2025 71.4%
  11. 11 Claude 3.7 Sonnet 32k thinking tokens 64.9%
  12. 12 o1 high 61.7%
  13. 13 Claude Sonnet 4 32k thinking 61.3%
  14. 14 o3-mini high 60.4%
  15. 15 Kimi K2 Jul 2025 59.1%
  16. 16 DeepSeek-R1 56.9%
  17. 17 DeepSeek-V3 Mar 2025 55.1%
  18. 18 Gemini 2.5 Flash May 2025 55.1%
  19. 19 Grok 3 53.3%
  20. 20 GPT-4.1 52.4%
  21. 21 Claude 3.5 Sonnet 51.6%
  22. 22 Grok-3 mini high 49.3%
  23. 23 Gemini 2.5 Flash Apr 2025 47.1%
  24. 24 GPT-4o Mar 2025 45.3%
  25. 25 GPT-4.5 44.9%
  26. 26 gpt-oss-120b high 41.8%
  27. 27 Qwen3-32B 40%
  28. 28 Gemini 2.0 Pro 35.6%
  29. 29 o1-mini 32.9%
  30. 30 GPT-4.1 mini 32.4%
  31. 31 Claude 3.5 Haiku 28%
  32. 32 GPT-4o Jan 2025 27.1%
  33. 33 GPT-4o Aug 2024 23.1%
  34. 34 Gemini 2.0 Flash Dec 2024 22.2%
  35. 35 QwQ-32B 20.9%
  36. 36 GPT-4o Nov 2024 18.2%
  37. 37 Gemini 2.0 Flash Thinking Jan 2025 18.2%
  38. 38 Qwen2.5-Coder-32B-Instruct 16.4%
  39. 39 Llama 4 Maverick 15.6%
  40. 40 Yi-Lightning 12.9%
  41. 41 Cohere Command A 12%
  42. 42 Codestral 11.1%
  43. 43 openhands-lm-32b-v0.1 10.2%
  44. 44 GPT-4.1 nano 8.90%
  45. 45 Gemma 3 27B 4.90%
  46. 46 GPT-4o mini 3.60%

Source: Official Aider Polyglot leaderboard

62 models. Scroll sideways for the rest.

  1. 88.0 GPT-5
  2. 84.9 o3-pro
  3. 83.1 Gemini 2.5 Pro
  4. 81.3 o3
  5. 80.0 Seed2.0 Pro
  6. 79.6 Grok 4
  7. 76.9 Gemini 2.5 Pro
  8. 76.3 DeepSeek-V3.1
  9. 76.1 DeepSeek-V3.1-Terminus
  10. 74.5 DeepSeek-V3.2-Exp
  11. 74.0 Gemini 2.5 Pro
  12. 72.0 o4-mini
  13. 72.0 Claude Opus 4
  14. 71.6 DeepSeek-R1
  15. 71.6 GPT-5 mini
  16. 66.2 Qwen3-Coder-Next
  17. 64.9 Claude 3.7 Sonnet
  18. 61.9 Gemini 2.5 Flash
  19. 61.8 Qwen3-Coder-480B-A35B
  20. 61.7 o1
  21. 61.3 Claude Sonnet 4
  22. 60.4 o3-mini
  23. 60.0 Kimi K2
  24. 57.3 gemini-robotics-er-1.5-preview
  25. 56.9 DeepSeek-R1
  26. 55.1 DeepSeek-V3
  27. 55.1 Gemini 2.5 Flash
  28. 54.2 Seed-Thinking-v1.5
  29. 53.3 Grok 3
  30. 52.9 GPT-4.1
  31. 51.6 Claude 3.5 Sonnet
  32. 51.1 Gemini 2.5 Flash
  33. 49.6 DeepSeek-V3
  34. 49.3 Grok-3 mini
  35. 48.4 GPT-5 nano
  36. 47.1 Magistral Medium 1.0
  37. 45.3 GPT-4o
  38. 44.9 GPT-4.5
  39. 44.4 gpt-oss-120b
  40. 40.0 Qwen3-32B
  41. 35.6 Gemini 2.0 Pro
  42. 35.1 Code World Model (CWM)
  43. 34.7 GPT-4.1 mini
  44. 34.2 gpt-oss-20b
  45. 32.9 o1-mini
  46. 28.0 Claude 3.5 Haiku
  47. 27.1 GPT-4o
  48. 27.1 Gemini 2.5 Flash-Lite
  49. 23.1 GPT-4o
  50. 22.2 Gemini 2.0 Flash
  51. 20.9 QwQ-32B
  52. 18.2 GPT-4o
  53. 18.2 Gemini 2.0 Flash Thinking
  54. 16.4 Qwen2.5-Coder-32B-Instruct
  55. 15.6 Llama 4 Maverick
  56. 14.7 Cohere Command A
  57. 12.9 Yi-Lightning
  58. 11.1 Codestral
  59. 10.2 openhands-lm-32b-v0.1
  60. 9.80 GPT-4.1 nano
  61. 4.90 Gemma 3 27B
  62. 3.60 GPT-4o mini
  1. 1 GPT-5 high 88%
  2. 2 o3-pro high 84.9%
  3. 3 Gemini 2.5 Pro Jun 2025 83.1%
  4. 4 o3 high 81.3%
  5. 5 Seed2.0 Pro 80%
  6. 6 Grok 4 high 79.6%
  7. 7 Gemini 2.5 Pro May 2025 76.9%
  8. 8 DeepSeek-V3.1 76.3%
  9. 9 DeepSeek-V3.1-Terminus 76.1%
  10. 10 DeepSeek-V3.2-Exp 74.5%
  11. 11 Gemini 2.5 Pro Mar 2025 74%
  12. 12 o4-mini high 72%
  13. 13 Claude Opus 4 32k thinking 72%
  14. 14 DeepSeek-R1 May 2025 71.6%
  15. 15 GPT-5 mini 71.6%
  16. 16 Qwen3-Coder-Next 66.2%
  17. 17 Claude 3.7 Sonnet 32k thinking tokens 64.9%
  18. 18 Gemini 2.5 Flash Jun 2025 61.9%
  19. 19 Qwen3-Coder-480B-A35B 61.8%
  20. 20 o1 high 61.7%
  21. 21 Claude Sonnet 4 32k thinking 61.3%
  22. 22 o3-mini high 60.4%
  23. 23 Kimi K2 Jul 2025 60%
  24. 24 gemini-robotics-er-1.5-preview 57.3%
  25. 25 DeepSeek-R1 56.9%
  26. 26 DeepSeek-V3 Mar 2025 55.1%
  27. 27 Gemini 2.5 Flash May 2025 55.1%
  28. 28 Seed-Thinking-v1.5 54.2%
  29. 29 Grok 3 53.3%
  30. 30 GPT-4.1 52.9%
  31. 31 Claude 3.5 Sonnet 51.6%
  32. 32 Gemini 2.5 Flash Apr 2025 51.1%
  33. 33 DeepSeek-V3 49.6%
  34. 34 Grok-3 mini high 49.3%
  35. 35 GPT-5 nano 48.4%
  36. 36 Magistral Medium 1.0 47.1%
  37. 37 GPT-4o Mar 2025 45.3%
  38. 38 GPT-4.5 44.9%
  39. 39 gpt-oss-120b 44.4%
  40. 40 Qwen3-32B 40%
  41. 41 Gemini 2.0 Pro 35.6%
  42. 42 Code World Model (CWM) 35.1%
  43. 43 GPT-4.1 mini 34.7%
  44. 44 gpt-oss-20b 34.2%
  45. 45 o1-mini 32.9%
  46. 46 Claude 3.5 Haiku 28%
  47. 47 GPT-4o Jan 2025 27.1%
  48. 48 Gemini 2.5 Flash-Lite Jun 2025 27.1%
  49. 49 GPT-4o Aug 2024 23.1%
  50. 50 Gemini 2.0 Flash Dec 2024 22.2%
  51. 51 QwQ-32B 20.9%
  52. 52 GPT-4o Nov 2024 18.2%
  53. 53 Gemini 2.0 Flash Thinking Jan 2025 18.2%
  54. 54 Qwen2.5-Coder-32B-Instruct 16.4%
  55. 55 Llama 4 Maverick 15.6%
  56. 56 Cohere Command A 14.7%
  57. 57 Yi-Lightning 12.9%
  58. 58 Codestral 11.1%
  59. 59 openhands-lm-32b-v0.1 10.2%
  60. 60 GPT-4.1 nano 9.80%
  61. 61 Gemma 3 27B 4.90%
  62. 62 GPT-4o mini 3.60%

Sources: Official Aider Polyglot leaderboard and labs’ model cards

#ModelScoreSettingDateSource
1 GPT-5 88% Aug 2025 Lab model card
2 Gemini 2.5 Pro Jun 2025 82.2% Jun 2025 Lab model card
3 o3 81.3% Apr 2025 Lab model card
4 Seed2.0 Pro 80% Feb 2026 Lab model card
5 DeepSeek-V3.1 76.3% Aug 2025 Lab model card
6 DeepSeek-V3.1-Terminus 76.1% Sep 2025 Lab model card
7 DeepSeek-V3.2-Exp 74.5% Sep 2025 Lab model card
8 Gemini 2.5 Pro Mar 2025 74% Mar 2025 Lab model card
9 DeepSeek-R1 May 2025 71.6% May 2025 Lab model card
10 GPT-5 mini 71.6% Aug 2025 Lab model card
11 o4-mini 68.9% Apr 2025 Lab model card
12 Qwen3-Coder-Next 66.2% Feb 2026 Lab model card
13 Gemini 2.5 Flash Jun 2025 61.9% Jun 2025 Lab model card
14 Qwen3-Coder-480B-A35B 61.8% Jul 2025 Lab model card
15 Kimi K2 Jul 2025 60% Jul 2025 Lab model card
16 gemini-robotics-er-1.5-preview 57.3% Sep 2025 Lab model card
17 Seed-Thinking-v1.5 54.2% Apr 2025 Lab model card
18 DeepSeek-R1 53.3% Jan 2025 Lab model card
19 GPT-4.1 52.9% Apr 2025 Lab model card
20 Gemini 2.5 Flash Apr 2025 51.1% Apr 2025 Lab model card
21 DeepSeek-V3 49.6% Dec 2024 Lab model card
22 GPT-5 nano 48.4% Aug 2025 Lab model card
23 Magistral Medium 1.0 47.1% Jun 2025 Lab model card
24 gpt-oss-120b 44.4% Aug 2025 Lab model card
25 Code World Model (CWM) 35.1% Sep 2025 Lab model card
26 GPT-4.1 mini 34.7% Apr 2025 Lab model card
27 gpt-oss-20b 34.2% Aug 2025 Lab model card
28 Gemini 2.5 Flash-Lite Jun 2025 27.1% Jun 2025 Lab model card
29 Cohere Command A 14.7% Apr 2025 Lab model card
30 GPT-4.1 nano 9.80% Apr 2025 Lab model card
#ModelScoreSettingDateSource
1 GPT-5 88% high, aider, diff edits Aug 2025 Official leaderboard
2 o3-pro 84.9% high, aider, diff edits Jun 2025 Official leaderboard
3 Gemini 2.5 Pro Jun 2025 83.1% aider, diff-fenced edits Jun 2025 Official leaderboard
4 o3 81.3% high, aider, diff edits Apr 2025 Official leaderboard
5 Grok 4 79.6% high, aider, diff edits Jul 2025 Official leaderboard
6 Gemini 2.5 Pro May 2025 76.9% aider, diff-fenced edits May 2025 Official leaderboard
7 Gemini 2.5 Pro Mar 2025 72.9% aider, diff-fenced edits Mar 2025 Official leaderboard
8 o4-mini 72% high, aider, diff edits Apr 2025 Official leaderboard
9 Claude Opus 4 72% 32k thinking, aider, diff edits May 2025 Official leaderboard
10 DeepSeek-R1 May 2025 71.4% aider, diff edits May 2025 Official leaderboard
11 Claude 3.7 Sonnet 64.9% 32k thinking tokens, aider, diff edits Feb 2025 Official leaderboard
12 o1 61.7% high, aider, diff edits Dec 2024 Official leaderboard
13 Claude Sonnet 4 61.3% 32k thinking, aider, diff edits May 2025 Official leaderboard
14 o3-mini 60.4% high, aider, diff edits Jan 2025 Official leaderboard
15 Kimi K2 Jul 2025 59.1% aider, diff edits Jul 2025 Official leaderboard
16 DeepSeek-R1 56.9% aider, diff edits Jan 2025 Official leaderboard
17 DeepSeek-V3 Mar 2025 55.1% aider, diff edits Mar 2025 Official leaderboard
18 Gemini 2.5 Flash May 2025 55.1% aider, diff edits May 2025 Official leaderboard
19 Grok 3 53.3% aider, diff edits Apr 2025 Official leaderboard
20 GPT-4.1 52.4% aider, diff edits Apr 2025 Official leaderboard
21 Claude 3.5 Sonnet 51.6% aider, diff edits Jun 2024 Official leaderboard
22 Grok-3 mini 49.3% high, aider, whole edits Apr 2025 Official leaderboard
23 Gemini 2.5 Flash Apr 2025 47.1% aider, diff edits Apr 2025 Official leaderboard
24 GPT-4o Mar 2025 45.3% aider, diff edits Mar 2025 Official leaderboard
25 GPT-4.5 44.9% aider, diff edits Feb 2025 Official leaderboard
26 gpt-oss-120b 41.8% high, aider, diff edits Aug 2025 Official leaderboard
27 Qwen3-32B 40% aider, diff edits Apr 2025 Official leaderboard
28 Gemini 2.0 Pro 35.6% aider, whole edits Feb 2025 Official leaderboard
29 o1-mini 32.9% aider, whole edits Sep 2024 Official leaderboard
30 GPT-4.1 mini 32.4% aider, diff edits Apr 2025 Official leaderboard
31 Claude 3.5 Haiku 28% aider, diff edits Oct 2024 Official leaderboard
32 GPT-4o Jan 2025 27.1% aider, diff edits Jan 2025 Official leaderboard
33 GPT-4o Aug 2024 23.1% aider, diff edits Aug 2024 Official leaderboard
34 Gemini 2.0 Flash Dec 2024 22.2% aider, whole edits Dec 2024 Official leaderboard
35 QwQ-32B 20.9% aider, diff edits Mar 2025 Official leaderboard
36 GPT-4o Nov 2024 18.2% aider, diff edits Nov 2024 Official leaderboard
37 Gemini 2.0 Flash Thinking Jan 2025 18.2% aider, diff edits Jan 2025 Official leaderboard
38 Qwen2.5-Coder-32B-Instruct 16.4% aider, whole edits Nov 2024 Official leaderboard
39 Llama 4 Maverick 15.6% aider, whole edits Apr 2025 Official leaderboard
40 Yi-Lightning 12.9% aider, whole edits Dec 2024 Official leaderboard
41 Cohere Command A 12% aider, whole edits Mar 2025 Official leaderboard
42 Codestral 11.1% aider, whole edits Jan 2025 Official leaderboard
43 openhands-lm-32b-v0.1 10.2% aider, whole edits Mar 2024 Official leaderboard
44 GPT-4.1 nano 8.90% aider, whole edits Apr 2025 Official leaderboard
45 Gemma 3 27B 4.90% aider, whole edits Mar 2025 Official leaderboard
46 GPT-4o mini 3.60% aider, whole edits Jul 2024 Official leaderboard
#ModelScoreSettingDateSource
1 GPT-5 88% high, aider, diff edits Aug 2025 Official leaderboard
2 o3-pro 84.9% high, aider, diff edits Jun 2025 Official leaderboard
3 Gemini 2.5 Pro Jun 2025 83.1% aider, diff-fenced edits Jun 2025 Official leaderboard
4 o3 81.3% high, aider, diff edits Apr 2025 Official leaderboard
5 Seed2.0 Pro 80% Feb 2026 Lab model card
6 Grok 4 79.6% high, aider, diff edits Jul 2025 Official leaderboard
7 Gemini 2.5 Pro May 2025 76.9% aider, diff-fenced edits May 2025 Official leaderboard
8 DeepSeek-V3.1 76.3% Aug 2025 Lab model card
9 DeepSeek-V3.1-Terminus 76.1% Sep 2025 Lab model card
10 DeepSeek-V3.2-Exp 74.5% Sep 2025 Lab model card
11 Gemini 2.5 Pro Mar 2025 74% Mar 2025 Lab model card
12 o4-mini 72% high, aider, diff edits Apr 2025 Official leaderboard
13 Claude Opus 4 72% 32k thinking, aider, diff edits May 2025 Official leaderboard
14 DeepSeek-R1 May 2025 71.6% May 2025 Lab model card
15 GPT-5 mini 71.6% Aug 2025 Lab model card
16 Qwen3-Coder-Next 66.2% Feb 2026 Lab model card
17 Claude 3.7 Sonnet 64.9% 32k thinking tokens, aider, diff edits Feb 2025 Official leaderboard
18 Gemini 2.5 Flash Jun 2025 61.9% Jun 2025 Lab model card
19 Qwen3-Coder-480B-A35B 61.8% Jul 2025 Lab model card
20 o1 61.7% high, aider, diff edits Dec 2024 Official leaderboard
21 Claude Sonnet 4 61.3% 32k thinking, aider, diff edits May 2025 Official leaderboard
22 o3-mini 60.4% high, aider, diff edits Jan 2025 Official leaderboard
23 Kimi K2 Jul 2025 60% Jul 2025 Lab model card
24 gemini-robotics-er-1.5-preview 57.3% Sep 2025 Lab model card
25 DeepSeek-R1 56.9% aider, diff edits Jan 2025 Official leaderboard
26 DeepSeek-V3 Mar 2025 55.1% aider, diff edits Mar 2025 Official leaderboard
27 Gemini 2.5 Flash May 2025 55.1% aider, diff edits May 2025 Official leaderboard
28 Seed-Thinking-v1.5 54.2% Apr 2025 Lab model card
29 Grok 3 53.3% aider, diff edits Apr 2025 Official leaderboard
30 GPT-4.1 52.9% Apr 2025 Lab model card
31 Claude 3.5 Sonnet 51.6% aider, diff edits Jun 2024 Official leaderboard
32 Gemini 2.5 Flash Apr 2025 51.1% Apr 2025 Lab model card
33 DeepSeek-V3 49.6% Dec 2024 Lab model card
34 Grok-3 mini 49.3% high, aider, whole edits Apr 2025 Official leaderboard
35 GPT-5 nano 48.4% Aug 2025 Lab model card
36 Magistral Medium 1.0 47.1% Jun 2025 Lab model card
37 GPT-4o Mar 2025 45.3% aider, diff edits Mar 2025 Official leaderboard
38 GPT-4.5 44.9% aider, diff edits Feb 2025 Official leaderboard
39 gpt-oss-120b 44.4% Aug 2025 Lab model card
40 Qwen3-32B 40% aider, diff edits Apr 2025 Official leaderboard
41 Gemini 2.0 Pro 35.6% aider, whole edits Feb 2025 Official leaderboard
42 Code World Model (CWM) 35.1% Sep 2025 Lab model card
43 GPT-4.1 mini 34.7% Apr 2025 Lab model card
44 gpt-oss-20b 34.2% Aug 2025 Lab model card
45 o1-mini 32.9% aider, whole edits Sep 2024 Official leaderboard
46 Claude 3.5 Haiku 28% aider, diff edits Oct 2024 Official leaderboard
47 GPT-4o Jan 2025 27.1% aider, diff edits Jan 2025 Official leaderboard
48 Gemini 2.5 Flash-Lite Jun 2025 27.1% Jun 2025 Lab model card
49 GPT-4o Aug 2024 23.1% aider, diff edits Aug 2024 Official leaderboard
50 Gemini 2.0 Flash Dec 2024 22.2% aider, whole edits Dec 2024 Official leaderboard
51 QwQ-32B 20.9% aider, diff edits Mar 2025 Official leaderboard
52 GPT-4o Nov 2024 18.2% aider, diff edits Nov 2024 Official leaderboard
53 Gemini 2.0 Flash Thinking Jan 2025 18.2% aider, diff edits Jan 2025 Official leaderboard
54 Qwen2.5-Coder-32B-Instruct 16.4% aider, whole edits Nov 2024 Official leaderboard
55 Llama 4 Maverick 15.6% aider, whole edits Apr 2025 Official leaderboard
56 Cohere Command A 14.7% Apr 2025 Lab model card
57 Yi-Lightning 12.9% aider, whole edits Dec 2024 Official leaderboard
58 Codestral 11.1% aider, whole edits Jan 2025 Official leaderboard
59 openhands-lm-32b-v0.1 10.2% aider, whole edits Mar 2024 Official leaderboard
60 GPT-4.1 nano 9.80% Apr 2025 Lab model card
61 Gemma 3 27B 4.90% aider, whole edits Mar 2025 Official leaderboard
62 GPT-4o mini 3.60% aider, whole edits Jul 2024 Official leaderboard

The benchmark's own leaderboard has 58 entries, 46 of them models we track. Epoch AI collected claims for 1 model from labs’ reports and papers. Switch the scores above to rank them together.

The best score, 88%, costs $29 for the full run

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI DeepSeek xAI Moonshot AI Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 $1,000 Cost of the full run, log scale GPT-5, high GPT-5, low DeepSeek-R1 Kimi K2 DeepSeek-V3 Grok-3 mini, high GPT-4.1 nano 0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 $1,000 Cost of the full run, log scale GPT-5, high GPT-5, low

Source: Official Aider Polyglot leaderboard

ModelEffortHarnessScoreCost of the full runOutput tokens
GPT-5 high aider, diff edits 88% $29 11,660
GPT-5 medium aider, diff edits 86.7% $18 6,528
o3-pro high aider, diff edits 84.9% $146 5,493
Gemini 2.5 Pro Jun 2025 aider, diff-fenced edits 83.1% $50 20,659
o3 high aider, diff edits 81.3% $21 9,101
GPT-5 low aider, diff edits 81.3% $10 3,465
Grok 4 high aider, diff edits 79.6% $60 15,162
Gemini 2.5 Pro Jun 2025 aider, diff-fenced edits 79.1% $46 18,410
o3 aider, diff edits 76.9% $14 5,132
Gemini 2.5 Pro May 2025 aider, diff-fenced edits 76.9% $37
Gemini 2.5 Pro Mar 2025 aider, diff-fenced edits 72.9%
o4-mini high aider, diff edits 72% $20
Claude Opus 4 32k thinking aider, diff edits 72% $66 1,614
DeepSeek-R1 May 2025 aider, diff edits 71.4% $4.80 8,224
Claude Opus 4 aider, diff edits 70.7% $69 1,692
Claude 3.7 Sonnet 32k thinking tokens aider, diff edits 64.9% $37
o1 high aider, diff edits 61.7% $187
Claude Sonnet 4 32k thinking aider, diff edits 61.3% $27 5,649
o3-mini high aider, diff edits 60.4% $18
Claude 3.7 Sonnet no thinking aider, diff edits 60.4% $18
Kimi K2 Jul 2025 aider, diff edits 59.1% $1.24 1,617
DeepSeek-R1 aider, diff edits 56.9% $5.42
Claude Sonnet 4 no thinking aider, diff edits 56.4% $16 1,926
DeepSeek-V3 Mar 2025 aider, diff edits 55.1% $1.12
Gemini 2.5 Flash May 2025 aider, diff edits 55.1% $8.56 12,014
o3-mini medium aider, diff edits 53.8% $8.86
Grok 3 aider, diff edits 53.3% $11
GPT-4.1 aider, diff edits 52.4% $9.86
Claude 3.5 Sonnet aider, diff edits 51.6% $14
Grok-3 mini high aider, whole edits 49.3% $0.73
Gemini 2.5 Flash Apr 2025 aider, diff edits 47.1% $1.84
GPT-4o Mar 2025 aider, diff edits 45.3% $20
GPT-4.5 aider, diff edits 44.9% $183
Gemini 2.5 Flash May 2025 aider, diff edits 44% $1.14 2,285
gpt-oss-120b high aider, diff edits 41.8% $0.74 3,807
Qwen3-32B aider, diff edits 40% $0.76 535
Gemini 2.0 Pro aider, whole edits 35.6%
Grok-3 mini low aider, whole edits 34.7% $0.79
o1-mini aider, whole edits 32.9% $19
GPT-4.1 mini aider, diff edits 32.4% $1.99
Claude 3.5 Haiku aider, diff edits 28% $6.06
GPT-4o Jan 2025 aider, diff edits 27.1% $14
GPT-4o Aug 2024 aider, diff edits 23.1% $7.03
Gemini 2.0 Flash Dec 2024 aider, whole edits 22.2%
QwQ-32B aider, diff edits 20.9%
GPT-4o Nov 2024 aider, diff edits 18.2% $6.74
Gemini 2.0 Flash Thinking Jan 2025 aider, diff edits 18.2%
Qwen2.5-Coder-32B-Instruct aider, whole edits 16.4%
Llama 4 Maverick aider, whole edits 15.6%
Yi-Lightning aider, whole edits 12.9%
Cohere Command A aider, whole edits 12%
Codestral aider, whole edits 11.1% $1.98
openhands-lm-32b-v0.1 aider, whole edits 10.2%
GPT-4.1 nano aider, whole edits 8.90% $0.43
Qwen2.5-Coder-32B-Instruct aider, diff edits 8.00%
Gemma 3 27B aider, whole edits 4.90%
GPT-4o mini aider, whole edits 3.60% $0.32

6 models have held number one in labs’ own numbers since its release in December 2024 8 models have held number one across all independent runs since its release in December 2024 8 models have held number one with claims included since its release in December 2024

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic OpenAI Google DeepMind DeepSeek
Own leaderboard Passed 85% in Aug 2025, by GPT-5
GPT-5 88%
Lab claims Passed 85% in Aug 2025, by GPT-5
GPT-5 88%

Sources: Official Aider Polyglot leaderboard and labs’ model cards

GPT-5 on the official leaderboard passed 85% in August 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card A claim Epoch AI collected from a report or paper Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) GPT-5 Gemini 2.5 Pro Gemini 2.5 Pro Claude 3.7 Sonnet Claude 3.5 Sonnet openhands-lm-32b-v0.1 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) GPT-5 o3

Sources: Official Aider Polyglot leaderboard, labs’ model cards and claims collected by Epoch AI (CC BY 4.0)

Labs headline it less often now, last in February 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

DeepSeek
Google DeepMind
ByteDance Seed
OpenAI
Qwen
Mistral AI
Moonshot AI
DateLaunch postScoreAs the post writes it
Feb 14, 2026 Seed2.0 Pro 80 Aider Polyglot
Feb 2, 2026 Qwen3-Coder-Next 66.2 Aider
Sep 29, 2025 DeepSeek-V3.2-Exp 74.5 Aider-Polyglot
Sep 22, 2025 DeepSeek-V3.1-Terminus 76.1 Aider-Polyglot
Aug 21, 2025 DeepSeek-V3.1 76.3 Aider-Polyglot (Acc.)
Aug 7, 2025 GPT-5 nano 48.4 Aider polyglot (diff)
Aug 7, 2025 GPT-5 mini 71.6 Aider polyglot (diff)
Aug 7, 2025 GPT-5 88 Aider Polyglot Multi-language code editing
Jul 22, 2025 Qwen3-Coder-480B-A35B 61.8 Aider-Polyglot
Jul 11, 2025 Kimi K2 60 Aider-Polyglot, Acc
Jun 17, 2025 Gemini 2.5 Flash 61.9 Aider Polyglot whole 61.9 / diff 56.7
Jun 17, 2025 Gemini 2.5 Flash-Lite 27.1 Aider Polyglot (whole), Thinking
Jun 17, 2025 Gemini 2.5 Pro 82.2 Aider Polyglot (diff-fenced)
Jun 10, 2025 Magistral Medium 1.0 47.1 Aider-Polyglot
May 28, 2025 DeepSeek-R1 71.6 Aider-Polyglot (Acc.)
Apr 29, 2025 Qwen3-235B-A22B Aider (Pass@2)
Apr 17, 2025 Gemini 2.5 Flash 51.1 Aider Polyglot whole 51.1 / diff-fenced 44.2
Apr 16, 2025 o4-mini 68.9 Aider Polyglot whole 68.9 (diff 58.2), o4-mini-high
Apr 16, 2025 o3 81.3 Aider Polyglot Code Editing, o3-high (whole)
Apr 14, 2025 GPT-4.1 nano 9.8 Aider's polyglot: whole 9.8 (diff 6.2)
Apr 14, 2025 GPT-4.1 mini 34.7 Aider's polyglot: whole 34.7 (diff 31.6)
Apr 14, 2025 GPT-4.1 52.9 Aider's polyglot: diff
Apr 10, 2025 Seed-Thinking-v1.5 54.2 Aider Polyglot
Mar 25, 2025 Gemini 2.5 Pro 74 Aider Polyglot, whole (diff = 68.6%)
Jan 20, 2025 DeepSeek-R1 53.3 Aider-Polyglot (Acc.)
Dec 26, 2024 DeepSeek-V3 49.6 Aider-Polyglot (Acc.)

17 system cards report it, from 9 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Feb 28, 2026 Qwen3-Coder-Next technical report 66.2 Aider-Polyglot (Table 7) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 80 Aider Polyglot (Table 11) Capabilities
Sep 30, 2025 Code World Model (CWM) technical report 35.1 Aider Polyglot, whole file format, pass 1@2 (Table 6) Capabilities
Sep 29, 2025 DeepSeek-V3.2-Exp technical report 74.5 Aider-Polyglot (Acc.), DeepSeek-V3.2-Exp (Table 1) Capabilities
Sep 25, 2025 gemini-robotics-er-1.5-preview technical report 57.3 Aider Polyglot (Table 20, thinking on). Thinking off 44.4 Capabilities
Aug 5, 2025 gpt-oss-20b model card 34.2 Aider Polyglot, high reasoning (Table 3) Capabilities
Aug 5, 2025 gpt-oss-120b model card 44.4 Aider Polyglot, high reasoning (Table 3) Capabilities
Jul 21, 2025 Kimi K2 technical report 60 Aider-Polyglot (Acc.) (Table 3) Capabilities
Jul 7, 2025 Gemini 2.5 Pro technical report 82.2 Aider Polyglot, diff or diff-fenced format, average of 3 trials (Table 3) Capabilities
Jul 7, 2025 Gemini 2.5 Flash technical report 56.7 Aider Polyglot, diff or diff-fenced format, average of 3 trials (Table 3) Capabilities
Jun 12, 2025 Magistral Medium 1.0 technical report 47.1 Aider Polyglot (Table 2) Capabilities
Apr 29, 2025 Gemini 2.5 Flash model card 51.1 Aider Polyglot, whole 51.1, diff 44.2 (Table, p.4) Capabilities
Apr 28, 2025 Gemini 2.5 Pro model card 74 Aider Polyglot, whole edit format; diff format = 68.6% (Table, p.4, column Gemini 2.5 Pro Experimental 03-25) Capabilities
Apr 10, 2025 Seed-Thinking-v1.5 technical report 54.2 Aider Polyglot (Table 2) Capabilities
Apr 1, 2025 Cohere Command A technical report 14.7 Aider Polyglot (Table 13) Capabilities
Jan 20, 2025 DeepSeek-R1 technical report 53.3 Aider-Polyglot (Acc.), DeepSeek-R1 (Table 8, same as final column of Table 3) Capabilities
Dec 26, 2024 DeepSeek-V3 technical report 49.6 Aider-Polyglot (Acc.), DeepSeek-V3 (Table 6) Capabilities

The lab's number comes out ahead in 9 of 12 pairs that differ

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 2 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Gemini 2.5 Flash Apr 2025 51.1% Aider Polyglot whole 51.1 / diff-fenced 44.2 47.1% Leaderboard, aider, diff edits +4.0 pts
DeepSeek-R1 53.3% Aider-Polyglot (Acc.) 56.9% Leaderboard, aider, diff edits −3.6 pts
o4-mini 68.9% Aider Polyglot whole 68.9 (diff 58.2), o4-mini-high 72% Leaderboard, high, aider, diff edits −3.1 pts
Cohere Command A 14.7% Aider Polyglot (Table 13) 12% Leaderboard, aider, whole edits +2.7 pts
gpt-oss-120b 44.4% Aider Polyglot, high reasoning (Table 3) 41.8% Leaderboard, high, aider, diff edits +2.6 pts
GPT-4.1 mini 34.7% Aider's polyglot: whole 34.7 (diff 31.6) 32.4% Leaderboard, aider, diff edits +2.3 pts
Gemini 2.5 Pro Mar 2025 74% Aider Polyglot, whole (diff = 68.6%) 72.9% Leaderboard, aider, diff-fenced edits +1.1 pts
GPT-4.1 nano 9.80% Aider's polyglot: whole 9.8 (diff 6.2) 8.90% Leaderboard, aider, whole edits +0.9 pts
Kimi K2 Jul 2025 60% Aider-Polyglot, Acc 59.1% Leaderboard, aider, diff edits +0.9 pts
Gemini 2.5 Pro Jun 2025 82.2% Aider Polyglot (diff-fenced) 83.1% Leaderboard, aider, diff-fenced edits −0.9 pts
GPT-4.1 52.9% Aider's polyglot: diff 52.4% Leaderboard, aider, diff edits +0.5 pts
DeepSeek-R1 May 2025 71.6% Aider-Polyglot (Acc.) 71.4% Leaderboard, aider, diff edits +0.2 pts
o3 81.3% Aider Polyglot Code Editing, o3-high (whole) 81.3% Leaderboard, high, aider, diff edits Same
GPT-5 88% Aider Polyglot Multi-language code editing 88% Leaderboard, high, aider, diff edits Same

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

DeepSeek Google DeepMind OpenAI
Aider Polyglot Passed 85% in Aug 2025, by GPT-5
GPT-5 88%
Aider-Edit Not tested since Oct 2025
DeepSeek-V3 79.7%
Aider Polyglot Passed 85% in Aug 2025, by GPT-5
GPT-5 88%

Source: Labs’ model cards

Replaces Aider-Edit

More in Coding

All 282 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471