Benchmark Almanac

Terminal-Bench 4.0

Claude Opus 5.5 64.8% leads since September 2026
Claude Sonnet 5.5 70.6% leads since September 2026
In use

Agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate.

It removed 8 saturated or flawed tasks from Terminal-Bench 3.0 and set a flat 8-hour agent timeout.

Released
August 2026
Kind
Terminal tasks
Level
Expert
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Strongest model tested
Claude Fable 5.2 #1 on the capability index
Last in a launch post
November 2026
Last model added
November 2026
Scores refreshed
November 13th, 2026
40
Models scored on its own leaderboard
26
Launch posts list it, from 16 labs
13
System cards report it, from 6 labs

Claude Opus 5.5 leads with 64.8%

Best score per model from the benchmark's own leaderboard, read on November 13th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

Claude Sonnet 5.5 leads with 70.6%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

40 models. Scroll sideways for the rest.

  1. 64.8 Claude Opus 5.5
  2. 61.8 Claude Sonnet 5.5
  3. 60.9 Kimi K3.1 Pro
  4. 58.2 GPT-6 Astra
  5. 58.2 GPT-6.1 Sol
  6. 57.9 Claude Fable 5.1
  7. 53.9 Claude Opus 5
  8. 51.8 GPT-6.1 Terra
  9. 49.4 GPT-6 Sol
  10. 47.0 Muse Wonder
  11. 44.5 Claude Fable 5
  12. 41.8 GLM-5.3
  13. 40.0 Claude Haiku 5.5
  14. 40.0 DeepSeek V4.1 Flash 1017
  15. 39.1 Tencent Hy4
  16. 37.6 Grok 4.7
  17. 37.3 GPT-5.6 Sol
  18. 36.7 GLM-5.3-Flash
  19. 27.9 Mistral Large 4
  20. 27.0 Qwen3.8 Max (0902)
  21. 27.0 MiMo-V2.6 Pro
  22. 23.6 Claude Opus 4.8
  23. 22.4 Kimi K3
  24. 21.5 GPT-5.6 Terra
  25. 20.3 Grok 4.6
  26. 19.1 Gemini 3.8 Flash
  27. 17.3 GPT-5.6 Luna
  28. 16.4 GPT-6 Luna
  29. 16.1 MiMo-V2.6 Flash
  30. 14.6 Muse Spark 1.3
  31. 12.4 Claude Sonnet 5
  32. 12.4 Grok 4.5
  33. 11.2 Gemini 3.7 Flash
  34. 7.27 MiniMax-M3
  35. 6.67 DeepSeek V4 Pro 0813
  36. 6.36 Beam 1.0
  37. 1.82 GPT-5.4 Mini
  38. 0.91 Inkling-Small
  39. 0.30 Nemotron 3.5 Super VL
  40. 0.0 Inkling
  1. 1 Claude Opus 5.5 max 64.8%
  2. 2 Claude Sonnet 5.5 max 61.8%
  3. 3 Kimi K3.1 Pro max 60.9%
  4. 4 GPT-6 Astra max 58.2%
  5. 5 GPT-6.1 Sol max 58.2%
  6. 6 Claude Fable 5.1 max 57.9%
  7. 7 Claude Opus 5 xhigh 53.9%
  8. 8 GPT-6.1 Terra max 51.8%
  9. 9 GPT-6 Sol max 49.4%
  10. 10 Muse Wonder xhigh 47%
  11. 11 Claude Fable 5 max 44.5%
  12. 12 GLM-5.3 max 41.8%
  13. 13 Claude Haiku 5.5 max 40%
  14. 14 DeepSeek V4.1 Flash 1017 max 40%
  15. 15 Tencent Hy4 max 39.1%
  16. 16 Grok 4.7 xhigh 37.6%
  17. 17 GPT-5.6 Sol max 37.3%
  18. 18 GLM-5.3-Flash max 36.7%
  19. 19 Mistral Large 4 max 27.9%
  20. 20 Qwen3.8 Max (0902) max 27%
  21. 21 MiMo-V2.6 Pro none 27%
  22. 22 Claude Opus 4.8 max 23.6%
  23. 23 Kimi K3 max 22.4%
  24. 24 GPT-5.6 Terra max 21.5%
  25. 25 Grok 4.6 high 20.3%
  26. 26 Gemini 3.8 Flash high 19.1%
  27. 27 GPT-5.6 Luna max 17.3%
  28. 28 GPT-6 Luna max 16.4%
  29. 29 MiMo-V2.6 Flash none 16.1%
  30. 30 Muse Spark 1.3 xhigh 14.6%
  31. 31 Claude Sonnet 5 max 12.4%
  32. 32 Grok 4.5 high 12.4%
  33. 33 Gemini 3.7 Flash high 11.2%
  34. 34 MiniMax-M3 none 7.27%
  35. 35 DeepSeek V4 Pro 0813 max 6.67%
  36. 36 Beam 1.0 none 6.36%
  37. 37 GPT-5.4 Mini xhigh 1.82%
  38. 38 Inkling-Small none 0.91%
  39. 39 Nemotron 3.5 Super VL none 0.30%
  40. 40 Inkling none 0%

Source: Official Terminal-Bench 4.0 leaderboard

53 models. Scroll sideways for the rest.

  1. 70.6 Claude Sonnet 5.5
  2. 67.6 Claude Fable 5.2
  3. 66.4 Claude Opus 5.5
  4. 65.7 Kimi K3.1 Pro
  5. 65.2 Gemini 4 Carbon
  6. 58.2 GPT-6 Astra
  7. 58.2 GPT-6.1 Sol
  8. 57.9 Claude Fable 5.1
  9. 57.4 Gemini 4 Argon
  10. 57.1 Muse Wonder Full
  11. 53.9 Claude Opus 5
  12. 52.0 Muse Wonder
  13. 51.8 GPT-6.1 Terra
  14. 49.4 GPT-6 Sol
  15. 46.0 Ling-3.1-flash
  16. 44.5 Claude Fable 5
  17. 44.4 Tencent Hy4
  18. 42.3 DeepSeek V4.1 Flash 1017
  19. 41.8 GLM-5.3
  20. 41.0 Mistral Large 4 (Le Chonk, open weights)
  21. 40.9 Muse Ember
  22. 40.0 Claude Haiku 5.5
  23. 38.5 LongCat-2.5
  24. 38.1 MiniMax-M3.1-Flash
  25. 38.0 Grok 4.7
  26. 37.3 GPT-5.6 Sol
  27. 36.7 GLM-5.3-Flash
  28. 34.9 MiMo-V2.6 Pro
  29. 33.3 Step 5 Preview
  30. 31.2 DeepSeek V4.1 Flash
  31. 28.3 Mistral Large 4
  32. 27.3 SWE-2
  33. 27.0 Qwen3.8 Max (0902)
  34. 23.6 Claude Opus 4.8
  35. 22.4 Kimi K3
  36. 21.5 GPT-5.6 Terra
  37. 20.3 Grok 4.6
  38. 20.0 SWE-2 Flash
  39. 19.1 Gemini 3.8 Flash
  40. 17.3 GPT-5.6 Luna
  41. 16.4 GPT-6 Luna
  42. 16.1 MiMo-V2.6 Flash
  43. 14.6 Muse Spark 1.3
  44. 12.4 Claude Sonnet 5
  45. 12.4 Grok 4.5
  46. 11.2 Gemini 3.7 Flash
  47. 7.27 MiniMax-M3
  48. 6.67 DeepSeek V4 Pro 0813
  49. 6.36 Beam 1.0
  50. 1.82 GPT-5.4 Mini
  51. 1.00 Nemotron 3.5 Super VL
  52. 0.91 Inkling-Small
  53. 0.0 Inkling
  1. 1 Claude Sonnet 5.5 70.6%
  2. 2 Claude Fable 5.2 67.6%
  3. 3 Claude Opus 5.5 66.4%
  4. 4 Kimi K3.1 Pro 65.7%
  5. 5 Gemini 4 Carbon 65.2%
  6. 6 GPT-6 Astra max 58.2%
  7. 7 GPT-6.1 Sol max 58.2%
  8. 8 Claude Fable 5.1 max 57.9%
  9. 9 Gemini 4 Argon 57.4%
  10. 10 Muse Wonder Full 57.1%
  11. 11 Claude Opus 5 xhigh 53.9%
  12. 12 Muse Wonder 52%
  13. 13 GPT-6.1 Terra max 51.8%
  14. 14 GPT-6 Sol max 49.4%
  15. 15 Ling-3.1-flash 46%
  16. 16 Claude Fable 5 max 44.5%
  17. 17 Tencent Hy4 44.4%
  18. 18 DeepSeek V4.1 Flash 1017 42.3%
  19. 19 GLM-5.3 max 41.8%
  20. 20 Mistral Large 4 (Le Chonk, open weights) 41%
  21. 21 Muse Ember 40.9%
  22. 22 Claude Haiku 5.5 max 40%
  23. 23 LongCat-2.5 38.5%
  24. 24 MiniMax-M3.1-Flash 38.1%
  25. 25 Grok 4.7 38%
  26. 26 GPT-5.6 Sol max 37.3%
  27. 27 GLM-5.3-Flash max 36.7%
  28. 28 MiMo-V2.6 Pro 34.9%
  29. 29 Step 5 Preview 33.3%
  30. 30 DeepSeek V4.1 Flash 31.2%
  31. 31 Mistral Large 4 28.3%
  32. 32 SWE-2 27.3%
  33. 33 Qwen3.8 Max (0902) max 27%
  34. 34 Claude Opus 4.8 max 23.6%
  35. 35 Kimi K3 max 22.4%
  36. 36 GPT-5.6 Terra max 21.5%
  37. 37 Grok 4.6 high 20.3%
  38. 38 SWE-2 Flash 20%
  39. 39 Gemini 3.8 Flash 19.1%
  40. 40 GPT-5.6 Luna max 17.3%
  41. 41 GPT-6 Luna max 16.4%
  42. 42 MiMo-V2.6 Flash none 16.1%
  43. 43 Muse Spark 1.3 xhigh 14.6%
  44. 44 Claude Sonnet 5 max 12.4%
  45. 45 Grok 4.5 high 12.4%
  46. 46 Gemini 3.7 Flash high 11.2%
  47. 47 MiniMax-M3 none 7.27%
  48. 48 DeepSeek V4 Pro 0813 max 6.67%
  49. 49 Beam 1.0 none 6.36%
  50. 50 GPT-5.4 Mini xhigh 1.82%
  51. 51 Nemotron 3.5 Super VL 1.00%
  52. 52 Inkling-Small none 0.91%
  53. 53 Inkling none 0%

Sources: Labs’ model cards and official Terminal-Bench 4.0 leaderboard

#ModelScoreSettingDateSource
1 Claude Opus 5.5 64.8% max, Claude Code Sep 2026 Official leaderboard
2 Claude Sonnet 5.5 61.8% max, Claude Code Sep 2026 Official leaderboard
3 Kimi K3.1 Pro 60.9% max, Claude Code Oct 2026 Official leaderboard
4 GPT-6 Astra 58.2% max, Codex Sep 2026 Official leaderboard
5 GPT-6.1 Sol 58.2% max, Codex Sep 2026 Official leaderboard
6 Claude Fable 5.1 57.9% max, Claude Code Sep 2026 Official leaderboard
7 Claude Opus 5 53.9% xhigh, Claude Code Jul 2026 Official leaderboard
8 GPT-6.1 Terra 51.8% max, Codex Oct 2026 Official leaderboard
9 GPT-6 Sol 49.4% max, Codex Sep 2026 Official leaderboard
10 Muse Wonder 47% xhigh, Muse Code Oct 2026 Official leaderboard
11 Claude Fable 5 44.5% max, Claude Code Jun 2026 Official leaderboard
12 GLM-5.3 41.8% max, Claude Code Aug 2026 Official leaderboard
13 Claude Haiku 5.5 40% max, Claude Code Oct 2026 Official leaderboard
14 DeepSeek V4.1 Flash 1017 40% max, Claude Code Oct 2026 Official leaderboard
15 Tencent Hy4 39.1% max, Claude Code Nov 2026 Official leaderboard
16 Grok 4.7 37.6% xhigh, Grok Build Sep 2026 Official leaderboard
17 GPT-5.6 Sol 37.3% max, Codex Jul 2026 Official leaderboard
18 GLM-5.3-Flash 36.7% max, Claude Code Aug 2026 Official leaderboard
19 Mistral Large 4 27.9% max, Claude Code Oct 2026 Official leaderboard
20 Qwen3.8 Max (0902) 27% max, Claude Code Sep 2026 Official leaderboard
21 MiMo-V2.6 Pro 27% none, mini-swe-agent Sep 2026 Official leaderboard
22 Claude Opus 4.8 23.6% max, Claude Code May 2026 Official leaderboard
23 Kimi K3 22.4% max, Claude Code Jul 2026 Official leaderboard
24 GPT-5.6 Terra 21.5% max, Codex Jul 2026 Official leaderboard
25 Grok 4.6 20.3% high, Grok Build Aug 2026 Official leaderboard
26 Gemini 3.8 Flash 19.1% high, mini-SWE-agent Sep 2026 Official leaderboard
27 GPT-5.6 Luna 17.3% max, Codex Jul 2026 Official leaderboard
28 GPT-6 Luna 16.4% max, Codex Sep 2026 Official leaderboard
29 MiMo-V2.6 Flash 16.1% none, mini-swe-agent Sep 2026 Official leaderboard
30 Muse Spark 1.3 14.6% xhigh, Muse Code Sep 2026 Official leaderboard
31 Claude Sonnet 5 12.4% max, Claude Code Jun 2026 Official leaderboard
32 Grok 4.5 12.4% high, Grok Build Jul 2026 Official leaderboard
33 Gemini 3.7 Flash 11.2% high, mini-SWE-agent Aug 2026 Official leaderboard
34 MiniMax-M3 7.27% none, mini-swe-agent Jun 2026 Official leaderboard
35 DeepSeek V4 Pro 0813 6.67% max, Claude Code Aug 2026 Official leaderboard
36 Beam 1.0 6.36% none, mini-swe-agent Oct 2026 Official leaderboard
37 GPT-5.4 Mini 1.82% xhigh, Codex Mar 2026 Official leaderboard
38 Inkling-Small 0.91% none, mini-swe-agent Jul 2026 Official leaderboard
39 Nemotron 3.5 Super VL 0.30% none, mini-swe-agent Oct 2026 Official leaderboard
40 Inkling 0% none, mini-swe-agent Jul 2026 Official leaderboard
#ModelScoreSettingDateSource
1 Claude Sonnet 5.5 70.6% Sep 2026 Lab model card
2 Claude Fable 5.2 67.6% Nov 2026 Lab model card
3 Claude Opus 5.5 66.4% Sep 2026 Lab model card
4 Kimi K3.1 Pro 65.7% Oct 2026 Lab model card
5 Gemini 4 Carbon 65.2% Oct 2026 Lab model card
6 GPT-6 Astra 58.2% max, Codex Sep 2026 Official leaderboard
7 GPT-6.1 Sol 58.2% max, Codex Sep 2026 Official leaderboard
8 Claude Fable 5.1 57.9% max, Claude Code Sep 2026 Official leaderboard
9 Gemini 4 Argon 57.4% Oct 2026 Lab model card
10 Muse Wonder Full 57.1% Oct 2026 Lab model card
11 Claude Opus 5 53.9% xhigh, Claude Code Jul 2026 Official leaderboard
12 Muse Wonder 52% Oct 2026 Lab model card
13 GPT-6.1 Terra 51.8% max, Codex Oct 2026 Official leaderboard
14 GPT-6 Sol 49.4% max, Codex Sep 2026 Official leaderboard
15 Ling-3.1-flash 46% Oct 2026 Lab model card
16 Claude Fable 5 44.5% max, Claude Code Jun 2026 Official leaderboard
17 Tencent Hy4 44.4% Nov 2026 Lab model card
18 DeepSeek V4.1 Flash 1017 42.3% Oct 2026 Lab model card
19 GLM-5.3 41.8% max, Claude Code Aug 2026 Official leaderboard
20 Mistral Large 4 (Le Chonk, open weights) 41% Oct 2026 Lab model card
21 Muse Ember 40.9% Nov 2026 Lab model card
22 Claude Haiku 5.5 40% max, Claude Code Oct 2026 Official leaderboard
23 LongCat-2.5 38.5% Nov 2026 Lab model card
24 MiniMax-M3.1-Flash 38.1% Oct 2026 Lab model card
25 Grok 4.7 38% Sep 2026 Lab model card
26 GPT-5.6 Sol 37.3% max, Codex Jul 2026 Official leaderboard
27 GLM-5.3-Flash 36.7% max, Claude Code Aug 2026 Official leaderboard
28 MiMo-V2.6 Pro 34.9% Sep 2026 Lab model card
29 Step 5 Preview 33.3% Sep 2026 Lab model card
30 DeepSeek V4.1 Flash 31.2% Sep 2026 Lab model card
31 Mistral Large 4 28.3% Oct 2026 Lab model card
32 SWE-2 27.3% Sep 2026 Lab model card
33 Qwen3.8 Max (0902) 27% max, Claude Code Sep 2026 Official leaderboard
34 Claude Opus 4.8 23.6% max, Claude Code May 2026 Official leaderboard
35 Kimi K3 22.4% max, Claude Code Jul 2026 Official leaderboard
36 GPT-5.6 Terra 21.5% max, Codex Jul 2026 Official leaderboard
37 Grok 4.6 20.3% high, Grok Build Aug 2026 Official leaderboard
38 SWE-2 Flash 20% Nov 2026 Lab model card
39 Gemini 3.8 Flash 19.1% Sep 2026 Lab model card
40 GPT-5.6 Luna 17.3% max, Codex Jul 2026 Official leaderboard
41 GPT-6 Luna 16.4% max, Codex Sep 2026 Official leaderboard
42 MiMo-V2.6 Flash 16.1% none, mini-swe-agent Sep 2026 Official leaderboard
43 Muse Spark 1.3 14.6% xhigh, Muse Code Sep 2026 Official leaderboard
44 Claude Sonnet 5 12.4% max, Claude Code Jun 2026 Official leaderboard
45 Grok 4.5 12.4% high, Grok Build Jul 2026 Official leaderboard
46 Gemini 3.7 Flash 11.2% high, mini-SWE-agent Aug 2026 Official leaderboard
47 MiniMax-M3 7.27% none, mini-swe-agent Jun 2026 Official leaderboard
48 DeepSeek V4 Pro 0813 6.67% max, Claude Code Aug 2026 Official leaderboard
49 Beam 1.0 6.36% none, mini-swe-agent Oct 2026 Official leaderboard
50 GPT-5.4 Mini 1.82% xhigh, Codex Mar 2026 Official leaderboard
51 Nemotron 3.5 Super VL 1.00% Oct 2026 Lab model card
52 Inkling-Small 0.91% none, mini-swe-agent Jul 2026 Official leaderboard
53 Inkling 0% none, mini-swe-agent Jul 2026 Official leaderboard

Labs report scores for 28 models in their cards. Switch the scores above to rank them together.

Pareto: accuracy against time

Each point is one agent, model and effort. The board plots total run time; this plots the mean time per trial, the same frontier at a fixed trial count. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0% 20% 40% 60% 80% 1001,00010K Mean time per trial (s), log scale, lower is better Accuracy GPT-5.4 Mini (xhigh): Accuracy 1.82%, mean time per trial 1,849 s Claude Opus 4.8 (max): Accuracy 23.6%, mean time per trial 5,163 s MiniMax-M3 (none): Accuracy 7.27%, mean time per trial 3,507 s Claude Fable 5 (max): Accuracy 44.5%, mean time per trial 4,203 s Claude Sonnet 5 (max): Accuracy 12.4%, mean time per trial 6,510 s Grok 4.5 (high): Accuracy 12.4%, mean time per trial 3,162 s GPT-5.6 Luna (max): Accuracy 17.3%, mean time per trial 4,088 s GPT-5.6 Sol (max): Accuracy 37.3%, mean time per trial 2,388 s GPT-5.6 Terra (max): Accuracy 21.5%, mean time per trial 2,515 s Inkling (none): Accuracy 0%, mean time per trial 1,403 s Kimi K3 (max): Accuracy 22.4%, mean time per trial 3,351 s Claude Opus 5 (xhigh): Accuracy 53.9%, mean time per trial 4,515 s Claude Opus 5 (max): Accuracy 51.8%, mean time per trial 4,793 s Claude Opus 5 (high): Accuracy 50.3%, mean time per trial 3,822 s Claude Opus 5 (medium): Accuracy 44.9%, mean time per trial 3,293 s Claude Opus 5 (low): Accuracy 34.9%, mean time per trial 2,947 s Grok 4.6 (high): Accuracy 20.3%, mean time per trial 2,352 s DeepSeek V4 Pro 0813 (max): Accuracy 6.67%, mean time per trial 3,140 s Gemini 3.7 Flash (high): Accuracy 11.2%, mean time per trial 1,689 s GLM-5.3 (max): Accuracy 41.8%, mean time per trial 5,831 s GLM-5.3-Flash (max): Accuracy 36.7%, mean time per trial 6,558 s GLM-5.3-Flash (none): Accuracy 35.8%, mean time per trial 6,210 s Claude Fable 5.1 (max): Accuracy 57.9%, mean time per trial 3,894 s Claude Fable 5.1 (xhigh): Accuracy 57.9%, mean time per trial 3,202 s Claude Fable 5.1 (high): Accuracy 54.5%, mean time per trial 3,018 s Claude Fable 5.1 (medium): Accuracy 53.9%, mean time per trial 2,528 s Claude Fable 5.1 (low): Accuracy 43.3%, mean time per trial 2,291 s Qwen3.8 Max (0902) (max): Accuracy 27%, mean time per trial 9,953 s Gemini 3.8 Flash (high): Accuracy 19.1%, mean time per trial 1,995 s Muse Spark 1.3 (xhigh): Accuracy 14.6%, mean time per trial 4,342 s GPT-6 Astra (xhigh): Accuracy 57.9%, mean time per trial 2,209 s Grok 4.7 (xhigh): Accuracy 37.6%, mean time per trial 5,701 s MiMo-V2.6 Flash (none): Accuracy 16.1%, mean time per trial 5,472 s MiMo-V2.6 Pro (none): Accuracy 27%, mean time per trial 5,265 s GPT-6 Luna (max): Accuracy 16.4%, mean time per trial 2,284 s GPT-6 Sol (max): Accuracy 49.4%, mean time per trial 2,662 s Claude Sonnet 5.5 (max): Accuracy 61.8%, mean time per trial 7,432 s GPT-6.1 Sol (max): Accuracy 58.2%, mean time per trial 3,177 s Mistral Large 4 (max): Accuracy 27.9%, mean time per trial 5,229 s Claude Haiku 5.5 (max): Accuracy 40%, mean time per trial 4,059 s Muse Wonder (xhigh): Accuracy 47%, mean time per trial 2,828 s Nemotron 3.5 Super VL (none): Accuracy 0.3%, mean time per trial 1,781 s DeepSeek V4.1 Flash 1017 (max): Accuracy 40%, mean time per trial 5,030 s GPT-6.1 Terra (max): Accuracy 51.8%, mean time per trial 2,291 s Tencent Hy4 (max): Accuracy 39.1%, mean time per trial 4,882 s Inkling-Small (none): Accuracy 0.91%, mean time per trial 475 s GPT-6 Astra (max): Accuracy 58.2%, mean time per trial 2,796 s GPT-6 Astra (high): Accuracy 57.9%, mean time per trial 2,114 s GPT-6 Astra (medium): Accuracy 54.2%, mean time per trial 1,886 s GPT-6 Astra (low): Accuracy 50.6%, mean time per trial 1,684 s Claude Opus 5.5 (max): Accuracy 64.8%, mean time per trial 4,790 s Beam 1.0 (none): Accuracy 6.36%, mean time per trial 1,440 s Kimi K3.1 Pro (max): Accuracy 60.9%, mean time per trial 4,047 s Claude Opus 5.5 (max)Kimi K3.1 Pro (max)GPT-6 Astra (max)GPT-6 Astra (high)GPT-6 Astra (medium)GPT-6 Astra (low)Beam 1.0 (none)Inkling-Small (none) 0% 20% 40% 60% 80% 1001,00010K Mean time per trial (s), log scale, lower is better Accuracy GPT-5.4 Mini (xhigh): Accuracy 1.82%, mean time per trial 1,849 s Claude Opus 4.8 (max): Accuracy 23.6%, mean time per trial 5,163 s MiniMax-M3 (none): Accuracy 7.27%, mean time per trial 3,507 s Claude Fable 5 (max): Accuracy 44.5%, mean time per trial 4,203 s Claude Sonnet 5 (max): Accuracy 12.4%, mean time per trial 6,510 s Grok 4.5 (high): Accuracy 12.4%, mean time per trial 3,162 s GPT-5.6 Luna (max): Accuracy 17.3%, mean time per trial 4,088 s GPT-5.6 Sol (max): Accuracy 37.3%, mean time per trial 2,388 s GPT-5.6 Terra (max): Accuracy 21.5%, mean time per trial 2,515 s Inkling (none): Accuracy 0%, mean time per trial 1,403 s Kimi K3 (max): Accuracy 22.4%, mean time per trial 3,351 s Claude Opus 5 (xhigh): Accuracy 53.9%, mean time per trial 4,515 s Claude Opus 5 (max): Accuracy 51.8%, mean time per trial 4,793 s Claude Opus 5 (high): Accuracy 50.3%, mean time per trial 3,822 s Claude Opus 5 (medium): Accuracy 44.9%, mean time per trial 3,293 s Claude Opus 5 (low): Accuracy 34.9%, mean time per trial 2,947 s Grok 4.6 (high): Accuracy 20.3%, mean time per trial 2,352 s DeepSeek V4 Pro 0813 (max): Accuracy 6.67%, mean time per trial 3,140 s Gemini 3.7 Flash (high): Accuracy 11.2%, mean time per trial 1,689 s GLM-5.3 (max): Accuracy 41.8%, mean time per trial 5,831 s GLM-5.3-Flash (max): Accuracy 36.7%, mean time per trial 6,558 s GLM-5.3-Flash (none): Accuracy 35.8%, mean time per trial 6,210 s Claude Fable 5.1 (max): Accuracy 57.9%, mean time per trial 3,894 s Claude Fable 5.1 (xhigh): Accuracy 57.9%, mean time per trial 3,202 s Claude Fable 5.1 (high): Accuracy 54.5%, mean time per trial 3,018 s Claude Fable 5.1 (medium): Accuracy 53.9%, mean time per trial 2,528 s Claude Fable 5.1 (low): Accuracy 43.3%, mean time per trial 2,291 s Qwen3.8 Max (0902) (max): Accuracy 27%, mean time per trial 9,953 s Gemini 3.8 Flash (high): Accuracy 19.1%, mean time per trial 1,995 s Muse Spark 1.3 (xhigh): Accuracy 14.6%, mean time per trial 4,342 s GPT-6 Astra (xhigh): Accuracy 57.9%, mean time per trial 2,209 s Grok 4.7 (xhigh): Accuracy 37.6%, mean time per trial 5,701 s MiMo-V2.6 Flash (none): Accuracy 16.1%, mean time per trial 5,472 s MiMo-V2.6 Pro (none): Accuracy 27%, mean time per trial 5,265 s GPT-6 Luna (max): Accuracy 16.4%, mean time per trial 2,284 s GPT-6 Sol (max): Accuracy 49.4%, mean time per trial 2,662 s Claude Sonnet 5.5 (max): Accuracy 61.8%, mean time per trial 7,432 s GPT-6.1 Sol (max): Accuracy 58.2%, mean time per trial 3,177 s Mistral Large 4 (max): Accuracy 27.9%, mean time per trial 5,229 s Claude Haiku 5.5 (max): Accuracy 40%, mean time per trial 4,059 s Muse Wonder (xhigh): Accuracy 47%, mean time per trial 2,828 s Nemotron 3.5 Super VL (none): Accuracy 0.3%, mean time per trial 1,781 s DeepSeek V4.1 Flash 1017 (max): Accuracy 40%, mean time per trial 5,030 s GPT-6.1 Terra (max): Accuracy 51.8%, mean time per trial 2,291 s Tencent Hy4 (max): Accuracy 39.1%, mean time per trial 4,882 s Inkling-Small (none): Accuracy 0.91%, mean time per trial 475 s GPT-6 Astra (max): Accuracy 58.2%, mean time per trial 2,796 s GPT-6 Astra (high): Accuracy 57.9%, mean time per trial 2,114 s GPT-6 Astra (medium): Accuracy 54.2%, mean time per trial 1,886 s GPT-6 Astra (low): Accuracy 50.6%, mean time per trial 1,684 s Claude Opus 5.5 (max): Accuracy 64.8%, mean time per trial 4,790 s Beam 1.0 (none): Accuracy 6.36%, mean time per trial 1,440 s Kimi K3.1 Pro (max): Accuracy 60.9%, mean time per trial 4,047 s Claude Opus 5.5 (max)Kimi K3.1 Pro (max)GPT-6 Astra (max)GPT-6 Astra (high)GPT-6 Astra (medium)GPT-6 Astra (low)Beam 1.0 (none)Inkling-Small (none)
ModelSettingAccuracyMean time per trial
Claude Opus 5.5 max, Claude Code 64.9% 4,790 s
Claude Sonnet 5.5 max, Claude Code 61.8% 7,432 s
Kimi K3.1 Pro max, Claude Code 60.9% 4,047 s
GPT-6 Astra max, Codex 58.2% 2,796 s
GPT-6.1 Sol max, Codex 58.2% 3,177 s
GPT-6 Astra high, Codex 57.9% 2,114 s
GPT-6 Astra xhigh, Codex 57.9% 2,209 s
Claude Fable 5.1 xhigh, Claude Code 57.9% 3,202 s
Claude Fable 5.1 max, Claude Code 57.9% 3,894 s
Claude Fable 5.1 high, Claude Code 54.6% 3,018 s
GPT-6 Astra medium, Codex 54.2% 1,886 s
Claude Fable 5.1 medium, Claude Code 53.9% 2,528 s
Claude Opus 5 xhigh, Claude Code 53.9% 4,515 s
GPT-6.1 Terra max, Codex 51.8% 2,291 s
Claude Opus 5 max, Claude Code 51.8% 4,793 s
GPT-6 Astra low, Codex 50.6% 1,684 s
Claude Opus 5 high, Claude Code 50.3% 3,822 s
GPT-6 Sol max, Codex 49.4% 2,662 s
Muse Wonder xhigh, Muse Code 47% 2,828 s
Claude Opus 5 medium, Claude Code 44.9% 3,293 s
Claude Fable 5 max, Claude Code 44.6% 4,203 s
Claude Fable 5.1 low, Claude Code 43.3% 2,291 s
GLM-5.3 max, Claude Code 41.8% 5,831 s
Claude Haiku 5.5 max, Claude Code 40% 4,059 s
DeepSeek V4.1 Flash 1017 max, Claude Code 40% 5,030 s
Tencent Hy4 max, Claude Code 39.1% 4,882 s
Grok 4.7 xhigh, Grok Build 37.6% 5,701 s
GPT-5.6 Sol max, Codex 37.3% 2,388 s
GLM-5.3-Flash max, Claude Code 36.7% 6,558 s
GLM-5.3-Flash none, Claude Code 35.8% 6,210 s
Claude Opus 5 low, Claude Code 34.9% 2,947 s
Mistral Large 4 max, Claude Code 27.9% 5,229 s
MiMo-V2.6 Pro none, mini-swe-agent 27% 5,265 s
Qwen3.8 Max (0902) max, Claude Code 27% 9,953 s
Claude Opus 4.8 max, Claude Code 23.6% 5,163 s
Kimi K3 max, Claude Code 22.4% 3,351 s
GPT-5.6 Terra max, Codex 21.5% 2,515 s
Grok 4.6 high, Grok Build 20.3% 2,352 s
Gemini 3.8 Flash high, mini-SWE-agent 19.1% 1,995 s
GPT-5.6 Luna max, Codex 17.3% 4,088 s
GPT-6 Luna max, Codex 16.4% 2,284 s
MiMo-V2.6 Flash none, mini-swe-agent 16.1% 5,472 s
Muse Spark 1.3 xhigh, Muse Code 14.6% 4,342 s
Grok 4.5 high, Grok Build 12.4% 3,162 s
Claude Sonnet 5 max, Claude Code 12.4% 6,510 s
Gemini 3.7 Flash high, mini-SWE-agent 11.2% 1,689 s
MiniMax-M3 none, mini-swe-agent 7.3% 3,507 s
DeepSeek V4 Pro 0813 max, Claude Code 6.7% 3,140 s
Beam 1.0 none, mini-swe-agent 6.4% 1,440 s
GPT-5.4 Mini xhigh, Codex 1.8% 1,849 s
Inkling-Small none, mini-swe-agent 0.9% 475.1 s
Nemotron 3.5 Super VL none, mini-swe-agent 0.3% 1,781 s
Inkling none, mini-swe-agent 0% 1,403 s

The best score, 64.8%, costs $14 a task

Runs on the cost frontier score higher than any run that's cheaper. Kimi K3.1 Pro at max comes within 3.9 points of the best for $3.29 a task, 4 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Thinking Machines Z.ai Anthropic OpenAI DeepSeek Moonshot AI Cost frontier One model at each effort level
0% 20% 40% 60% 80% $0.10 $1.00 $10 $100 Cost per task, log scale Claude Opus 5.5, max Kimi K3.1 Pro, max DeepSeek V4.1 Flash 1017, max GLM-5.3-Flash, none GPT-6 Luna, max Inkling-Small, none 0% 20% 40% 60% 80% $0.10 $1.00 $10 $100 Cost per task, log scale Claude Opus 5.5, max GPT-6.1 Sol, max

Source: Official Terminal-Bench 4.0 leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 max Claude Code 64.8% $14 269,135
Claude Sonnet 5.5 max Claude Code 61.8% $22 565,388
Kimi K3.1 Pro max Claude Code 60.9% $3.29 80,956
GPT-6 Astra max Codex 58.2% $9.90 72,694
GPT-6.1 Sol max Codex 58.2% $1.92 94,892
Claude Fable 5.1 max Claude Code 57.9% $19 191,277
Claude Fable 5.1 xhigh Claude Code 57.9% $15 152,809
GPT-6 Astra xhigh Codex 57.9% $7.12 45,304
GPT-6 Astra high Codex 57.9% $6.88 41,527
Claude Fable 5.1 high Claude Code 54.5% $12 113,421
GPT-6 Astra medium Codex 54.2% $5.80 32,638
Claude Opus 5 xhigh Claude Code 53.9% $18 179,281
Claude Fable 5.1 medium Claude Code 53.9% $8.59 78,763
Claude Opus 5 max Claude Code 51.8% $18 200,037
GPT-6.1 Terra max Codex 51.8% $2.65 100,138
GPT-6 Astra low Codex 50.6% $4.72 24,207
Claude Opus 5 high Claude Code 50.3% $14 144,242
GPT-6 Sol max Codex 49.4% $4.40 92,642
Muse Wonder xhigh Muse Code 47% $7.43 129,324
Claude Opus 5 medium Claude Code 44.9% $9.67 101,450
Claude Fable 5 max Claude Code 44.5% $22 177,636
Claude Fable 5.1 low Claude Code 43.3% $7.15 60,409
GLM-5.3 max Claude Code 41.8% $8.27 208,061
Claude Haiku 5.5 max Claude Code 40% $2.89 386,483
DeepSeek V4.1 Flash 1017 max Claude Code 40% $0.56 497,265
Tencent Hy4 max Claude Code 39.1% $1.05 368,811
Grok 4.7 xhigh Grok Build 37.6% $11 69,627
GPT-5.6 Sol max Codex 37.3% $7.70 71,065
GLM-5.3-Flash max Claude Code 36.7% $1.14 459,702
GLM-5.3-Flash none Claude Code 35.8% $0.36 135,014
Claude Opus 5 low Claude Code 34.9% $7.25 72,229
Mistral Large 4 max Claude Code 27.9% $2.18 253,650
Qwen3.8 Max (0902) max Claude Code 27% $14 252,708
MiMo-V2.6 Pro none mini-swe-agent 27% $0.86 245,849
Claude Opus 4.8 max Claude Code 23.6% $20 269,131
Kimi K3 max Claude Code 22.4% $5.82 195,427
GPT-5.6 Terra max Codex 21.5% $5.25 100,681
Grok 4.6 high Grok Build 20.3% $11 98,843
Gemini 3.8 Flash high mini-SWE-agent 19.1% $5.54 206,232
GPT-5.6 Luna max Codex 17.3% $1.05 185,705
GPT-6 Luna max Codex 16.4% $0.22 137,851
MiMo-V2.6 Flash none mini-swe-agent 16.1% $0.59 200,090
Muse Spark 1.3 xhigh Muse Code 14.6% 265,112
Claude Sonnet 5 max Claude Code 12.4% $29 353,330
Grok 4.5 high Grok Build 12.4% $6.35 98,149
Gemini 3.7 Flash high mini-SWE-agent 11.2% $3.82 157,982
MiniMax-M3 none mini-swe-agent 7.27% $3.48 234,421
DeepSeek V4 Pro 0813 max Claude Code 6.67% $2.20 245,205
Beam 1.0 none mini-swe-agent 6.36% $0.71 64,523
GPT-5.4 Mini xhigh Codex 1.82% $2.04 169,597
Inkling-Small none mini-swe-agent 0.91% $0.15 20,104
Nemotron 3.5 Super VL none mini-swe-agent 0.30% $0.51 159,709
Inkling none mini-swe-agent 0% $3.06 39,308

4 models have held number one on the official leaderboard since its release in August 2026 5 models have held number one with claims included since its release in August 2026

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic OpenAI
Own leaderboard 4 leaders
Claude Opus 5.5 64.8%
Lab claims 4 leaders
Claude Sonnet 5.5 70.6%

Sources: Official Terminal-Bench 4.0 leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Sonnet 5.5 Claude Opus 5.5 GPT-6 Astra Claude Opus 5 Claude Fable 5 Claude Opus 4.8 GPT-5.4 Mini 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Sonnet 5.5 GPT-6 Astra

Sources: Official Terminal-Bench 4.0 leaderboard and labs’ model cards

16 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
OpenAI
DeepSeek
Cognition
StepFun
xAI
Anthropic
Xiaomi Corp
Mistral AI
Meta
Ant Group
NVIDIA
MiniMax
Moonshot AI
Meituan
Tencent
DateLaunch postScoreAs the post writes it
Nov 9, 2026 Muse Ember 40.9 Terminal-Bench 4.0, best published harness, Muse Ember max reasoning
Nov 4, 2026 Tencent Hy4 44.4 Terminal Bench 4.0
Nov 4, 2026 SWE-2 Flash 20 Terminal-Bench 4.0
Nov 2, 2026 LongCat-2.5 38.5 Terminal Bench v4.0
Oct 29, 2026 Mistral Large 4 (Le Chonk, open weights) 41 Terminal-Bench 4.0 (Artificial Analysis Coding Agent Index run)
Oct 28, 2026 Gemini 4 Carbon 65.2 Terminal-Bench 4.0 (AA)
Oct 26, 2026 GPT-6.1 Terra 50.3 Terminal-Bench 4.0, maximum reasoning effort
Oct 24, 2026 Kimi K3.1 Pro 65.7 Terminal-Bench 4.0 (Claude Code)
Oct 19, 2026 MiniMax-M3.1-Flash 38.1 Terminal Bench 4.0
Oct 17, 2026 DeepSeek V4.1 Flash 1017 42.3 Terminal-Bench 4.0
Oct 15, 2026 Nemotron 3.5 Super VL 1 Terminal Bench 4 [offline]
Oct 13, 2026 Ling-3.1-flash 46 Terminal-Bench-4.0 (LingCode)
Oct 13, 2026 Gemini 4 Argon 57.4 Terminal-bench 4.0
Oct 12, 2026 Muse Wonder 52 Terminal-Bench 4.0-AA
Oct 12, 2026 Muse Wonder Full 57.1 Terminal-Bench 4.0-AA
Oct 7, 2026 Claude Haiku 5.5 39.2 Agentic coding Terminal-Bench 4.0
Oct 6, 2026 Mistral Large 4 28.3 Coding - Terminal Bench 4.0: AA - Terminal-Bench 4 (chart 28; text 28.3%)
Sep 28, 2026 Claude Sonnet 5.5 70.6 Agentic coding Terminal-Bench 4.0
Sep 22, 2026 MiMo-V2.6 Pro 34.9 Terminal Bench 4.0
Sep 22, 2026 Claude Opus 5.5 66.4 Agentic coding Terminal-Bench 4.0 (xhigh effort)
Sep 21, 2026 Grok 4.7 37.6 Terminal-Bench 4.0 (Multi-hour terminal work)
Sep 20, 2026 Step 5 Preview 33.3 Terminal-Bench v4
Sep 10, 2026 SWE-2 27.3 Terminal-Bench 4.0
Sep 10, 2026 DeepSeek V4.1 Flash 31.2 Terminal-Bench 4.0 (Pass@1)
Sep 3, 2026 GPT-6 Astra 57.9 Terminal-Bench 4.0
Sep 2, 2026 Gemini 3.8 Flash 19.1 Terminal-bench 4.0

13 system cards report it, from 6 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Nov 9, 2026 Muse Ember technical report 40.9 Terminal-Bench 4.0, best published harness, Muse Ember max reasoning Capabilities
Nov 9, 2026 Claude Fable 5.2 system card 67.6 Terminal-Bench 4.0, safeguards enabled, Claude Code --bare, max effort, no internet access, 10 trials per task. Mythos-only score 70.5% excluded (Section 8.5) Capabilities
Oct 13, 2026 Gemini 4 Argon model card 57.4 Terminal-bench 4.0 (results table, p.5) Capabilities
Oct 12, 2026 Muse Wonder model card 52 Terminal-Bench 4.0-AA accuracy from Artificial Analysis, Muse Wonder (max) (capabilities table image, page 3) Capabilities
Oct 12, 2026 Muse Wonder Full model card 57.1 Terminal-Bench 4.0-AA accuracy from Artificial Analysis, Muse Wonder Full (max) (capabilities table image, page 3) Capabilities
Oct 7, 2026 Claude Haiku 5.5 system card 39.2 Terminal-Bench 4.0, max effort, safeguards on with no fallback, no internet egress (8.4) Capabilities
Sep 28, 2026 Claude Sonnet 5.5 system card 70.6 Terminal-Bench 4.0, max effort with safeguards and fallbacks, five trials per task (8.5) Capabilities
Sep 22, 2026 MiMo-V2.6 Pro technical report 34.9 Terminal Bench 4.0 (Table 3) Capabilities
Sep 22, 2026 Claude Opus 5.5 system card 66.4 Terminal-Bench 4.0 at xhigh effort, the setting the summary table marks (64.8% at max), Claude Code --bare mode, safeguards with fallback, five trials per task (Table 8.1.A, 8.5) Capabilities
Sep 21, 2026 Grok 4.7 model card 38 Terminal-Bench 4.0, task success rate, xhigh, Grok Build harness, run by Harbor (2.3) Capabilities
Sep 10, 2026 DeepSeek V4.1 Flash technical report 31.2 Terminal-Bench 4.0 (Pass@1) (Table 3) Capabilities
Sep 2, 2026 Gemini 3.8 Flash model card 19.1 Terminal-bench 4.0 (results table, p.5) Capabilities
Sep 1, 2026 Claude Fable 5.1 system card 55.8 Terminal-Bench 4.0, Claude Code --bare mode, max effort, 15 trials per task; Table 8.1.A rounds to 56% (Mythos 5.1 61%) (8.6) Capabilities

The lab's number comes out ahead in 10 of 14 pairs that differ

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Claude Sonnet 5.5 70.6% Agentic coding Terminal-Bench 4.0 61.8% Leaderboard, max, Claude Code +8.8 pts
MiMo-V2.6 Pro 34.9% Terminal Bench 4.0 27% Leaderboard, none, mini-swe-agent +7.9 pts
Tencent Hy4 44.4% Terminal Bench 4.0 39.1% Leaderboard, max, Claude Code +5.3 pts
Muse Wonder 52% Terminal-Bench 4.0-AA 47% Leaderboard, xhigh, Muse Code +5.0 pts
Kimi K3.1 Pro 65.7% Terminal-Bench 4.0 (Claude Code) 60.9% Leaderboard, max, Claude Code +4.8 pts
DeepSeek V4.1 Flash 1017 42.3% Terminal-Bench 4.0 40% Leaderboard, max, Claude Code +2.3 pts
Claude Fable 5.1 55.8% Terminal-Bench 4.0, Claude Code --bare mode, max effort, 15 trials per task; Table 8.1.A rounds to 56% (Mythos 5.1 61%) (8.6) 57.9% Leaderboard, max, Claude Code −2.1 pts
Claude Opus 5.5 66.4% Agentic coding Terminal-Bench 4.0 (xhigh effort) 64.8% Leaderboard, max, Claude Code +1.6 pts
GPT-6.1 Terra 50.3% Terminal-Bench 4.0, maximum reasoning effort 51.8% Leaderboard, max, Codex −1.5 pts
Claude Haiku 5.5 39.2% Agentic coding Terminal-Bench 4.0 40% Leaderboard, max, Claude Code −0.8 pts
Nemotron 3.5 Super VL 1.00% Terminal Bench 4 [offline] 0.30% Leaderboard, none, mini-swe-agent +0.7 pts
Grok 4.7 38% Terminal-Bench 4.0, task success rate, xhigh, Grok Build harness, run by Harbor (2.3) 37.6% Leaderboard, xhigh, Grok Build +0.4 pts
Mistral Large 4 28.3% Coding - Terminal Bench 4.0: AA - Terminal-Bench 4 (chart 28; text 28.3%) 27.9% Leaderboard, max, Claude Code +0.4 pts
GPT-6 Astra 57.9% Terminal-Bench 4.0 58.2% Leaderboard, max, Codex −0.3 pts
Gemini 3.8 Flash 19.1% Terminal-bench 4.0 19.1% Leaderboard, high, mini-SWE-agent Same

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Google DeepMind Anthropic OpenAI
Terminal-Bench 4.1 20 leaders
Claude Opus 5.5 65.8%
Terminal-Bench Replaced by 2.0, Nov 2025
GPT-5 49.6%
Terminal-Bench 2.0 Replaced by 2.1, May 2026
GPT-5.5 84.7%
Terminal-Bench 2.1 Passed 85% in Sep 2026, by GPT-6 Astra
GPT-6 Astra 87.4%
Terminal-Bench 3.0 Replaced by 4.0, Aug 2026
Claude Opus 5 42.7%
Terminal-Bench 4.0 4 leaders
Claude Opus 5.5 64.8%
Terminal-Bench 4.1 1 leader
Claude Opus 5.5 65.8%

Sources: Official leaderboards and boards copied by Epoch AI (CC BY 4.0)

Replaces Terminal-Bench 3.0 Replaced by Terminal-Bench 4.1

More in Terminal agents

All 19 in Terminal agents
  • SkillsBench In use 86-87 containerized expert tasks in 11 domains, run with and without Agent Skills. Verifier tests give pass rate and skill lift. GPT-5.5 67.3%
  • Terminal-Bench Hard In use Agents complete 44 hard tasks from Terminal-Bench Core in the Terminus 2 harness. Artificial Analysis scores pass@1 averaged over 3 repeats. Gemma 4 31B IT 36%
  • Harbor-Index In use Agents solve about 80 hard tasks drawn from 29 benchmarks run through Harbor adapters, such as HLE, GSO and SWE-bench Pro. Each task keeps its source verifier. GPT-5.5 28.1%
  • Terminal-Bench 2.1 Extended Not in model cards Agents complete long command-line tasks in containers, scored by verification tests. Mercor adds 99 new expert-written tasks to the 89 public ones and reports both scores. Muse Wonder 54%
  • o11y-bench Not in model cards Agents query metrics, logs and traces in a live Grafana stack, investigate incidents and fix dashboards. Grading checks claims against facts read from the same environment. Claude Opus 5.5 93.8%
  • CompileBench Not in model cards Agents build real open-source projects through dependency problems, legacy toolchains and complex build systems, for example compiling 22-year-old code. Claude Opus 4.1 100%