Benchmark Almanac

Terminal-Bench Hard

Gemma 4 31B IT 36% leads since July 2026
In use

Agents complete 44 hard tasks from Terminal-Bench Core in the Terminus 2 harness. Artificial Analysis scores pass@1 averaged over 3 repeats.

Artificial Analysis retired it in June 2026 in favor of Terminal-Bench 2.1.

Released
September 2025
Kind
Terminal tasks
Level
Expert
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Strongest model tested
Gemma 4 31B IT #100 on the capability index
Last in a launch post
July 2026
Last model added
July 2026
13
Models scored in lab cards
9
Launch posts list it, from 7 labs
9
System cards report it, from 5 labs

Gemma 4 31B IT leads with 36% among models released by July 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in July 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

13 models. Scroll sideways for the rest.

  1. 36.0 Gemma 4 31B IT
  2. 33.3 GLM-4.7
  3. 31.1 North Mini Code 1.0
  4. 30.5 MiMo-V2-Flash
  5. 28.3 Solar Open 2
  6. 25.8 Nemotron 3 Super
  7. 25.0 Command A+
  8. 24.0 MiniMax-M2
  9. 14.0 Gemma 4 26B A4B
  10. 8.51 Nemotron 3 Nano 30B-A3B
  11. 8.00 Gemma 4 E4B
  12. 4.96 Falcon H1R 7B
  13. 3.00 Gemma 4 E2B
  1. 1 Gemma 4 31B IT 36%
  2. 2 GLM-4.7 33.3%
  3. 3 North Mini Code 1.0 31.1%
  4. 4 MiMo-V2-Flash 30.5%
  5. 5 Solar Open 2 28.3%
  6. 6 Nemotron 3 Super 25.8%
  7. 7 Command A+ 25%
  8. 8 MiniMax-M2 24%
  9. 9 Gemma 4 26B A4B 14%
  10. 10 Nemotron 3 Nano 30B-A3B 8.51%
  11. 11 Gemma 4 E4B 8.00%
  12. 12 Falcon H1R 7B 4.96%
  13. 13 Gemma 4 E2B 3.00%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 Gemma 4 31B IT 36% Jul 2026 Lab model card
2 GLM-4.7 33.3% Dec 2025 Lab model card
3 North Mini Code 1.0 31.1% Jun 2026 Lab model card
4 MiMo-V2-Flash 30.5% Dec 2025 Lab model card
5 Solar Open 2 28.3% Jul 2026 Lab model card
6 Nemotron 3 Super 25.8% Mar 2026 Lab model card
7 Command A+ 25% May 2026 Lab model card
8 MiniMax-M2 24% Oct 2025 Lab model card
9 Gemma 4 26B A4B 14% Jul 2026 Lab model card
10 Nemotron 3 Nano 30B-A3B 8.51% Dec 2025 Lab model card
11 Gemma 4 E4B 8.00% Jul 2026 Lab model card
12 Falcon H1R 7B 4.96% Jan 2026 Lab model card
13 Gemma 4 E2B 3.00% Jul 2026 Lab model card

4 models have held number one in labs’ own numbers since its release in September 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

MiniMax Xiaomi Corp Z.ai Google DeepMind
Lab claims Not tested since Jul 2026
Gemma 4 31B IT 36%

Source: Labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Gemma 4 31B IT GLM-4.7 MiniMax-M2 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Gemma 4 31B IT MiniMax-M2

Source: Labs’ model cards

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

MiniMax
NVIDIA
Xiaomi Corp
Z.ai
Technology Innovation Institute
Cohere
Upstage
DateLaunch postScoreAs the post writes it
Jul 22, 2026 Solar Open 2 28.3 Terminal Bench Hard
Jun 9, 2026 North Mini Code 1.0 31.1 Terminal-Bench Hard (Terminus-2)
May 20, 2026 Command A+ 25 Terminal-Bench Hard (Terminus-2, Artificial Analysis method)
Mar 11, 2026 Nemotron 3 Super 25.8 Terminal Bench (hard subset)
Jan 5, 2026 Falcon H1R 7B 4.96 TB Hard
Dec 22, 2025 GLM-4.7 33.3 Terminal Bench Hard
Dec 16, 2025 MiMo-V2-Flash 30.5 Terminal-Bench Hard
Dec 15, 2025 Nemotron 3 Nano 30B-A3B 8.5 Terminal Bench (hard subset)
Oct 27, 2025 MiniMax-M2 24 Terminal-Bench-Hard (AA)

9 system cards report it, from 5 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Jul 24, 2026 Gemma 4 E4B technical report 8 Terminal Bench Hard, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 E2B technical report 3 Terminal Bench Hard, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 31B IT technical report 36 Terminal Bench Hard, thinking (Table 5) Capabilities
Jul 24, 2026 Gemma 4 26B A4B technical report 14 Terminal Bench Hard, thinking (Table 5) Capabilities
Jul 22, 2026 Solar Open 2 technical report 28.3 Terminal Bench Hard (Table 3) Capabilities
Mar 11, 2026 Nemotron 3 Super technical report 25.8 Terminal Bench (hard subset), 48 tasks (Table 5) Capabilities
Jan 6, 2026 MiMo-V2-Flash technical report 30.5 Terminal-Bench Hard (Table 9) Capabilities
Jan 5, 2026 Falcon H1R 7B technical report 4.9 TB Hard (Table 5, 4.96 on the launch page) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B technical report 8.51 Terminal Bench (hard subset) (Table 3) Capabilities

More in Terminal agents

All 19 in Terminal agents
  • SkillsBench In use 86-87 containerized expert tasks in 11 domains, run with and without Agent Skills. Verifier tests give pass rate and skill lift. GPT-5.5 67.3%
  • Harbor-Index In use Agents solve about 80 hard tasks drawn from 29 benchmarks run through Harbor adapters, such as HLE, GSO and SWE-bench Pro. Each task keeps its source verifier. GPT-5.5 28.1%
  • Terminal-Bench 2.1 Extended Not in model cards Agents complete long command-line tasks in containers, scored by verification tests. Mercor adds 99 new expert-written tasks to the 89 public ones and reports both scores. Muse Wonder 54%
  • Terminal-Bench 4.1 Not in model cards The Terminal-Bench 4.0 tasks with hardened verifiers: agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate. Claude Opus 5.5 65.8%
  • o11y-bench Not in model cards Agents query metrics, logs and traces in a live Grafana stack, investigate incidents and fix dashboards. Grading checks claims against facts read from the same environment. Claude Opus 5.5 93.8%
  • CompileBench Not in model cards Agents build real open-source projects through dependency problems, legacy toolchains and complex build systems, for example compiling 22-year-old code. Claude Opus 4.1 100%