Terminal agents
We track 25 benchmarks in terminal agents. They run from everyday level to research frontier level.
OpenAI held the top terminal agents score 33% of the contested time
That's about 15 of the 46 months since January 2023, averaged over these benchmarks. We only count time while other labs are still testing, and stop once a model passes 85%. Faded stretches no longer count.
Mistral AI OpenAI Google DeepMind Anthropic MiniMax Xiaomi Corp Z.ai Moonshot AI xAI DeepSeek Meta
Sources: Official leaderboards, boards copied by Epoch AI (CC BY 4.0) and labs’ model cards
Share of contested time at number one
OpenAI 33% 
Google DeepMind 23% Anthropic 16% Z.ai 11% Moonshot AI 3% 
MiniMax 3% 
DeepSeek 3% xAI 3% 
Meta 2% 
Mistral AI 2%
Google DeepMind 23% Anthropic 16% Z.ai 11% Moonshot AI 3% MiniMax 3% DeepSeek 3% xAI 3% Meta 2% Mistral AI 2% Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Each benchmark counts from the one source its own page leads with: its official leaderboard when that is current, else Epoch AI’s own runs, else labs’ own numbers. We date independent runs by the model’s release and lab reports by when they came out.
Anthropic Mistral AI OpenAI Google DeepMind Moonshot AI xAI DeepSeek Meta
Everyday Most adults, without training
Expert Specialists with years of training, often over hours of work
Sources: Official leaderboards and boards copied by Epoch AI (CC BY 4.0)
Share of contested time at number one, independent runs only
OpenAI 38% Google DeepMind 27% Anthropic 19% Moonshot AI 4% DeepSeek 4% xAI 3% Meta 3% Mistral AI 2%
Google DeepMind 27% Anthropic 19% Moonshot AI 4% DeepSeek 4% xAI 3% Meta 3% Mistral AI 2% Sources: Official leaderboards and Epoch AI (CC BY 4.0)
Every independent run: each benchmark’s own leaderboard, Epoch AI’s runs and boards Epoch AI copied, dated by the model’s release. We leave out benchmarks that only labs report.
Mistral AI OpenAI Google DeepMind Anthropic Cognition MiniMax Xiaomi Corp Z.ai Qwen Moonshot AI xAI DeepSeek Meta
Sources: Official leaderboards and labs’ model cards
Share of contested time at number one, claimed scores included
OpenAI 29% Anthropic 24% Google DeepMind 17% Z.ai 10% 
Qwen 6% Moonshot AI 3% MiniMax 3% DeepSeek 3% xAI 2% Meta 2% Mistral AI 2%
Google DeepMind 17% Z.ai 10% Qwen 6% Moonshot AI 3% MiniMax 3% DeepSeek 3% xAI 2% Meta 2% Mistral AI 2% Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Every score, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.
From everyday to research frontier level
Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.
Expert
Specialists with years of training, often over hours of work
- Terminal-Bench 2.0
- Terminal-Bench 2.1
- Terminal-Bench 4.0
- Terminal-Bench
- SkillsBench
- Terminal-Bench 2.1 Extended
- Terminal-Bench 4.1
- Long-Horizon-Terminal-Bench (LHTB)
- o11y-bench
- Terminal-Bench Hard
- CompileBench
- Terminal-Bench 3.0
- Long-Horizon-Terminal-Bench 2.0
- ITBench
- AgencyBench
- DeepTerminalBench
- EdgeBench
- OpenThoughts-TBLite
- aws-bench
- ITBench-AA
- Terminal-Bench 2.0 Verified (Z.ai)
- Terminal-X
Research frontier
Researchers, on open problems or tasks that take days
By kind
Terminal tasks
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Terminal-Bench 2.0 89 hard terminal tasks across 16 categories in containers. Hidden tests verify the final state. 5 trials per task. | Expert | Checked automatically | 84.7% | In use |
| Terminal-Bench 2.1 89 hard terminal tasks across 16 categories in containers. Hidden tests verify the final state. 5 trials per task. | Expert | Checked automatically | 87.4% | In use |
| Terminal-Bench 4.0 Agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate. | Expert | Checked automatically | 64.8% | In use |
| Terminal-Bench Agents complete about 80 hard tasks in sandboxed Linux terminals, such as compiling code or configuring servers. Test scripts check the final container state. | Expert | Checked automatically | 49.6% | Fading |
| SkillsBench 86-87 containerized expert tasks in 11 domains, run with and without Agent Skills. Verifier tests give pass rate and skill lift. | Expert | Checked automatically | 67.3% | In use |
| Terminal-Bench 2.1 Extended Agents complete long command-line tasks in containers, scored by verification tests. Mercor adds 99 new expert-written tasks to the 89 public ones and reports both scores. | Expert | Checked automatically | 54% | Not in cards |
| Terminal-Bench 4.1 The Terminal-Bench 4.0 tasks with hardened verifiers: agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate. | Expert | Checked automatically | 65.8% | Not in cards |
| o11y-bench Agents query metrics, logs and traces in a live Grafana stack, investigate incidents and fix dashboards. Grading checks claims against facts read from the same environment. | Expert | Checked automatically | 93.8% | Not in cards |
| Terminal-Bench Hard Agents complete 44 hard tasks from Terminal-Bench Core in the Terminus 2 harness. Artificial Analysis scores pass@1 averaged over 3 repeats. | Expert | Checked automatically | 36% | In use |
| CompileBench Agents build real open-source projects through dependency problems, legacy toolchains and complex build systems, for example compiling 22-year-old code. | Expert | Checked automatically | 100% | Not in cards |
| Terminal-Bench 3.0 Agents complete 74 tasks across 7 domains in sandboxed terminal environments. Verifier scripts grade each trial as pass or fail. | Expert | Checked automatically | 42.7% | In use |
| ITBench Agents handle IT automation tasks in Kubernetes environments: site reliability incidents, compliance and security checks, and cloud cost work. | Expert | Checked automatically | 58.3% | Not in cards |
| TUA-Bench Agents use a terminal for general computer tasks beyond coding, such as file, system and everyday productivity work. | Everyday | Checked automatically | 65.8% | Not in cards |
| Harbor-Index Agents solve about 80 hard tasks drawn from 29 benchmarks run through Harbor adapters, such as HLE, GSO and SWE-bench Pro. Each task keeps its source verifier. | Research frontier | Checked automatically | 28.1% | In use |
| DeepTerminalBench Agents get one attempt at each of 50 deep engineering tasks in Docker, in the Terminal-Bench 2.0 format. A task passes only if every hidden test passes. Tasks were calibrated so Claude Opus 4.6 still fails some at pass@4. | Expert | Checked automatically | 34% | Not in cards |
| OpenThoughts-TBLite Agents solve 100 Terminal-Bench tasks picked to track Terminal-Bench 2 scores while running faster. Tasks are balanced across four difficulty bands using Claude Haiku 4.5 pass rates. | Expert | Checked automatically | 75.1% | Not in cards |
| aws-bench Coding agents and model pairs diagnose misconfigurations, provision infrastructure and operate live cloud setups in disposable real AWS accounts. | Expert | Checked automatically | Not in cards | |
| ITBench-AA Agents read an offline snapshot of a failing Kubernetes cluster and name the entities that caused the incident, in structured JSON. | Expert | Graded by a model | Not in cards | |
| Terminal-Bench 2.0 Verified (Z.ai) Agents do the Terminal-Bench 2.0 tasks after Z.ai fixed environments for 89 tasks and instructions for 11. A further 2026-08-18 round fixed tests and instructions in 6 more tasks. | Expert | Checked automatically | Not in cards | |
| Terminal-X A suite of three terminal datasets for coding agents. DeepTerminalBench tests single-shot depth, EvoCode-Bench tests multi-round iteration and RoadmapBench tests version upgrades on real repositories. | Expert | Checked automatically | Not in cards |
Long terminal sessions
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Long-Horizon-Terminal-Bench (LHTB) Terminal-Bench-style container tasks needing hundreds of steps. Subtasks graded deterministically for dense partial-credit reward. Solved at reward >=0.95. | Expert | Checked automatically | 60.2% | Not in cards |
| Long-Horizon-Terminal-Bench 2.0 Agents work for up to 3 hours per task on 77 container tasks, including 30 tasks that rebuild a scientific tool from its outputs. Hidden verifiers give partial-credit rewards. | Expert | Checked automatically | 69.2% | Not in cards |
| AgencyBench Agents complete 138 long tasks drawn from daily AI use, grouped into 32 scenarios that cover six agentic capabilities such as coding, front-end work, game development and research. A typical task takes about 90 tool calls, 1 million tokens and hours of work. A simulated user gives feedback during the run, and a Docker sandbox checks the deliverables against task rubrics. | Expert | Graded by a model | 56.5% | Not in cards |
| EdgeBench Agents iterate for up to 12 hours on 134 executable real-world tasks with multi-level feedback. Scores are tracked at 2 hour steps to measure learning from the environment. | Expert | Checked automatically | 51.3 | Not in cards |
| Terminal-Bench Challenges Agents build a large project end to end with no time limit, such as a faster Rust compiler or a WebGL renderer in WebAssembly. Each challenge has its own test bed and metric. | Research frontier | Checked automatically | Not in cards |