Tool use
We track 157 benchmarks in tool use. They run from everyday level to expert level.
Anthropic held the top tool use score 33% of the contested time
That's about 15 of the 46 months since January 2023, averaged over these benchmarks. We only count time while other labs are still testing, and stop once a model passes 85%. Faded stretches no longer count.
OpenAI Anthropic Z.ai Meta Amazon Qwen ByteDance Seed xAI Xiaomi Corp Arcee AI Kuaishou Moonshot AI MiniMax NVIDIA Inception Labs Google DeepMind DeepSeek
Everyday Most adults, without training
Berkeley Function Calling Leaderboard (BFCL) Passed 85% in Jul 2024, by Llama 3.1-405B
Llama 3.1-405B 88.5% University Undergraduates and entry-level professionals
Expert Specialists with years of training, often over hours of work
Sources: Official leaderboards and labs’ model cards
Share of contested time at number one
Anthropic 33% OpenAI 29% 
Google DeepMind 11% 
Qwen 9% 
ByteDance Seed 4% 
DeepSeek 3% Moonshot AI 3% 
Amazon 2% Xiaomi Corp 2% Meta 2% Z.ai 1% xAI 1%
Google DeepMind 11% Qwen 9% ByteDance Seed 4% DeepSeek 3% Moonshot AI 3% Amazon 2% Xiaomi Corp 2% Meta 2% Z.ai 1% xAI 1% Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Closed out: 8 benchmarks passed 85%
Meta 2 
NVIDIA 1 Xiaomi Corp 1 
Arcee AI 1 Anthropic 1 OpenAI 1 
MiniMax 1 Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
- Jul 2026 MCP Atlas Muse Spark 1.1
- Jun 2026 tau3-bench Nemotron 3 Ultra
- May 2026 PinchBench 2.0 MiMo-V2.5
- Apr 2026 PinchBench Trinity Large Thinking
- Feb 2026 DeepSearchQA Claude Opus 4.6
- Dec 2025 GAIA GPT-5.2
- Oct 2025 tau2-bench MiniMax-M2
- Jul 2024 Berkeley Function Calling Leaderboard (BFCL) Llama 3.1-405B
Each benchmark counts from the one source its own page leads with: its official leaderboard when that is current, else Epoch AI’s own runs, else labs’ own numbers. We date independent runs by the model’s release and lab reports by when they came out.
OpenAI Anthropic Salesforce Google DeepMind Xiaomi Corp Qwen xAI Moonshot AI Z.ai Meta DeepSeek
Everyday Most adults, without training
Berkeley Function Calling Leaderboard (BFCL) Replaced by BFCL V4, Jul 2025
Llama-xLAM-2-70b-fc-r 78.5% University Undergraduates and entry-level professionals
Expert Specialists with years of training, often over hours of work
Source: Official leaderboards
Share of contested time at number one, independent runs only
OpenAI 43% Anthropic 33% Google DeepMind 11% Salesforce 4% DeepSeek 3% Moonshot AI 3% Meta 2% xAI 1%
Google DeepMind 11% Salesforce 4% DeepSeek 3% Moonshot AI 3% Meta 2% xAI 1% Sources: Official leaderboards and Epoch AI (CC BY 4.0)
Closed out: 6 benchmarks passed 85%
Google DeepMind 2 Meta 1 Xiaomi Corp 1 Qwen 1 OpenAI 1 Sources: Official leaderboards and Epoch AI (CC BY 4.0)
- Jul 2026 MCP Atlas Muse Spark 1.1
- May 2026 PinchBench 2.0 MiMo-V2.5
- Mar 2026 tau3-bench Qwen3.5 397B-A17B
- Feb 2026 PinchBench Gemini 3 Flash
- Dec 2025 GAIA GPT-5.2
- Nov 2025 tau2-bench Gemini 3 Pro
Every independent run: each benchmark’s own leaderboard, Epoch AI’s runs and boards Epoch AI copied, dated by the model’s release. We leave out benchmarks that only labs report.
OpenAI Anthropic ByteDance Seed Z.ai Meta Google DeepMind Arcee AI Kuaishou Xiaomi Corp Qwen xAI Moonshot AI MiniMax Inception Labs DeepSeek Mistral AI
Everyday Most adults, without training
Berkeley Function Calling Leaderboard (BFCL) Passed 85% in Jul 2024, by Llama 3.1-405B
Llama 3.1-405B 88.5% University Undergraduates and entry-level professionals
Expert Specialists with years of training, often over hours of work
Sources: Official leaderboards and labs’ model cards
Share of contested time at number one, claimed scores included
Anthropic 52% OpenAI 26% Google DeepMind 6% Qwen 4% DeepSeek 3% Moonshot AI 3% Z.ai 2% Meta 1% xAI 1% ByteDance Seed 1%
Google DeepMind 6% Qwen 4% DeepSeek 3% Moonshot AI 3% Z.ai 2% Meta 1% xAI 1% ByteDance Seed 1% Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Closed out: 8 benchmarks passed 85%
Meta 2 Xiaomi Corp 1 Qwen 1 Anthropic 1 Google DeepMind 1 OpenAI 1 MiniMax 1 Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
- Jul 2026 MCP Atlas Muse Spark 1.1
- May 2026 PinchBench 2.0 MiMo-V2.5
- Mar 2026 tau3-bench Qwen3.5 397B-A17B
- Feb 2026 DeepSearchQA Claude Opus 4.6
- Feb 2026 PinchBench Gemini 3 Flash
- Dec 2025 GAIA GPT-5.2
- Oct 2025 tau2-bench MiniMax-M2
- Jul 2024 Berkeley Function Calling Leaderboard (BFCL) Llama 3.1-405B
Every score, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.
From everyday to expert level
Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.
Everyday
Most adults, without training
- tau2-bench
- Berkeley Function Calling Leaderboard (BFCL)
- BFCL V4
- GAIA
- PinchBench
- τ³-bench Banking
- PinchBench 2.0
- Agent Arena
- Claw-Eval
- VITA-Bench
- tau-bench
- LMArena Search Arena
- tau3-bench
- DeepPlanning
- Galileo Agent Leaderboard
- QwenClawBench
- AceBench
- Galileo Agent Leaderboard v2
- VitaBench 2.0
- Meta Agents Research Environments (ARE) / Gaia2
- T-Eval
- GTA
- tau2-bench Verified
- Cresta-Bench
- AgentBench FC
- AppWorld Test-Challenge
- AppWorld Test-Normal
- CAR-bench
- UniClawBench
- When2Call
- AdaPlanBench
- ToolSandbox
- CRMArena
- WorldTravel
- FocusBench (Meta)
- Gaia2-CLI
- MM-ClawBench
- SuperCLUE-Agent
- Chinese Tool-Use Benchmark
- SeedClawBench
- tau-bench Retail
- ToolcallFormatIFBench
- AgentBench
- WANDR
- WebWalkerQA
- AgentBoard
- VAKRA
- API-Bank
- Customer Support Q&A (ByteDance Seed in-house)
- Gorilla APIBench
- ComplexFuncBench Audio (OpenAI)
- m-tau-bench (Cohere)
- Mobile Actions
- PAgentBench
- RelyToolBench
- tau-bench Airline
- tau2-Noise (LongCat)
- VitaBench-Noise (LongCat)
- Mistral internal function calling benchmark
- Search triggering
- StableToolBench
- Template Following generalization (Qwen)
- ToolCall-Badcase
- ToolUse (Qwen)
- Agent Arena (Gorilla)
- AI Consumer Index
- AppWorld
- CRMArena-Pro
- MyClaw Bench
- ToolBench
- ToolQA
- TravelPlanner
- UserBench
School
Students, from grade school to high school
University
Undergraduates and entry-level professionals
Expert
Specialists with years of training, often over hours of work
- ClawProBench
- DeepSearchQA
- BrowseComp-ZH
- ToolComp
- DeepResearch Bench
- Deep Research Bench (FutureSearch)
- LiveClawBench
- MM-BrowseComp
- EnterpriseOps-Gym
- Hermes Bench
- DRACO
- MCP-Bench
- BrowseComp-Plus
- AutoResearchBench
- ResearchRubrics
- Web Context (Nimble)
- BrowseComp-VL
- RWSearch
- DuetBench
- PlanBench-XL
- TraceTrial-Bench
- DeR2 Bench
- LiveResearchBench
- Research-Eval
- X Browse (xAI)
- xbench-DeepSearch
- Toloka Arena
- Vals Web Search Index
- DeepConsult
- MMSearch-Plus
- STATE-Bench Main 0.7.0
- xbench-DeepSearch-2510
- RISE
- EnterpriseRAG-Bench
- Hy-BrowseComp-Pro2
- Hy-LifeSearch
- Hy-SkillsWorld
- ImageMining
- Mind2Web 2
- MMSearch
- ProdBench (Tencent)
- STATE-Bench
- SWebBench
- Wiki Live Challenge
- ZClawBench
- Agent Behavior Bench
- AgentIF-OneDay
- Deep Research Bench (Moonshot)
- DeepResearch Bench II
- KAET (Kimi Autonomous Execution Tasks)
- Multi-agent search (Anthropic internal benchmark)
- Artificial Analysis Search Index
- ATLAS (Exa)
- DeepResearch Arena
- DeepResearch Bench II (Gemini 2.5 Pro judge)
- DeepResearchEval
- DeepResearchEval v1
- GTA-2
- LHAW
- TRAIL
By kind
Function calling
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) Model emits function calls for single, parallel, multi-turn and agentic scenarios. Scored by AST and execution checks. | Everyday | Checked automatically | 88.5% | In use |
| BFCL V4 Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes. | Everyday | Checked automatically | 75% | In use |
| C3-Bench Agents complete multi-task tool use dialogues built to stress tool dependencies, long context dependencies and frequent switches between policy types. | University | Checked automatically | 12.5% | Dropped |
| ToolComp Models chain several tool calls to answer 485 human-verified prompts. Most final answers are numbers. ToolComp-Enterprise gives 11 tools and ToolComp-Chat targets general chat tool use. Step labels also support process reward model tests. | Expert | Checked automatically | 70.1% | Not in cards |
| Galileo Agent Leaderboard Models pick and call tools on samples drawn from existing tool-use benchmarks. Galileo scores Tool Selection Quality with an LLM judge and averages the datasets equally. | Everyday | Graded by a model | 0.95 | Not in cards |
| AceBench Models call tools from 4,538 Chinese and English APIs in normal, special, and multi-turn agent cases. Answers are checked without LLM judges or live APIs. | Everyday | Checked automatically | 85.4% | Fading |
| ComplexFuncBench Models make function calls for 1,000 travel booking style samples that need several steps, user constraints, implicit parameter reasoning, long parameter values or 128K context. | University | Checked automatically | 61% | Dropped |
| T-Eval Models use tools step by step, and each sub-process is scored separately: instruction following, planning, reasoning, retrieval, understanding, and review. | Everyday | Checked automatically | 86.4% | Dropped |
| GTA Agents answer 229 human-written queries that need 1 to 4 real deployed tools for perception, operation, logic and creativity, often with image inputs. Queries leave the tools and steps implicit. | Everyday | Checked automatically | 46.6% | Not in cards |
| AgentBench FC Agents work five containerized AgentBench environments, ALFWorld, DB, knowledge graph, OS and WebShop, through function-calling prompts instead of the original text format. | Everyday | Checked automatically | 58.9% | Not in cards |
| When2Call Models decide when to call a tool, when to ask a follow-up question and when to say the tools cannot answer. Scored on multiple-choice and LLM-judged test sets. | Everyday | Checked automatically | 64.6% | In use |
| PlanBench-XL Tool-use agents solve 327 retail tasks over 1,665 tools that they must first retrieve, chaining calls to uncover evidence. An optional blocking mode simulates tool failures. | Expert | Checked automatically | 77.1% | Not in cards |
| SuperCLUE-Agent Chinese language models are tested on agent skills in Chinese tasks. These include tool use with several API calls, task planning and memory over long dialogues. | Everyday | Graded by a model | 78.4 | Not in cards |
| Chinese Tool-Use Benchmark Models pick and call plugins in ReAct format for Chinese prompts. Scores are tool selection accuracy, Rouge-L on tool inputs, and false positive rate. | Everyday | Checked automatically | 98% | Dropped |
| ToolcallFormatIFBench Models make tool calls that must follow format rules across many agent scaffolds and tool sets. Qwen uses it to measure real-world tool call following. | Everyday | Checked automatically | 86.1% | Fading |
| Code Interpreter Benchmark Math Models use a Python interpreter to solve math problems. The score is execution accuracy on the Math task set. | School | Checked automatically | 73.1% | Dropped |
| Nexus Models write function calls for queries over real-world APIs, including nested and parallel calls. Scoring is the function calling success rate. | University | Checked automatically | 58.7% | Dropped |
| HuggingFace Agent Benchmark Agents write Python code that calls Hugging Face Transformers tools to solve short tasks. Scores cover tool selection, tools used, and code correctness. | School | Checked automatically | 93.5% | Dropped |
| VAKRA Agents answer multi-step questions by calling more than 8,000 locally hosted APIs over real databases in 62 domains, along with matching document collections. Tasks need chains of 3 to 7 steps across APIs and documents. A deterministic evaluator replays tools and checks the full trajectory. | Everyday | Checked automatically | 38.1% | Not in cards |
| API-Bank Models make API calls in dialogues that need tools such as calendars and search. Scored on call accuracy. | Everyday | Checked automatically | 92.3% | System cards only |
| Code Interpreter Benchmark Models use a Python code interpreter for math, data visualization, and general tasks. Scoring covers result accuracy and the share of code that runs. | School | Checked automatically | 81.7% | System cards only |
| Gorilla APIBench Models write calls to machine learning APIs from Hugging Face, TorchHub and TensorHub. Scored by AST match. | Everyday | Checked automatically | 35.3% | System cards only |
| Code Interpreter Benchmark General Models use a Python interpreter for general tasks. The score is the percentage of generated programs that execute. | School | Checked automatically | 65.5% | Not in cards |
| ComplexFuncBench Audio (OpenAI) Speech-to-speech models solve ComplexFuncBench function-calling tasks whose text prompts OpenAI converted into speech. Covers multi-step calls, constraints, implicit parameters and long inputs. | Everyday | Checked automatically | 66.5% | Dropped |
| Mobile Actions Model converts natural-language phone requests into on-device function calls (calendar, email, maps, flashlight, Wi-Fi). Call accuracy. | Everyday | Checked automatically | 88.4% | Fading |
| RelyToolBench Agents answer queries with simulated APIs built on the StableToolBench server. Scoring targets tool hallucinations, such as calling wrong or missing tools. | Everyday | Graded by a model | 69.3% | In use |
| Mistral internal function calling benchmark Models call functions on Mistral's in-house test set. The Magistral paper does not describe the tasks or the metric. | Everyday | Checked automatically |  87.4% | System cards only |
| Search triggering Models decide whether to call search for queries that need fresh facts or answer directly when search is unneeded. Reports F1. | Everyday | Checked automatically | 88% | System cards only |
| StableToolBench Models solve user requests by calling tools from ToolBench's large set of real-world APIs. A virtual API server with a cache and API simulators stands in for live APIs, so results stay stable over time. GPT-4 judges whether each request was solved and which of two answers is better, giving a solvable pass rate and a win rate. | Everyday | Graded by a model | 26.4% | Not in cards |
| Template Following generalization (Qwen) Models follow system prompts and emit exactly formatted XML or JSON tool calls for five scaffolds derived from popular IDE and CLI coding agents. | Everyday | Checked automatically | 92.7% | System cards only |
| ToolCall-Badcase Models choose a tool and its arguments for 200 cases drawn from real user failure reports. Each case has a ground-truth call. | Everyday | Checked automatically | 95.8% | System cards only |
| ToolUse (Qwen) Models make single-turn, multi-turn and multi-step tool calls. The score combines intent recognition, format and parameter accuracy. | Everyday | Checked automatically | 85.5% | System cards only |
| ToolBench Models plan and call real REST APIs from RapidAPI, over 16,000 of them, to answer instructions that need one or many tools. ToolEval judges pass rate and win rate. | Everyday | Graded by a model | Not in cards | |
| ToolQA Models answer questions that need outside information, using 13 tools such as database queries, text retrieval and a calculator over fixed data sources. The questions are built so that a model's internal knowledge cannot answer them. An easy and a hard set separate single lookups from multi-step tool use, and answers are checked by exact match. | Everyday | Checked automatically | Not in cards | |
| VisualToolBench Vision-language models solve 1,204 image tasks with tools, half single-turn and half multi-turn, graded per turn against weighted rubrics. | University | Graded by a model | Not in cards |
Customer service
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| tau2-bench Dual-control customer-service simulations (retail, airline, telecom) where agent and simulated user both act. Graded on final state. | Everyday | Checked automatically | 98.9% | In use |
| τ³-bench Banking Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. | Everyday | Checked automatically | 55.1% | In use |
| VITA-Bench Agents handle multi-turn requests in simulated food delivery, in-store, and travel services with 66 tools. A rubric-based judge scores task success. | Everyday | Graded by a model | 32.5% | Fading |
| tau-bench Agents chat with a simulated user and call retail or airline APIs under policy rules. A task passes if the final database matches the goal. | Everyday | Checked automatically | 70.5% | Dropped |
| tau3-bench Agents use tools and follow policies while talking with simulated users in airline, retail, telecom, and banking domains. Scoring is task success rate. | Everyday | Checked automatically | 96% | In use |
| Galileo Agent Leaderboard v2 Agents handle simulated multi-turn customer support conversations in banking, healthcare, insurance, investment and telecom, with synthetic personas and domain tools. Scores are Action Completion and Tool Selection Quality. | Everyday | Graded by a model | 62% | Not in cards |
| VitaBench 2.0 Agents serve one simulated user across a sequence of life-service subtasks over days or months. They must infer, use and update the user's preferences, judged by rubrics. | Everyday | Graded by a model | 50.3% | In use |
| tau2-bench Verified Agent serves simulated customers in airline, retail and telecom under policies, with Amazon-fixed faulty tasks from τ²-bench. Pass rate. | Everyday | Checked automatically | 82.7% | Fading |
| Cresta-Bench Contact-center agents take simulated calls for lending, health and airline lines. Tool calls get deterministic checks and LLM judges grade the rest. | Everyday | Graded by a model | 82.6% | Not in cards |
| CRMArena Agents do nine customer-service CRM tasks for service agent, analyst and manager personas inside a real Salesforce org. The org holds 16 linked object types with seeded hidden patterns. | Everyday | Checked automatically | 64.3% | Not in cards |
| DuetBench Agents diagnose and improve customer service agents on simulated deployments with procedures, tools and synthetic traffic. A blinded judge compares responses head to head. | Expert | Graded by a model | 1,035 | Not in cards |
| tau-bench Retail Agents help simulated retail customers through tools while following store policies on the original tasks. The score is pass^1 as a percentage. | Everyday | Checked automatically | 67.8% | Dropped |
| Toloka Arena Agents handle private simulated enterprise workflows with live databases, API tools and strict business rules across nine domains. The main metric pass^5 needs all five trials to succeed. | Expert | Checked automatically | 33% | Not in cards |
| STATE-Bench Main 0.7.0 Agents complete enterprise tasks in travel, customer support, and shopping. The score is overall pass@1 for this track and version. | Expert | Graded by a model | 55.7% | Not in cards |
| Customer Support Q&A (ByteDance Seed in-house) Models answer user questions from noisy enterprise knowledge base results, such as policy questions in e-commerce, government services and telecom. | Everyday | Graded by a model | 72.4% | Fading |
| τ-bench 2.0 Agents serve simulated customers in airline, retail, telecom and banking domains in six languages, by text or voice. Release 2.0 adds frozen task manifests and an intake domain for entity capture. | University | Checked automatically | 79.9% | Not in cards |
| m-tau-bench (Cohere) Agents handle tau-bench retail and airline customer service tasks translated into Japanese, Korean, Arabic, Spanish and French, plus English. Scored by average pass^1 over 10 runs. | Everyday | Checked automatically |  46.5% | Dropped |
| STATE-Bench Agents work multi-step enterprise tasks in travel, customer support and shopping with a sandbox database, domain tools and a simulated user. They must apply the right policy. | Expert | Graded by a model | 58.9% | Not in cards |
| tau-bench Airline Agents help simulated airline customers through tools while following airline policies on the original tasks. The score is pass^1 as a percentage. | Everyday | Checked automatically | 46% | Not in cards |
| tau2-Noise (LongCat) Agents run tau2-bench tasks after a pipeline injects instruction noise and tool failures into the environment. The score is the average over several noisy copies. | Everyday | Checked automatically |  67.1% | Fading |
| CRMArena-Pro Agents do 19 expert-checked sales, service and quoting tasks in B2B and B2C Salesforce orgs, in single-turn and persona-driven multi-turn settings. It also tests whether agents keep confidential data private. | Everyday | Checked automatically | Not in cards |
Apps and MCP servers
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| AutomationBench Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state. | University | Checked automatically | 53.9% | In use |
| MCP Atlas Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. | University | Graded by a model | 90.7% | In use |
| Toolathlon 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. | University | Checked automatically | 56.5% | In use |
| Toolathlon-Verified 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. | University | Checked automatically | 78.4% | In use |
| MCPMark Agent stress-tests realistic MCP tool use (GitHub, Notion, Postgres, browser). Graded by programmatic verification scripts. | University | Checked automatically | 94.5% | In use |
| MCP-Universe Agents complete tasks through 11 real MCP servers across six domains, such as navigation, repositories, finance, and browsing. Scored by execution-based success rate. | University | Checked automatically | 44.6% | Fading |
| EnterpriseOps-Gym Agents do 1,150 expert-written tasks in eight enterprise domains, such as IT, HR and customer service, using 512 tools over live MCP servers. SQL checks on the final database state score each task. | Expert | Checked automatically | 45.9% | In use |
| MCP-Bench Agents solve multi-step tasks using 28 live MCP servers with 250 tools in areas such as finance, travel, scientific computing and academic search. Scoring covers tool use, coordination and planning. | Expert | Graded by a model | 0.75 | Not in cards |
| AppWorld Test-Challenge Agents complete 417 app tasks that require Amazon or Gmail APIs held out from training. The score is the percentage passing every task test. | Everyday | Checked automatically | 92.8% | Not in cards |
| AppWorld Test-Normal Agents complete 168 everyday app tasks from familiar applications by calling APIs through code. The score is the percentage passing every task test. | Everyday | Checked automatically | 94% | Not in cards |
| Era by Eon Generates a simulated enterprise (Salesforce, Slack, Jira, Zendesk, etc.) with exact ground truth. Agents answer questions through MCP/REST. | University | Checked automatically | 76.8% | Not in cards |
| MCPMark Verified Agents complete 127 tasks through version-pinned MCP servers. Stabilized verifiers score single-run task success. | University | Checked automatically | 96.1% | Not in cards |
| EnvSimBench Language models act as simulators of tool environments for agent training. Given a trajectory of agent actions, the model predicts the next environment state and feedback. Labels come from real executors, so the score checks whether simulated states match what the real environment would return. | University | Checked automatically | 42.3% | Not in cards |
| Random Complex Tasks (LongCat) Agents solve 100 tasks sampled from a generator that builds random executable tool environments across more than four domains. Scored by Avg@4, averaged over three runs. | University | Checked automatically | 35.8% | Fading |
| VitaBench-Noise (LongCat) Agents run VitaBench life-service tasks after a pipeline injects instruction noise and tool failures into the environment. The score is the average over several noisy copies. | Everyday | Graded by a model | 20.5% | Fading |
| AppWorld Agents write code against 457 APIs of 9 simulated everyday apps to finish 750 tasks for about 100 fictional users. State-based unit tests check the outcome and collateral damage. | Everyday | Checked automatically | Not in cards |
Deep research
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| DeepSearchQA Questions that need multi-step search and synthesis. Final answers graded against references. | Expert | Checked automatically | 95% | In use |
| BrowseComp-ZH Agents browse the Chinese web to answer 289 multi-hop questions with short, verifiable answers. Scored by accuracy. | Expert | Checked automatically | 82.4% | Fading |
| LMArena Search Arena People compare answers from two anonymous search-augmented models on their own questions and vote for the better one. Votes become Bradley-Terry ratings. | Everyday | Ranked by votes | 1,255 | Fading |
| DeepResearch Bench Agents write cited research reports for 100 PhD-level tasks across 22 fields. Judges score report quality against references and check citation accuracy. | Expert | Graded by a model | 55.3% | Fading |
| Deep Research Bench (FutureSearch) Web research agents work on multi-step research tasks against RetroSearch, a frozen copy of 10,000 to 100,000 web pages per task. Answers are compared with answers worked out by people. | Expert | Checked automatically | 0.55 | Not in cards |
| MM-BrowseComp Browsing agents answer 400 hand-written questions whose evidence sits in web images and videos. Each question has a checklist for analysis of the reasoning path. | Expert | Checked automatically | 24.3% | Fading |
| DRACO 100 deep-research tasks from real Perplexity queries across 10 domains, graded on about 40 expert rubric criteria each. | Expert | Graded by a model | 88.6% | In use |
| BrowseComp-Plus Deep research agents answer 830 BrowseComp questions by searching a fixed corpus of about 100,000 human-verified documents. | Expert | Graded by a model | 70.1% | In use |
| AutoResearchBench Agents search the scientific literature, either tracking down one target paper through multi-step probing or collecting every paper that meets given conditions. | Expert | Checked automatically | 9.39% | Not in cards |
| ResearchRubrics 101 deep-research prompts with 2,593 expert-written weighted rubric criteria (including penalties). Reports graded for grounding, reasoning and clarity. | Expert | Graded by a model | 81% | In use |
| OmniGAIA Agents answer multi-hop questions that need reasoning and multi-turn tool use across video, audio and image inputs. Questions come from an omni-modal event graph built on real data. | University | Checked automatically | 62.5% | Fading |
| Web Context (Nimble) Agents with only web search and page reading enrich company records, find matching entities or write research reports. LLM judges check each fact against source pages. | Expert | Graded by a model | 80% | Not in cards |
| BrowseComp-VL Agents answer BrowseComp-style questions that need web search across both images and text. Answers are short and checked against a key. | Expert | Checked automatically |  59% | Fading |
| RWSearch Agents search the live web to answer 200 expert-written Chinese questions that need many retrieval steps, filtering and arithmetic. Each question has one objective answer. | Expert | Checked automatically | 82% | Fading |
| DeR2 Bench Models reason over supplied scientific documents in a sandbox that separates evidence access from reasoning. Four regimes range from instructions only to relevant documents plus distractors. | Expert | Checked automatically | 83.8% | Fading |
| LiveResearchBench Deep research agents write cited reports for user-centric questions that need current information from many live web sources. Rubric-based judges score the reports. | Expert | Graded by a model | 73.7 | Not in cards |
| Research-Eval Search-augmented models answer 374 grounded multi-hop questions that need sources across many kinds of websites. An LLM judge checks each answer against a checklist of requirements. | Expert | Graded by a model | 66% | Dropped |
| X Browse (xAI) Agents answer questions that need multi-hop search and browsing on the X platform. Scored by answer accuracy. | Expert | Checked automatically | 58% | Dropped |
| xbench-DeepSearch Agents search the web to answer 100 Chinese questions that need multi-step retrieval. Scored by accuracy against reference answers. | Expert | Checked automatically | 83.7% | Fading |
| Vals Web Search Index Agents work Finance Agent v2 and Legal Research Bench tasks with one web search tool as their only outside source. It compares native provider search with Exa and Keenable. | Expert | Graded by a model | 57.5% | Not in cards |
| WANDR 500 wide-search tasks: find tens to hundreds of entities and facts, then return structured records with citations. | Everyday | Graded by a model | 82.1% | In use |
| WebWalkerQA Agents start from a website's home page and traverse its subpages to find the facts that answer a question. Answers are judged against references. | Everyday | Graded by a model | 33.3% | Not in cards |
| DeepConsult Deep research agents answer business and consulting queries. A model judge compares each report pairwise with an OpenAI Deep Research baseline answer. | Expert | Graded by a model | 61.1% | Fading |
| MMSearch-Plus Multimodal browsing agents solve 311 search tasks that need fine-grained visual reasoning, provenance checks and long tool use. Answers are short and checked against a key. | Expert | Checked automatically | 41.4% | Fading |
| xbench-DeepSearch-2510 Agents answer 100 new Chinese deep search questions that need many web searches and cross-checks. Answers are matched to a key. | Expert | Checked automatically | 56.3% | Fading |
| RISE Agents answer real questions from human experts that need multi-step search and interaction with information-dense web pages. MiniMax has not published the scoring method. | Expert | Checked automatically | 64.3% | Fading |
| EnterpriseRAG-Bench Retrieval systems and agents answer 500 questions over a synthetic company's internal documents, slightly over 500,000 of them. | Expert | Graded by a model | 91.5% | Not in cards |
| Hy-BrowseComp-Pro2 Search agents answer hard browsing questions in a Tencent in-house set named after BrowseComp. Task and scoring details are not published. | Expert | Checked automatically |  56.1% | In use |
| Hy-LifeSearch Search agents answer Tencent in-house everyday life questions with web search. Task and scoring details are not published. | Expert | Checked automatically | 49.2% | In use |
| ImageMining Multimodal agents retrieve and reason over image information for image mining tasks. | Expert | Checked automatically | 30.7% | Fading |
| Mind2Web 2 Agents browse the live web for 130 long tasks and return answers with citations. Task-specific judge agents built from rubric trees grade correctness and attribution. | Expert | Graded by a model | 28% | Not in cards |
| MMSearch Multimodal agents search and answer questions that require visual and textual evidence. | Expert | Checked automatically | 72.9% | Fading |
| SWebBench Agents browse the web to retrieve information and do research tasks. iFlytek built it in house. | Expert | Checked automatically |  71.7% | In use |
| Wiki Live Challenge Deep research agents write articles on topics of 100 recent Wikipedia Good Articles, which serve as expert references. Scored on 39 writing criteria and factual verifiability. | Expert | Graded by a model | 53.6 | Not in cards |
| Deep Research Bench (Moonshot) Agents answer deep-research queries curated by domain experts. Expert-aligned rubrics grade the reports. | Expert | Graded by a model | 90% | System cards only |
| DeepResearch Bench II Deep research agents write reports for 132 research briefs in 22 domains, each derived from a review article they may not consult. A judge model checks 9,287 expert criteria. | Expert | Graded by a model | 43.1% | Not in cards |
| Multi-agent search (Anthropic internal benchmark) Agents solve difficult information retrieval problems with web search, alone or as orchestrators that delegate to subagents. | Expert | Graded by a model | 74.8% | System cards only |
| Artificial Analysis Search Index One fixed agent answers DeepSearchQA, BrowseComp and AA-Omniscience questions while the search API provider changes. The index is the equal-weighted mean of the three scores. | Expert | Checked automatically | Not in cards | |
| ATLAS (Exa) Agents search the web to build a full table of every entity that meets a set of conditions, with 2 to 10 attributes each. A row counts only if every cell is correct. | Expert | Graded by a model | Not in cards | |
| DeepResearch Arena Deep research agents get research tasks generated from academic seminar transcripts, which reflect expert discussion and reduce data leakage. | Expert | Graded by a model | Not in cards | |
| DeepResearch Bench II (Gemini 2.5 Pro judge) Systems conduct deep research under the legacy Gemini 2.5 Pro evaluator. Scores remain separate from the current GPT-5.5 evaluator. | Expert | Graded by a model | Not in cards | |
| DeepResearchEval Deep research systems answer complex research tasks that require evidence from many web sources. A persona-driven pipeline writes the tasks and keeps only those that need outside search. A model judge scores each report on general and task-specific dimensions, and an agent fact-checks the report's statements with web search even when citations are missing. | Expert | Graded by a model | Not in cards | |
| DeepResearchEval v1 Research agents prepare reports. The score is average report quality on the original evaluation tasks. | Expert | Graded by a model | Not in cards |
General assistants
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| GAIA Agents answer real-world questions that need web browsing, file handling, multimodal input, and tools, at three difficulty levels. Scored by exact answer match. | Everyday | Checked automatically | 91.7% | Fading |
| PinchBench Real-world tasks (scheduling, email triage, coding, research, files) for an LLM driving an OpenClaw agent. Graded by automated checks and LLM judge. | Everyday | Graded by a model |  94.9% | In use |
| PinchBench 2.0 Models drive an OpenClaw agent through 148 real-world tasks, including data and log analysis, meeting notes, DevOps, PDFs, research and writing. Graded by checks and a Haiku judge. | Everyday | Graded by a model | 93.5% | Not in cards |
| Agent Arena Arena ranks models as the orchestrator in real Agent Mode sessions on arena.ai. Scores use behavioral signals such as file downloads, disapproval events, retries and steerability. | Everyday | Ranked by votes | 30.4% | System cards only |
| ClawProBench Agents on stateful runtimes are scored with their traces, covering evidence gathering, runtime routing, safety boundaries and repeated runs in frozen workplace setups. | Expert | Checked automatically | 80% | Not in cards |
| Claw-Eval 300 human-verified agent tasks (service orchestration, multimodal, multi-turn dialogue) audited via traces and snapshots. Rubric scores with Pass^3 consistency. | Everyday | Graded by a model | 81.4% | In use |
| Claw-Eval Pass^3 Agents complete 300 human-verified tasks in service orchestration, multimodal interaction, and professional dialogue. Pass^3 credits a task only if all three trials pass. | University | Graded by a model | 78.3% | In use |
| WildClawBench 60 original in-the-wild tasks run inside a live OpenClaw assistant environment, chaining 10-60+ tool calls. Scored by task-level graders. | University | Checked automatically | 67.2% | In use |
| DeepPlanning Agents use tools to plan multi-day trips and multi-item shopping orders under time and budget limits. Plans are checked against verifiable constraints. | Everyday | Checked automatically | 58.9% | Fading |
| QwenClawBench Agents complete 100 tasks across 8 domains in isolated OpenClaw workspaces drawn from real user patterns. Automated checks and an LLM judge score each task. | Everyday | Checked automatically | 60.5% | Fading |
| LiveClawBench Agents act as OpenClaw personal assistants on 134 multi-step tasks across 10 app domains backed by stateful mock services. A run passes when its reward exceeds 0.8. | Expert | Checked automatically | 76% | In use |
| Meta Agents Research Environments (ARE) / Gaia2 Simulated app environments for agents with time and events. Hosts the Gaia2 benchmark. | Everyday | Checked automatically | 42.1% | Fading |
| Hermes Bench Agents in Hermes Agent do 150 tasks across 25 categories, such as research, diagrams, memory, tool use and safety. Graders check the files and state left behind. | Expert | Checked automatically | 76.7% | Not in cards |
| CAR-bench Agents act as an in-car voice assistant that talks with an LLM-simulated driver and uses 58 connected tools for navigation, productivity, charging and vehicle control under set policies. Besides ordinary tasks, hallucination tasks remove a needed tool or fact, and disambiguation tasks need a clarifying question or a lookup. Scores reward consistent success across repeated runs. | Everyday | Checked automatically | 70% | Not in cards |
| Claw-Eval-Live Agents complete 105 workflow tasks in 17 families drawn from live ClawHub workflow signals, scored on both the workflow and the final workspace state. | University | Graded by a model | 66.7% | Not in cards |
| UniClawBench Agents act as proactive assistants on real-world tasks with live tools across many turns. Tasks are grouped by the capability they test, so failures can be traced to one skill. | Everyday | Checked automatically | 47.5% | In use |
| AdaPlanBench Agents plan under world and user constraints that are revealed step by step during interaction, and must adapt plans as new constraints appear. | Everyday | Checked automatically | 67.8% | Not in cards |
| ToolSandbox Models use stateful tools in multi-turn conversations with a simulated user. Tools have hidden state dependencies, and runs are scored against milestones. | Everyday | Checked automatically | 80.7% | In use |
| TraceTrial-Bench Agent harnesses, not only models, are tested on reliability across seven components, with traces used to attribute each failure to its cause. | Expert | Checked automatically | 77.6% | Not in cards |
| WorldTravel Agents plan trips for 150 real-world scenarios across 5 cities with more than 15 tightly coupled time and logic constraints each. A web environment supplies multimodal travel data. | Everyday | Checked automatically | 32.7% | Fading |
| FocusBench (Meta) Agents do personal tasks in domains such as housing, travel, IT and finance. The test checks whether they keep working through real-world blockers across several interactions as the environment changes. | Everyday | Checked automatically | 54.2% | In use |
| Gaia2-CLI Agents complete realistic tasks through command-line tools. The benchmark measures task success across search, execution, adaptability, ambiguity, and time constraints. | Everyday | Checked automatically | 57% | Not in cards |
| MM-ClawBench Agents complete common OpenClaw tasks such as study planning, office documents, scheduled research, and code maintenance. Scoring is accuracy. | Everyday | Graded by a model | 75.4% | Fading |
| SeedClawBench Agents complete 100 OpenClaw-style user tasks built from Ark usage logs and crowdsourced tasks, such as data analysis and office work. An agent judge checks rubric items against the trace and deliverables. | Everyday | Graded by a model | 66.6% | In use |
| AgentBench 8 environments (OS, DB, KG, games, web) testing LLMs as agents. | Everyday | Checked automatically | 4.01 | Not in cards |
| AgentBoard LLM agents act in 9 partially observable, multi-turn environments that cover embodied tasks, games, web browsing and tool use. Besides final success, AgentBoard reports a progress rate, the share of annotated subgoals the agent reaches. An open toolkit breaks results down by skill. | Everyday | Checked automatically | 48.9% | Not in cards |
| Hy-SkillsWorld Agents use skills to complete Tencent in-house working tasks. Task and scoring details are not published. | Expert | Checked automatically | 45.8% | In use |
| KAT Claw Bench Agents complete business tool-use tasks from Kuaishou internal agent needs in seven categories, such as office work, content operations, data analysis and monitoring. Automatic checks grade objective outputs, and model and human review grade open deliverables. | University | Graded by a model | 85.5% | In use |
| PAgentBench Agents act as personal assistants for task management, file operations and system interactions. iFlytek built it in house. | Everyday | Checked automatically | 74.1% | In use |
| ProdBench (Tencent) Agents complete Tencent in-house productivity tasks in the OpenClaw harness. Scored by pass^3. | Expert | Checked automatically | 23% | In use |
| ZClawBench Agents do end-to-end OpenClaw workflows such as environment setup, development, information retrieval, data analysis and content creation, drawn from real OpenClaw use. | Expert | Checked automatically | 57.6% | Fading |
| Agent Behavior Bench Agents are scored on process quality, including tool-use behavior, efficiency and discipline, together with task completion. | Expert | Graded by a model | 65% | System cards only |
| AgentIF-OneDay Agents complete 104 tasks that stand for a full day of human work, life and study. Tasks include tedious workflows, inferring rules from examples and changing requirements. | Expert | Graded by a model | 62.6% | Not in cards |
| KAET (Kimi Autonomous Execution Tasks) Agents execute long-horizon tasks that simulate real user requests and enterprise system operations. | Expert | Checked automatically | 83.5% | System cards only |
| Agent Arena (Gorilla) Users give a task, two agents built from different models, tools and frameworks run it, and users vote. A Bradley-Terry extension rates models, tools and frameworks. | Everyday | Ranked by votes | Not in cards | |
| AI Consumer Index Models with web search handle everyday consumer requests in shopping, food, gaming and DIY. Prompt-specific rubrics with hurdle and grounding criteria grade each answer. | Everyday | Graded by a model | Not in cards | |
| GTA-2 Agents face two tiers: the original atomic tool-use queries and new long-horizon workflows with open-ended deliverables. Workflows are graded with recursive checkpoints that split each goal into checkable parts. | Expert | Graded by a model | Not in cards | |
| LHAW Agents face 285 underspecified variants of long-horizon tasks with goals, constraints, inputs or context removed. Scored on strategic clarification and task success. | Expert | Checked automatically | Not in cards | |
| MyClaw Bench Models run inside the OpenClaw agent on 45 tasks in four difficulty tiers built from real usage data. Some tasks use an LLM judge. | Everyday | Graded by a model | Not in cards | |
| TRAIL Models read long agent workflow traces and must find and classify the errors in them, using a formal taxonomy of agent error types. The set has 148 large human-annotated traces from established agent benchmarks. | Expert | Checked automatically | Not in cards | |
| TravelPlanner Agents plan multi-day trips from a user request, using tools to search nearly four million records of flights, hotels, restaurants and attractions. Each plan must meet the user's hard constraints, such as budget, and commonsense rules. The headline final pass rate counts a plan only when it meets every constraint. | Everyday | Checked automatically | Not in cards | |
| UserBench Agents help simulated users who start with vague goals and reveal preferences bit by bit. Agents must ask clarifying questions and use tools to reach choices that fit the user. | Everyday | Checked automatically | Not in cards |