RoboArena
Double-blind pairwise A/B trials of generalist DROID policies on real Franka arms, free-form tasks by volunteer evaluators. Elo-style rating.
Official leaderboard
Stanford University
We track 75 benchmarks and 1 arena that Stanford University made or helped make. Its models are on its lab page.
9 in use 64 with public code 47 with a leaderboard here
Arena leaderboards
Top five in each of its 1 arena with three or more rated models, the most rated first.
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Terminal-Bench 2.0 89 hard terminal tasks across 16 categories in containers. Hidden tests verify the final state. 5 trials per task. | Terminal tasks | Nov 2025 | 84.7% | In use |
| Terminal-Bench 2.1 89 hard terminal tasks across 16 categories in containers. Hidden tests verify the final state. 5 trials per task. | Terminal tasks | May 2026 | 87.4% | In use |
| Terminal-Bench 4.0 Agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate. | Terminal tasks | Aug 2026 | 64.8% | In use |
| Terminal-Bench Agents complete about 80 hard tasks in sandboxed Linux terminals, such as compiling code or configuring servers. Test scripts check the final container state. | Terminal tasks | May 2025 | 49.6% | Fading |
| AlpacaEval 2.0 Models answer 805 open instructions, and a GPT-4 Turbo judge compares each reply with GPT-4 Turbo's own. The score is a win rate. | Overall chat quality | Jan 2024 |  87.6% | Fading |
| Cybench 40 professional capture-the-flag tasks from four competitions. Agent must recover the flag, with optional subtasks. | Capture-the-flag challenges | Aug 2024 |  65.4% | Fading |
| Terminal-Bench-Science 70 scientific research-workflow tasks across five science areas, run in Harbor containers. Hidden tests give the resolution rate. | Research science | Aug 2026 | 68.1% | In use |
| QuAC Models answer free-form questions in information-seeking dialogs about a hidden Wikipedia section, across 100K questions. Scored by word-level F1. | Reading comprehension | Aug 2018 | 51.1% | Dropped |
| ProgramBench tests passed Agents rebuild 200 programs from only their compiled binaries and docs. The score is the share of hidden behavioral tests the rebuilt program passes, averaged over tasks. | Building whole projects | May 2026 | 88.5% | In use |
| ProgramBench 200 tasks: rebuild a full program from only its compiled binary and docs. Hidden behavioral tests (248k total) must match the original. | Building whole projects | May 2026 | 18.5% | In use |
| Design2Code Models turn screenshots of 484 real webpages into HTML and CSS. Automatic metrics compare the rendered page with the original on layout, text, position, color and CLIP similarity. | Web and app front ends | Mar 2024 | 87.2 | Fading |
| ProgramBench almost solved Agents rebuild 200 programs from only their compiled binaries and docs. The score is the share of tasks whose rebuilt program passes at least 95% of the hidden behavioral tests. | Building whole projects | May 2026 | 70.5% | In use |
| SQuAD v2 Models extract answers from Wikipedia paragraphs and must abstain on over 50,000 adversarial unanswerable questions. Scored by exact match and F1. | Reading comprehension | Jun 2018 | 81.8% | Dropped |
| SQuAD Models extract answer spans from Wikipedia paragraphs for over 100,000 crowdsourced questions. Scored by exact match and F1. | Reading comprehension | Jun 2016 | 89.3% | Dropped |
| MicroVQA Models answer expert-written multiple choice questions about biology microscopy images that test image understanding, hypothesis generation and experiment proposal. | General visual questions | Mar 2025 | 59.4% | Fading |
| LongCodeBench Coding model answers repository QA and fixes GitHub issues with context from 32K to 1M tokens. Accuracy and resolve rate. | Reasoning over long documents | May 2025 |  84% | Fading |
| AIR-Bench 2024 Models respond to 5,694 harmful prompts across 314 risk categories drawn from 8 government regulations and 16 company policies. Category-specific LLM judges score refusals. | Refusing harmful requests | Jul 2024 | 93.2% | Dropped |
| SimplerEnv Real-to-sim replicas of Google Robot and WidowX/Bridge setups. Policies scored by success rate on pick, move, drawer and Bridge tasks. | Simulated robots | May 2024 |  63% | Fading |
| GQA Models answer compositional questions about real-world images, generated from scene graphs. Scored by exact-match accuracy, often on the balanced test-dev split. | General visual questions | Feb 2019 | 62% | Fading |
| Adversarial SQuAD Models answer SQuAD questions whose passages contain an added distracting sentence. | Reading comprehension | Jul 2017 | System cards only | |
| Kernel Bench L3 Models write fast GPU kernels for the 50 full model architectures in KernelBench Level 3. Scored on correctness and speedup over PyTorch. | Speed and optimization | Dec 2024 | Fading | |
| Terminal-Bench 4.1 The Terminal-Bench 4.0 tasks with hardened verifiers: agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate. | Terminal tasks | Nov 2026 | 65.8% | In use |
| PrivacyLens Language model agents write emails, posts or messages from a prior tool trajectory without leaking sensitive details. A judge scores information leakage and helpfulness. | Refusing harmful requests | Aug 2024 | 49.2% | System cards only |
| HotpotQA Models answer 113k Wikipedia questions that need facts from several documents. The distractor setting supplies gold and distractor paragraphs. Scored by exact match or F1. | Reading comprehension | Sep 2018 |  92.1% | System cards only |
| KernelBench Replace PyTorch operators with faster custom GPU kernels. Scored on correctness and speedup. | GPU kernels | Dec 2024 |  28.8% | In use |
| RelBench Agents build models for prediction tasks over multi-table relational databases, graded on held-out targets. | Machine learning engineering | Jul 2024 | 38.4% | System cards only |
| STARE Models solve 4K spatial tasks such as 2D and 3D transformations, cube net folding, tangrams, perspective and temporal reasoning that people solve by visual simulation. | Spatial reasoning and counting | Jun 2025 |  61.8% | System cards only |
| WebQuestions Models answer factual questions mined from Google Suggest, with answers taken from the Freebase knowledge base. Scored by exact match. | Trivia and open questions | Oct 2013 | 28.2% | Dropped |
| BEHAVIOR Challenge 2025 50 long-horizon household activities on a Galaxea R1 Pro in OmniGibson, trained from 10,000 demos. Scored by Q-score and success. | Simulated robots | Sep 2025 | Fading | |
| LegalBench Models do 162 legal reasoning tasks written by legal professionals, covering six types of legal reasoning such as issue spotting and rule application. | Law | Aug 2023 | 88.6% | Not in cards |
| HELM Safety Stanford CRFM runs five safety benchmarks over six risk categories, such as violence, fraud and discrimination, and reports one leaderboard. | Refusing harmful requests | Nov 2024 | 98.6 | Not in cards |
| HELM Lite A lightweight HELM board that runs models with in-context learning on ten scenarios. They include NarrativeQA, NaturalQuestions, OpenbookQA, MMLU, GSM8K, MATH, LegalBench, MedQA and WMT 2014. | Composite indices | Dec 2023 | 93.8 | Not in cards |
| SEA-HELM Models are evaluated in Filipino, Indonesian, Tamil, Thai and Vietnamese across five pillars: NLP classics, LLM-specific tasks, linguistics, culture and safety. | Reasoning in many languages | Feb 2025 |  77.4 | Not in cards |
| HELM Capabilities Stanford CRFM runs language models on a curated set of scenarios, one per key capability, and averages the scores. Prompt-level results are public. | Composite indices | Mar 2025 | 81.9 | Not in cards |
| VHELM Vision-language models are scored across nine aspects, including perception, reasoning, knowledge, bias, fairness, multilinguality, robustness, toxicity and safety, from many existing datasets. | General visual questions | Oct 2024 | 87.1 | Not in cards |
| SMT 2025 Models answer 53 final-answer problems from the 2025 Stanford Math Tournament. MathArena scores the average accuracy over four attempts. | Competition problems with answers | May 2025 | 93.4% | Not in cards |
| IneqMath Models solve Olympiad-level inequality problems, recast as two checkable tasks: estimating a tight bound and predicting the relation between two expressions. IMO-level medalists wrote and reviewed the 200 test problems. Five model judges check the final answer and look for toy-case reasoning, logical gaps and numerical shortcuts, and a solution counts only if it passes all five. | Written proofs | Jun 2025 | 47% | Not in cards |
| HELM Arabic Stanford CRFM runs models on 7 Arabic benchmarks in the HELM harness and ranks them by mean score. The tasks are AlGhafa, ArabicMMLU, Arabic EXAMS, MadinahQA, AraTrust, ALRAGE and human-translated MMLU. | Knowledge in many languages | Dec 2025 | 84.1 | Not in cards |
| ThaiExam Models answer multiple-choice questions from standardized examinations in Thailand, such as ONET, IC, TGAT, TPAT-1 and A-Level. | Knowledge in many languages | Sep 2024 | 75.1% | Not in cards |
| HELM Arabic Enterprise Models do Arabic enterprise tasks in six parts. They write corporate articles, answer finance questions in three formats, and answer legal questions written by Arabic legal experts with and without the statute. | Knowledge in many languages | May 2026 | 77.9 | Not in cards |
| RoboRewardBench Vision-language models act as reward models on real robot videos from Open X-Embodiment and RoboArena, judging task success and partial progress. Counterfactual relabeling adds calibrated failures and near misses. | Real robots | Jan 2026 | 0.66 | Not in cards |
| AlpacaEval Models answer 805 open instructions, and a GPT-4 judge compares each reply with text-davinci-003's reply. The score is the win rate. | Overall chat quality | May 2023 | 97.7% | Not in cards |
| Collaborative Intelligence Benchmark A model works in a team with scripted teammates and live model partners, and a teammate creates a test: pressure to take an unsafe shortcut, misconduct to report, a need nobody states, a conflict, or a cheat that a weaker model left behind. The 240 tasks across 8 dimensions of teamwork come from real records, such as aviation accident reports and clinical cases, and from existing benchmarks, such as SWE-bench and negotiation datasets. Each task has a checkable outcome. A model is scored only on the tasks it solves alone, so the score measures teamwork, not ability. | Multi-agent games | Nov 2026 | 82.2% | Not in cards |
| Image2Struct Vision-language models turn images of webpages, LaTeX and musical scores into code that renders them. The render is compared with the input image. | Documents, charts and text in images | Oct 2024 | 94.7 | Not in cards |
| Visual Aesthetic Benchmark Models see a set of images with the same subject and intent and must pick the best one, and in a harder setting both the best and the worst. The 400 sets hold 1,195 commissioned works across fine art, photography and illustration. Labels come from expert judgments, and only sets where experts largely agreed are kept. | Taste and style | May 2026 | 40.3% | Not in cards |
| HELM Classic Models are evaluated with standardized prompts on the scenarios from the original HELM paper. Each scenario reports accuracy, calibration, robustness, fairness, bias, toxicity and efficiency. | Composite indices | Nov 2022 | 94.3 | Not in cards |
| MedHELM Models run a suite of medical tasks chosen by clinicians across five categories, from clinical decision support and note writing to patient messages and administration. | Medicine | Feb 2025 | 70.3 | Not in cards |
| TERMS-Bench Agents negotiate the price of a single item with simulated counterparts over up to 10 rounds of alternating offers. Counterparts follow six behavior families across three regimes. A program scores each agent from its logged actions on four separate axes: surplus captured, agreement calibration, belief error about the opponent and procedural violations. Extra modes score merchant profit and a bankroll over many sessions. | Multi-agent games | May 2026 | 0.69 | Not in cards |
| AHELM Audio-language models are scored across 10 aspects with standard prompts. The suite combines existing datasets with two new ones, PARADE for stereotypes and CoRe-Bench for conversational reasoning. | Audio understanding and voice | Aug 2025 | 80.3 | Not in cards |
| CaP-X (CaP-Bench) Coding agents write perception and control code for 100+ sim manipulation tasks across 8 abstraction and interaction tiers. Scored by task success. | Simulated robots | Mar 2026 | 32.3% | Not in cards |
| Embodied First Exam Coding agents answer frozen robot tasks from many benchmark suites with a program. Each program is verified in a separate sandbox, run on Harbor. | Simulated robots | Oct 2026 | 49.6% | Not in cards |
| EurekaBench Agents run long experiments and propose mechanisms that explain observations in six sciences. Mechanisms are scored on known constraints, predictive accuracy and the 306 expert-listed insights they should support. | Research science | Sep 2026 | 29.8 | Not in cards |
| FutureX Agents predict upcoming events collected daily from 195 websites in 11 domains, within a one-week window. Answers are scored once real results are crawled, with harder tiers weighted more. | Real-world events | Aug 2025 | 27.9 | Not in cards |
| HarnessAudit-Bench Agents run inside execution harnesses such as Claude Code, Codex and OpenClaw on 210 tasks across eight domains, from finance to software engineering, with safety rules built in. The audit reads the full trajectory through channels the agent cannot change. It scores whether the agent stays inside its permission and information-flow policies, completes checkpoints, and keeps both under prompt injection, vague goals and tool errors. | Refusing harmful requests | May 2026 | 0.41 | Not in cards |
| CooperBench Two coding agents each get a different feature for the same repository and must coordinate so both changes work together. Over 600 tasks span 12 libraries in 4 languages. | Fixing real repositories | Jan 2026 | 27.9% | Not in cards |
| DS-1000 Models write Python for 1,000 data science problems from StackOverflow across seven libraries such as NumPy and Pandas. Test cases and API constraints check each answer. | Data science and SQL | Nov 2022 | 53.9% | Not in cards |
| ParallelKernelBench Models replace PyTorch and NCCL reference code with multi-GPU CUDA kernels for 87 problems from codebases such as Megatron-LM and DeepSpeed. Scored on correctness and speed over the baseline. | GPU kernels | Jun 2026 | 32.2% | Not in cards |
| ScholarCatalyst Given a research question, retrieve the 'catalyst papers' that 184 lead authors labeled for 207 projects, searching only prior literature. Recall@20. | Research science | Oct 2026 | 53% | Not in cards |
| FinanceBench Models answer open-book questions about public companies using their filings, such as 10-Ks and earnings reports. The full set has 10,231 questions with answers and evidence strings, and 150 of them are public. The makers graded answers by hand, and later users often grade with a model judge. | Finance | Nov 2023 | 41% | Not in cards |
| MLAgentBench Agents run machine learning experiments on 13 tasks, from improving CIFAR-10 accuracy to research problems like BabyLM. They read and write files, run code and inspect outputs. | Machine learning engineering | Oct 2023 | 18.3% | Not in cards |
| AgentClinic Doctor agents interview simulated patients, order tests and reach a diagnosis in multimodal clinical scenarios built from MedQA and NEJM cases. | Medicine | May 2024 | Not in cards | |
| BEHAVIOR Challenge 2026 Robot policies solve 100 full-length household tasks in seven BEHAVIOR-1K scenes from RGB, depth and proprioception. Ranking uses average task success with partial credit. | Simulated robots | Jul 2026 | Not in cards | |
| BEHAVIOR-1K Embodied agents do 1,000 everyday household activities, such as cleaning and cooking, in realistic simulation built on OmniGibson. Success is checked against task goal definitions. | Simulated robots | Mar 2024 | Not in cards | |
| BountyBench Agents work on 25 real systems with bug bounty programs in three task types: detect a new vulnerability, exploit a given one, and patch it. Results are also valued in bounty dollars. | Finding and exploiting real bugs | May 2025 | Not in cards | |
| CaseHOLD Models read a passage from a US court decision with the holding of a cited case masked out and pick the correct holding from five choices. The dataset has more than 53,000 questions drawn from US case law. | Law | Apr 2021 | Not in cards | |
| CheXpert Models read chest radiographs and label up to 14 observations. The test labels come from board-certified radiologists. | Medicine | Jan 2019 | Not in cards | |
| CoQA Models answer a series of linked conversational questions about a passage, where later questions depend on earlier turns. Scored by word-level F1. | Reading comprehension | Aug 2018 | Not in cards | |
| HEIM Text-to-image models are scored on 12 aspects, such as alignment, quality, aesthetics, originality, reasoning, knowledge, bias and toxicity, with human raters and automatic metrics. | Image generation and editing | Nov 2023 | Not in cards | |
| HourVideo Multimodal models answer questions about first-person videos that run 20 to 120 minutes. The 500 videos come from Ego4D, and the 12,976 five-way multiple-choice questions cover summarization, perception, visual reasoning and navigation. Score is accuracy. | Long videos | Nov 2024 | Not in cards | |
| MedAgentBench Agents use FHIR APIs in a virtual electronic health record to do 300 patient tasks in 10 categories written by physicians. | Medicine | Jan 2025 | Not in cards | |
| MiniWoB++ Agents operate over 100 small synthetic web pages, such as forms, menus and date pickers, from a natural-language instruction. Each task returns a success reward. | Web browsing | Feb 2018 | Not in cards | |
| STaRK Retrieval systems find the right items for complex queries over knowledge bases that mix free text with relations between entities. The three domains are product search, academic paper search and precision medicine. Most queries are synthesized and checked by people, and a smaller set is written by people. | Questions that need search | Apr 2024 | Not in cards | |
| VIMA-Bench Robot policies carry out tabletop manipulation tasks given as multimodal prompts that mix text with images of objects or goal scenes. Thousands of tasks are generated procedurally, with over 600,000 expert trajectories for imitation learning. A four-level protocol tests generalization from new object placements up to unseen tasks, and scores are success rates. | Simulated robots | Oct 2022 | Not in cards | |
| WikiTableQuestions Models answer complex questions about semi-structured HTML tables from Wikipedia. Answers often need counting, comparison or aggregation across rows. Scored by answer accuracy against the gold answers. | Language reasoning | Aug 2015 | Not in cards | |
| WorldScore 3,000 next-scene generation tasks with camera-trajectory layouts. Scores 3D/4D and video world generators on controllability, quality, dynamics. | Video world models | Apr 2025 | Not in cards |