Robotics
We track 110 benchmarks in robotics. They run from everyday level to expert level.
OpenAI held the top robotics score 22% of the contested time
That's about 10 of the 46 months since January 2023, averaged over these benchmarks. We only count time while other labs are still testing, and stop once a model passes 85%. Faded stretches no longer count.
Stanford University Physical Intelligence Ai2 Qwen Anthropic Shanghai AI Lab Alibaba International Digital Commerce (AIDC-AI) Z.ai OpenAI ByteDance Seed Google DeepMind
Everyday Most adults, without training
University Undergraduates and entry-level professionals
Expert Specialists with years of training, often over hours of work
Sources: Official leaderboards and labs’ model cards
Share of contested time at number one
OpenAI 22% Anthropic 19% 
Qwen 15% 
ByteDance Seed 11% 
Google DeepMind 11% Physical Intelligence 7% 
Alibaba International Digital Commerce (AIDC-AI) 5% 
Stanford University 3% Z.ai 3% 
Shanghai AI Lab 2%
Qwen 15% ByteDance Seed 11% Google DeepMind 11% Physical Intelligence 7% Alibaba International Digital Commerce (AIDC-AI) 5% Stanford University 3% Z.ai 3% Shanghai AI Lab 2% Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Each benchmark counts from the one source its own page leads with: its official leaderboard when that is current, else Epoch AI’s own runs, else labs’ own numbers. We date independent runs by the model’s release and lab reports by when they came out.
Qwen Anthropic Shanghai AI Lab Alibaba International Digital Commerce (AIDC-AI) Google DeepMind Physical Intelligence OpenAI Black Forest Labs Ai2
Everyday Most adults, without training
University Undergraduates and entry-level professionals
Expert Specialists with years of training, often over hours of work
Source: Official leaderboards
Share of contested time at number one, independent runs only
Anthropic 28% Physical Intelligence 19% Google DeepMind 19% OpenAI 14% Shanghai AI Lab 9% Alibaba International Digital Commerce (AIDC-AI) 8% Qwen 3%
Google DeepMind 19% OpenAI 14% Shanghai AI Lab 9% Alibaba International Digital Commerce (AIDC-AI) 8% Qwen 3% Sources: Official leaderboards and Epoch AI (CC BY 4.0)
Every independent run: each benchmark’s own leaderboard, Epoch AI’s runs and boards Epoch AI copied, dated by the model’s release. We leave out benchmarks that only labs report.
Stanford University Physical Intelligence Ai2 Qwen Anthropic Shanghai AI Lab Alibaba International Digital Commerce (AIDC-AI) Google DeepMind OpenAI ByteDance Seed
Everyday Most adults, without training
University Undergraduates and entry-level professionals
Expert Specialists with years of training, often over hours of work
Sources: Official leaderboards and labs’ model cards
Share of contested time at number one, claimed scores included
OpenAI 21% Anthropic 18% Google DeepMind 17% Qwen 14% ByteDance Seed 10% Physical Intelligence 6% Shanghai AI Lab 6% Alibaba International Digital Commerce (AIDC-AI) 5% Stanford University 3%
Google DeepMind 17% Qwen 14% ByteDance Seed 10% Physical Intelligence 6% Shanghai AI Lab 6% Alibaba International Digital Commerce (AIDC-AI) 5% Stanford University 3% Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Every score, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.
From everyday to expert level
Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.
Everyday
Most adults, without training
- ERQA
- RefSpatial-Bench
- RoboRewardBench
- EmbodiedBench EB-ALFRED
- EmbodiedBench EB-Habitat
- EmbodiedBench EB-Manipulation
- EmbodiedBench EB-Navigation
- LIBERO
- InteractiveBench
- DataCenterBench Tier 1
- Embodied First Exam
- RoboColiseum
- RoboDojo-RC Tier 1
- RoboCasa
- RoboCasa GR1 Tabletop Tasks
- RoboDojo
- RoboLab
- VLA-Arena
- Instrument reading (Google DeepMind)
- OST-Bench
- PhAIL DROID: Everyday tasks
- Progress classification (Google DeepMind)
- Human proximity (Google DeepMind robotics)
- OpenEQA
- RobotWorld
- SimplerEnv
- WorldAuditBench
- ArmnetBench
- Butter-Bench
- LBM Eval
- LIBERO-MAX
- RoboCasa365
- Success detection (Google DeepMind)
- Where2Place
- Act2Answer
- DriveBench Behavior
- DriveBench Perception
- DriveBench Planning
- DriveBench Prediction
- LIBERO-Plus
- LIBERO-PRO
- RoboArena
- RoboChallenge
- RoboEval
- Roboharm
- LADDER
- MolmoSpaces
- Moment finding (Google DeepMind)
- Pantograph frontier-LLM robot benchmark (Pandroid)
- PhAIL
- RoboGate
- RoboTwin 2.0
- SafeLIBERO
- BEHAVIOR Challenge 2025
- LIBERO-Para
- RMBench
- RoboChallenge Table30 v2
- RoboMemArena
- AA-Robotics
- ALFRED
- ALFWorld
- ASIMOV v1
- ASIMOV-Agentic
- AutoEval
- BEHAVIOR Challenge 2026
- BEHAVIOR-1K
- CALVIN
- DriveBench
- DuoBench
- EBench
- EmbodiedBench
- EmbodiedCLBench
- ESI-Bench
- FlashDexRetarget 50-motion benchmark
- Galahad
- GaP: Graph-as-Policy
- Gemini Robotics generalization benchmark
- HumanoidBench
- HumanoidToolBench
- ManipArena
- ManiSkill3
- Meta-World
- metal-arm-harness
- MolmoAct2 sim_eval
- OCBench
- ReactHuman
- REALM
- RefSpatial-Expand-Bench
- RefSpatial-Expand-Bench Location
- RefSpatial-Expand-Bench Placement
- RLBench
- Robocurve physical tool-use benchmark (unidentified)
- Robocurve simple tasks (LLM control of YAM arms)
- Robot Capabilities Index (Robocurve)
- Robotic Origami Challenge (IROS 2026)
- RoboTwin 1.0
- Room-Across-Room (RxR)
- TEACh
- TraceSpatial-Bench
- UMI-Bench
- VIMA-Bench
- VLABench
Expert
Specialists with years of training, often over hours of work
Another 1 has no level. Levels don’t apply to arenas, safety evaluations or composite indices.
By kind
Simulated robots
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| ArtifactArena Models design robot bodies, controllers and strategy for a simulated sumo-style arena. Artifacts fight head-to-head and models get Elo ratings. | Expert | Ranked by matches | 1,539 | Not in cards |
| EmbodiedBench EB-ALFRED Agents follow household instructions through high-level actions in AI2-THOR. The score is the published average task success percentage across EB-ALFRED capability groups. | Everyday | Checked automatically | 67.7% | Not in cards |
| EmbodiedBench EB-Habitat Agents follow instructions to find and rearrange objects through high-level skills in Habitat 2.0. The score is the published average task success percentage. | Everyday | Checked automatically | 58.7% | Not in cards |
| EmbodiedBench EB-Manipulation Agents use visual observations to control a robot arm for object manipulation in simulation. The score is the published average task success percentage. | Everyday | Checked automatically | 28.5% | Not in cards |
| EmbodiedBench EB-Navigation Agents use visual observations and low-level movement actions to reach target objects in AI2-THOR. The score is the published average task success percentage. | Everyday | Checked automatically | 53.7% | Not in cards |
| LIBERO Simulated Franka tabletop tasks in four 10-task suites. Policies fine-tuned on 50 demos per task. Success rate over ~50 trials. | Everyday | Checked automatically |  97.2% | Fading |
| InteractiveBench Multimodal models act as a wearable assistant that sees through a simulated person's eyes and guides that person through household tasks. State-based checks in a BEHAVIOR-1K home score the outcome. | Everyday | Checked automatically | 41.7% | Not in cards |
| Dual-Arm Sim Frontier models control a two-armed robot in a simulated operating room from cameras and sensors. Tasks include swapping instruments, replacing a drug cartridge and packing a surgical kit. | Expert | Checked automatically | 59.8 | Not in cards |
| CaP-X (CaP-Bench) Coding agents write perception and control code for 100+ sim manipulation tasks across 8 abstraction and interaction tiers. Scored by task success. | University | Checked automatically | 32.3% | Not in cards |
| Embodied First Exam Coding agents answer frozen robot tasks from many benchmark suites with a program. Each program is verified in a separate sandbox, run on Harbor. | Everyday | Checked automatically | 49.6% | Not in cards |
| RoboColiseum AgiBot Genie robot in Genie Sim 3.0 (Isaac Sim): instruction, robustness, manipulation and spatial boards after post-training on provided sim data. | Everyday | Checked automatically | 76.3% | Not in cards |
| RoboCasa Simulated mobile manipulators and humanoids do 100 kitchen tasks, 25 atomic and 75 composite, across 120 scenes. Scored by success rate. | Everyday | Checked automatically |  70.8% | Fading |
| RoboCasa GR1 Tabletop Tasks A simulated Fourier GR-1 humanoid with dexterous hands does 24 tabletop pick-and-place tasks, including unseen object and container pairs. Success rate per task. | Everyday | Checked automatically | 50% | Fading |
| RoboDojo Generalist manipulation policies run 42 Isaac Sim tasks and 18 remote real-robot tasks. Scored by task score and success rate. | Everyday | Checked automatically | 29% | In use |
| RoboLab Zero-shot DROID Franka policies in Isaac Lab on 120 tabletop tasks at three instruction levels, 10 episodes each. Success rate. | Everyday | Checked automatically | 42.9% | In use |
| VLA-Arena VLAs fine-tuned on level-0 tabletop tasks, tested on 170 tasks in 11 suites at harder unseen levels L1/L2. Success rate. | Everyday | Checked automatically | 72.1% | Not in cards |
| Robo-bench General agents control a simulated 7-DoF Franka arm and a 29-DoF humanoid through motor commands, sensors and camera images only, across 24 tasks. | Expert | Checked automatically | 28.3% | Not in cards |
| RobotWorld Multimodal agents control robots through robot interfaces in simulation. Its 84 tasks span manipulation, mobile manipulation, locomotion, driving and aerial control, with executable success checks. | Everyday | Settled by events | 19.1% | Not in cards |
| SimplerEnv Real-to-sim replicas of Google Robot and WidowX/Bridge setups. Policies scored by success rate on pick, move, drawer and Bridge tasks. | Everyday | Checked automatically | 63% | Fading |
| LBM Eval Bimanual robot policies run 49 manipulation tasks in the Drake simulator from random starting conditions. Scored by success rate per task. | Everyday | Checked automatically | 72.4% | Fading |
| LIBERO-MAX Robot policies face 8,000 paired cases where the scene changes during execution, such as a target moving, the viewpoint shifting or an obstacle appearing. | Everyday | Checked automatically | 66.9% | Not in cards |
| RoboCasa365 Mobile-manipulator kitchen sim: train on 300-task Human300 data, test on 50 tasks incl. 16 unseen composite tasks. Success rate. | Everyday | Checked automatically | 23.9% | Not in cards |
| Act2Answer Vision-language-action models answer commonsense and world-knowledge questions by acting. Each question becomes a short tabletop episode, and the robot answers by placing an object on one of the candidate answers. The score is the success rate per knowledge category, which reduces the effect of weak low-level control. | Everyday | Checked automatically | 68% | Not in cards |
| LIBERO-Plus LIBERO policies run 10,030 perturbed tasks across seven dimensions: layout, camera, robot start state, language, light, background and sensor noise. Success rate. | Everyday | Checked automatically | 69.6% | Not in cards |
| LIBERO-PRO LIBERO policies tested under object, position, instruction, task and environment perturbations. 50 episodes per task. Success rate. | Everyday | Checked automatically | 53% | Not in cards |
| Real2Sim Coding agents get a robot recording from DROID (three camera videos, stereo calibration, joint and gripper commands and the task instruction) and build an interactive simulation of the scene. Each scene is scored on how closely it matches the recording, on replaying the recorded actions, and on whether a fixed robot policy completes the task in it. | Expert | Checked automatically | 0.58 | Not in cards |
| RoboEval Robot policies perform eight bimanual manipulation tasks with controlled variations in simulation. Scores add behavior and progress metrics, such as coordination and efficiency, to task success. | Everyday | Checked automatically | 44.3% | Fading |
| RoboWorld Coding agents control simulated robots through an HTTP API to do physical jobs such as welding, sewing, pouring and forklift driving. Each simulation checks the task and returns a reward from 0 to 1. | Expert | Checked automatically | 0.06 | Not in cards |
| LADDER Policies, orchestrators and frontier-model agents do 145 long-horizon tabletop tasks of 1 to 10 subgoals in three simulators. A task counts only when every object ends in place. | Everyday | Checked automatically | 63.3% | Not in cards |
| MolmoSpaces Zero-shot Franka (DROID) policies in held-out procedurally generated MuJoCo houses: pick, place, open, close. ~1,000 episodes per task. | Everyday | Checked automatically | 37.7% | Fading |
| RoboGate 68 industrial pick-and-place scenarios (nominal, edge, adversarial, randomized) for a Franka arm in Isaac Sim. Organizers run submitted VLAs and report success. | Everyday | Checked automatically | 0% | Not in cards |
| RoboTwin 2.0 Bimanual Aloha-AgileX sim: 50 tasks, policies trained on 50 clean demos per task, 100 trials each in clean and domain-randomized scenes. | Everyday | Checked automatically | 58.3% | Not in cards |
| SafeLIBERO VLA policies run 16 LIBERO tasks with an added obstacle at two levels, near the target or in the path. 50 episodes per scenario. Scored by collision avoidance and task success. | Everyday | Checked automatically | 57.8% | Not in cards |
| BEHAVIOR Challenge 2025 50 long-horizon household activities on a Galaxea R1 Pro in OmniGibson, trained from 10,000 demos. Scored by Q-score and success. | Everyday | Checked automatically | Fading | |
| LIBERO-Para VLA policies run LIBERO tasks with over 4,000 paraphrased instructions that vary object and action wording separately. Scored by success rate and the difficulty-aware PRIDE metric. | Everyday | Checked automatically | 71.4% | Not in cards |
| RMBench Policies run 9 simulated bimanual manipulation tasks on RoboTwin 2.0 that need memory of earlier observations, at several levels of memory complexity. Success rate. | Everyday | Checked automatically | 10.4% | Not in cards |
| AA-Robotics Language models in coding harnesses control a 6-axis robot arm with a gripper over HTTP, using camera views, to finish eight everyday tabletop tasks. | Everyday | Checked automatically | Not in cards | |
| ALFRED Agents follow natural-language instructions to finish household tasks in the AI2-THOR simulator from first-person views, such as rinsing a mug and putting it in the coffee maker. The benchmark has expert demonstrations for 25,000 directives with both high-level goals and step-by-step instructions. Scores count tasks completed and goal conditions met in seen and unseen rooms. | Everyday | Checked automatically | Not in cards | |
| ALFWorld Agents complete household goals such as putting a washed apple in the fridge in a text world aligned with the ALFRED embodied simulator. Success is whether the goal state is reached. | Everyday | Checked automatically | Not in cards | |
| BEHAVIOR Challenge 2026 Robot policies solve 100 full-length household tasks in seven BEHAVIOR-1K scenes from RGB, depth and proprioception. Ranking uses average task success with partial credit. | Everyday | Checked automatically | Not in cards | |
| BEHAVIOR-1K Embodied agents do 1,000 everyday household activities, such as cleaning and cooking, in realistic simulation built on OmniGibson. Success is checked against task goal definitions. | Everyday | Checked automatically | Not in cards | |
| CALVIN Simulated Franka follows chains of five language instructions from 34 tasks. ABC-to-D tests an unseen environment. Average chain length. | Everyday | Checked automatically | Not in cards | |
| DuoBench Eleven bimanual Franka FR3 Duo tasks in four coordination categories, in simulation and partly real. Stage-based scoring beyond binary success. | Everyday | Checked automatically | Not in cards | |
| EBench 26 Isaac Sim household mobile-manipulation task types annotated by 5 capability and 4 generalization dimensions. Platform-scored success rate. | Everyday | Checked automatically | Not in cards | |
| EmbodiedBench Multimodal LLMs act as embodied agents on 1,128 tasks in four simulators, from household planning (EB-ALFRED, EB-Habitat) to navigation and manipulation. Success rate. | Everyday | Checked automatically | Not in cards | |
| EmbodiedCLBench Self-evolving embodied agents learn tasks in sequence in simulation. The test measures whether they keep past experience and transfer it to new tasks. | Everyday | Checked automatically | Not in cards | |
| FlashDexRetarget 50-motion benchmark Retarget 50 human hand-object motions (HOT3D, TACO, OakInk2) to XHand and Sharpa hands in Isaac Sim. Scored by object pose-error success thresholds. | Everyday | Checked automatically | Not in cards | |
| Galahad Vision-language-action policies are tested on whether they follow instructions. One word of the instruction changes and the policy must change its behavior. | Everyday | Checked automatically | Not in cards | |
| GaP: Graph-as-Policy Multi-agent harness that compiles language tasks into typed robot skill graphs. Tested on 8 variational-automation tasks (4 sim, 4 real) by success rate. | Everyday | Checked automatically | Not in cards | |
| HumanoidBench Policies control a simulated Unitree H1 humanoid with dexterous hands on 15 whole-body manipulation and 12 locomotion tasks. Scored by task return or success. | Everyday | Checked automatically | Not in cards | |
| ManiSkill3 GPU-parallel robot simulator and task suite across 12 domains for manipulation learning. Success rate per task. | Everyday | Checked automatically | Not in cards | |
| Meta-World Robot policies do simulated tabletop manipulation tasks, scored by success rate. Splits test multi-task learning, such as MT10 and MT50, and meta-learning, such as ML45. | Everyday | Checked automatically | Not in cards | |
| MolmoAct2 sim_eval Zero-shot ManiSkill evaluation of MolmoAct2 policies: put-everything-in-box tasks for Franka DROID and bimanual YAM, 10 episodes per task. | Everyday | Checked automatically | Not in cards | |
| OCBench 28 simulated manipulation tasks with human-like scripted demonstrators for controlled BC and RL studies. Policies scored by task success rate. | Everyday | Checked automatically | Not in cards | |
| ReactHuman Multimodal models act as the decision core of a household robot. They must react at once to sudden physical hazards, such as catching a slipping plate. | Everyday | Settled by events | Not in cards | |
| REALM High-fidelity simulation benchmark for VLA generalization: 15 perturbation factors, 7 skills, 3,500+ objects. Success rate validated against real-robot results. | Everyday | Checked automatically | Not in cards | |
| RLBench Robot arm policies attempt 100 hand-designed manipulation tasks in simulation, from reaching a target and opening a door to multi-stage tasks such as putting a tray in an oven. Each task has camera, depth and joint observations and an unlimited supply of motion-planned demonstrations. Scores are task success rates. | Everyday | Checked automatically | Not in cards | |
| RoboTwin 1.0 Dual-arm policies train on generated digital-twin demos and are tested on simulated single-arm and dual-arm tasks, with matching real-world data. Success rate. | Everyday | Checked automatically | Not in cards | |
| Room-Across-Room (RxR) Agents follow navigation instructions through photo-realistic indoor scenes to reach a goal. The instructions are in English, Hindi and Telugu, and each word is time-aligned to the annotator's position. Scores include success rate and how closely the path follows the reference route. | Everyday | Checked automatically | Not in cards | |
| TEACh Agents complete household tasks in simulated homes, such as making coffee or preparing breakfast, guided by a dialogue between a Commander who knows the task and a Follower who acts. The dataset holds over 3,000 human dialogues and defines three benchmarks: acting from dialogue history, predicting the trajectory from a dialogue, and completing the task as two agents. Scores are task success rates. | Everyday | Checked automatically | Not in cards | |
| VIMA-Bench Robot policies carry out tabletop manipulation tasks given as multimodal prompts that mix text with images of objects or goal scenes. Thousands of tasks are generated procedurally, with over 600,000 expert trajectories for imitation learning. A four-level protocol tests generalization from new object placements up to unseen tasks, and scores are success rates. | Everyday | Checked automatically | Not in cards | |
| VLABench 100 language-conditioned manipulation task categories needing world knowledge, implicit intent and long-horizon reasoning. Evaluates VLAs and VLMs. | Everyday | Checked automatically | Not in cards |
Real robots
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| RoboRewardBench Vision-language models act as reward models on real robot videos from Open X-Embodiment and RoboArena, judging task success and partial progress. Counterfactual relabeling adds calibrated failures and near misses. | Everyday | Checked automatically | 0.66 | Not in cards |
| Drone-Bench Agent writes code for a small real drone across five tasks (reconstruct, localize, navigate, detect, follow). Each scored against a human-written baseline. | Expert | Checked automatically | 97.3% | Not in cards |
| DataCenterBench Tier 1 Frontier models watch through cameras and drive two real robot arms on tabletop tasks rebuilt from data-center equipment manuals. LLM judges score each try from photos. | Everyday | Graded by a model | 40% | Not in cards |
| RoboDojo-RC Tier 1 Six real RoboDojo tasks on I2RT YAM arms, 20 attempts each. Robot policies and tool-calling LLMs scored by rubric progress. | Everyday | Graded by people | 48.2% | Not in cards |
| PhAIL DROID: Everyday tasks Policies run 286 language-instructed tabletop tasks worded as DROID annotations on a real Franka FR3. Operators mark stages reached within 240 seconds. | Everyday | Graded by people | 68.1% | Not in cards |
| ArmnetBench 12 tabletop tasks on low-cost SO-101 arms. Each policy trained on the same 50 demos per task. 30 human-scored rollouts. | Everyday | Graded by people | 47.6% | Not in cards |
| Butter-Bench A language model controls a real robot to find and deliver a pack of butter in an office. It is scored on completing each step of the task. | Everyday | Checked automatically | 37% | Not in cards |
| RoboArena Double-blind pairwise A/B trials of generalist DROID policies on real Franka arms, free-form tasks by volunteer evaluators. Elo-style rating. | Everyday | Ranked by votes | 1,608 | Not in cards |
| RoboChallenge Fine-tuned policies run 10 real rollouts on each of 30 tabletop tasks across four robot types. Scored by success rate and progress. | Everyday | Checked automatically | 42.7% | Not in cards |
| Pantograph frontier-LLM robot benchmark (Pandroid) GPT-6 Astra and Claude Fable 5.1 drive Pandroid robots through a fixed tool harness on eight simple manipulation tasks, 10 blind-graded attempts each. | Everyday | Graded by people | 36.3% | Not in cards |
| PhAIL Real Franka bin-to-bin picking of four object types. VLAs scored on throughput and completion against human and teleop references. | Everyday | Checked automatically | 75.1% | Not in cards |
| RoboChallenge Table30 v2 Policies run 30 new real tabletop tasks on ARX5, UR5, ALOHA and DOS-W1 robots. Each task has a 10-point stage rubric for partial credit. | Everyday | Checked automatically | 0.73% | Not in cards |
| RoboMemArena Robot policies do long-horizon tasks in partially observable settings that need memory of past observations and actions, in simulation and on real robots. | Everyday | Checked automatically | 21.5% | Not in cards |
| AutoEval Autonomous real-robot evaluation: WidowX arms run five Bridge tasks with learned success detectors and resets. 50 rollouts per task. | Everyday | Graded by a model | Not in cards | |
| Gemini Robotics generalization benchmark Robot policies run 85 real bi-arm manipulation tasks. About 80% test generalization to new scenes, new wording of instructions or new actions. Scored by average progress and success rate. | Everyday | Settled by events | Not in cards | |
| HumanoidToolBench Humanoid robot policies pick a suitable tool and use it, walking when needed, across 18 tasks in simulation and on a real Unitree G1. | Everyday | Settled by events | Not in cards | |
| ManipArena 20 real bimanual tasks on X2Robot platforms: 10 execution, 5 semantic reasoning and 5 mobile manipulation, with in-domain and shifted trials. Partial-credit scoring. | Everyday | Graded by people | Not in cards | |
| metal-arm-harness Safety-enveloped CLI that lets any coding agent drive a real MakerMods Metal or Maker arm one command at a time from camera JPEGs. | Everyday | Checked automatically | Not in cards | |
| Robocurve physical tool-use benchmark (unidentified) Unnamed Robocurve physical tool-use robot task. | Everyday | Graded by people | Not in cards | |
| Robocurve simple tasks (LLM control of YAM arms) Frontier LLMs control bimanual I2RT YAM arms through Inspect Robots tools on block-into-bowl, puzzle, push-ball and stack-bowls. Staged scores, 20 trials. | Everyday | Checked automatically | Not in cards | |
| Robot Capabilities Index (Robocurve) An index fitted like Epoch ECI over five RoboDojo-RC Tier 1 tasks run by frontier LLMs on real YAM arms. Anchored at GPT-5.6 Sol minus 3 and GPT-6 Astra plus 3. | Everyday | Graded by people | Not in cards | |
| Robotic Origami Challenge (IROS 2026) Fold a six-fold paper airplane on a supplied bimanual Sharpa-hand rig. Origami judges give pass/fail, and passes are ranked by folding time. | Everyday | Graded by people | Not in cards | |
| UMI-Bench 10 real tabletop tasks trained from standardized UMI hand-held gripper demos. 50 rollouts per task. Progress score and success rate. | Everyday | Checked automatically | Not in cards |
Embodied reasoning
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| ERQA 400 multiple-choice visual questions from robot and egocentric footage on spatial, trajectory, action and state reasoning. Accuracy. | Everyday | Checked automatically | 78.5% | In use |
| RefSpatial-Bench Real images where the model points to an object or free space defined by multi-step spatial relations. Scored by mask hit. | Everyday | Checked automatically | 73.6% | Fading |
| Instrument reading (Google DeepMind) Models read gauges, sight glasses, digital displays, linear scales, rulers and thermometers from robot camera images. ER 2 tests 10 instrument types. | Everyday | Checked automatically | 65.7% | Fading |
| OST-Bench Vision-language models act as an agent exploring a 3D scene and answer spatial questions from the views seen so far. The set has 10k questions over 1.4k scenes. | Everyday | Checked automatically | 53.4% | Not in cards |
| Progress classification (Google DeepMind) Models assign each frame of a robot video to one of five task-progress bands from 0 to 100 percent. Scored by accuracy. | Everyday | Checked automatically | 57.4% | In use |
| Human proximity (Google DeepMind robotics) Models judge from robot camera views whether a human is within 1 meter, a spatial awareness check for safe robot behavior. Scored by accuracy. | Everyday | Checked automatically | 93% | In use |
| OpenEQA Agents answer over 1,600 open-vocabulary questions about real indoor scenes from recorded episodes or active exploration. A judge model scores answers against human answers. | Everyday | Graded by a model | 57.9 | Dropped |
| WorldAuditBench Multimodal agents explore interactive 3D worlds to find anomalies such as floating objects, walls that can be walked through or out-of-place objects. | Everyday | Checked automatically | 42.3% | Not in cards |
| ASIMOV Models spot physical hazards in text and generated videos from real injury reports, time safety interventions in video, and follow robot payload and handling constraints. | Checked automatically | 97.9% | Fading | |
| Success detection (Google DeepMind) Embodied reasoning models judge from robot camera images or video whether a manipulation task succeeded. Later versions use raw video to catch mid-execution failures. | Everyday | Checked automatically | 90% | Fading |
| Where2Place 100 cluttered real photos with free-space placement instructions. The model points, scored correct if inside the ground-truth mask. | Everyday | Checked automatically | 64% | System cards only |
| DriveBench Behavior Models answer driving behavior questions. The score is the published GPT evaluation score for this task family. | Everyday | 54.6 | Not in cards | |
| DriveBench Perception Models answer driving perception questions. The score is the published GPT evaluation score for this task family. | Everyday | 35.2 | Not in cards | |
| DriveBench Planning Models answer driving planning questions. The score is the published GPT evaluation score for this task family. | Everyday | 57.0 | Not in cards | |
| DriveBench Prediction Models answer driving prediction questions. The score is the published GPT evaluation score for this task family. | Everyday | 48.9 | Not in cards | |
| Roboharm Agents and VLAs controlling bimanual YAM arms get five harmful-looking instructions in fixed scenes, such as stabbing a doll. Reviewers label refusals and completions. | Everyday | Graded by people | 20% | System cards only |
| Moment finding (Google DeepMind) Models find the moment in a robot video when an event happens. Scored by accuracy and mean absolute time distance in seconds. | Everyday | Checked automatically | 91.3% | In use |
| ASIMOV v1 Models judge whether robot actions in scenes generated from real images and hospital injury reports are undesirable or unsafe. Scored by alignment with human labels. | Everyday | Checked automatically | Not in cards | |
| ASIMOV-Agentic Robot reasoning agents decide when to refuse, stop or ask a human. Parts cover safety constraints, people within one meter, safe VLA tool calls, task feasibility and unclear instructions or scenes. | Everyday | Checked automatically | System cards only | |
| DriveBench Vision-language models answer 20,498 questions about 19,200 driving frames, covering perception, prediction, planning and behavior. Inputs come in 17 settings: clean images, corrupted images and text only. The test checks whether answers rest on the image or on general knowledge. | Everyday | Not in cards | ||
| ESI-Bench Embodied agents act to reveal what is unseen, such as occluded structure, containment and function, then reason about how observations change with action. | Everyday | Checked automatically | Not in cards | |
| RefSpatial-Expand-Bench Models point to an object or free space described by multi-step spatial relations, in more indoor scenes plus new outdoor streets and parking lots. Scored by mask hit. | Everyday | Checked automatically | Not in cards | |
| RefSpatial-Expand-Bench Location Models solve spatial location questions in the RefSpatial expansion set. The score is the published overall accuracy. | Everyday | Checked automatically | Not in cards | |
| RefSpatial-Expand-Bench Placement Models solve spatial placement questions in the RefSpatial expansion set. The score is the published overall accuracy. | Everyday | Checked automatically | Not in cards | |
| TraceSpatial-Bench Models plan an object-centric 3D trace, an ordered list of metric 3D waypoints, from language instructions in real cluttered indoor scenes with depth. | Everyday | Checked automatically | Not in cards |