SimplerEnv
Real-to-sim replicas of Google Robot and WidowX/Bridge setups. Policies scored by success rate on pick, move, drawer and Bridge tasks.
Self-reported under mixed zero-shot and fine-tuned protocols. Papers quote conflicting baselines. The WidowX track is near ceiling. Google Robot tracks still separate models.
- Made by
- UC San Diego + Stanford + UC Berkeley + Google DeepMind
- Released
- May 2024
- Kind
- Simulated robots
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Agent task
- Tasks
- All public
- Measures
- Capability
- Language
- English
- In and out
- Text, Images, Time series → Actions
- Interaction
- Agentic
- Environment
- A simulator
- Items from
- Generated
- Refresh
- One fixed set
- Run by
- Anyone
- Based on
- A version of another benchmark
- Last in a launch post
- November 2026
- Last model added
- November 2026
In the authors’ words
The field of robotics has made significant advances towards generalist robot manipulation policies. However, real-world evaluation of such policies is not scalable and faces reproducibility challenges, which are likely to worsen as policies broaden the spectrum of tasks they can perform. We identify control and visual disparities between real and simulated environments as key challenges for reliable simulated evaluation and propose approaches for mitigating these gaps without needing to craft full-fidelity digital twins of real-world environments. We then employ these approaches to create SIMPLER, a collection of simulated environments for manipulation policy evaluation on common real robot setups. Through paired sim-and-real evaluations of manipulation policies, we demonstrate strong correlation between policy performance in SIMPLER environments and in the real world. Additionally, we find that SIMPLER evaluations accurately reflect real-world policy behavior modes such as sensitivity to various distribution shifts. We open-source all SIMPLER environments along with our workflow for creating new environments at https://simpler-env.github.io to facilitate research on general-purpose manipulation policies and simulated evaluation frameworks.
Praxis-1 leads with 80%
Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.
- 1 Praxis-1 80%
- 2
GR00T N1.5 63%
- 3
GR00T N1.7 62.3%
- 4
GR00T N1.6 62.1%
Source: Labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Praxis-1 | 80% | Nov 2026 | Lab model card | |
| 2 | | 63% | Sep 2025 | Lab model card | |
| 3 | | 62.3% | Jul 2026 | Lab model card | |
| 4 | | 62.1% | Dec 2025 | Lab model card |
2 models have held number one in labs’ own numbers since its release in May 2024
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Labs’ model cards
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Labs’ model cards
2 labs put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Nov 11, 2026 | Praxis-1 | 80 | 80.0% success on SimplerEnv |
| Jul 7, 2026 | 62.3 | SimplerEnv Bridge (WidowX) average 62.3%. Fractal (Google robot) average 72.5%. N1.6 re-run 56.6% and 52.0% (examples/SimplerEnv README) | |
| Dec 16, 2025 | 62.1 | SimplerEnv Bridge (WidowX) average 62.07%, GR00T-N1.6-bridge checkpoint. Fractal (Google robot) average 67.66% (examples/SimplerEnv README) |
One system card reports it
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Sep 26, 2025 | | 63 | SimplerEnv Bridge (WidowX) average 63% over 7 tasks with 300 trials each. Fractal (Google robot) average 41% (examples/SimplerEnv README) | Capabilities |
More in Robotics
- ERQA In use 400 multiple-choice visual questions from robot and egocentric footage on spatial, trajectory, action and state reasoning. Accuracy.
Gemini Robotics ER 2 78.5%
- RoboLab In use Zero-shot DROID Franka policies in Isaac Lab on 120 tabletop tasks at three instruction levels, 10 episodes each. Success rate. Praxis-1 46.2%
- RoboDojo In use Generalist manipulation policies run 42 Isaac Sim tasks and 18 remote real-robot tasks. Scored by task score and success rate. GPT-6 Astra 27.6%
- Progress classification (Google DeepMind) In use Models assign each frame of a robot video to one of five task-progress bands from 0 to 100 percent. Scored by accuracy.
Gemini Robotics ER 2 57.4%
- Human proximity (Google DeepMind robotics) In use Models judge from robot camera views whether a human is within 1 meter, a spatial awareness check for safe robot behavior. Scored by accuracy.
Gemini Robotics ER 2 93%
- Moment finding (Google DeepMind) In use Models find the moment in a robot video when an event happens. Scored by accuracy and mean absolute time distance in seconds.
Gemini Robotics ER 2 91.3%
Data audit
All 5 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
- Note Independent runs Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (1 score).
- Note Models scored Only 4 models we track.
- Pass Newest score The newest is from November 2026, for Praxis-1.
- Note Frontier models None of its models are on the capabilities index.
- Pass Headroom The best is 80%, under 85%, so it still separates models.
Leaderboard import 1 note
- Note Importer No public leaderboard to read.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.