RoboLab
Zero-shot DROID Franka policies in Isaac Lab on 120 tabletop tasks at three instruction levels, 10 episodes each. Success rate.
Best entries on November 1st, 2026: agentic LM harness AMP1 at 86.0% success, then SenseNova-RoboRSI 57.1%. Best VLA or world model is Lakeside at 51.0%. NVIDIA runs internal rows, including its own models.
- Made by
- NVIDIA
- Released
- April 2026
- Kind
- Simulated robots
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Agent task
- Tasks
- All public
- Measures
- Capability
- Language
- English
- In and out
- Text, Images, Time series → Actions
- Interaction
- Agentic
- Environment
- Isaac Lab
- Items from
- Generated
- Refresh
- One fixed set
- Run by
- Anyone
- Last in a launch post
- November 2026
- Last model added
- November 2026
- Scores refreshed
- November 20th, 2026
In the authors’ words
The pursuit of general-purpose robotics has yielded impressive foundation models, yet simulation-based benchmarking remains a bottleneck due to rapid performance saturation and a lack of true generalization testing. Existing benchmarks often exhibit significant domain overlap between training and evaluation, trivializing success rates and obscuring insights into robustness. We introduce RoboLab, a simulation benchmarking framework designed to address these challenges. Concretely, our framework is designed to answer two questions: (1) to what extent can we understand the performance of a real-world policy by analyzing its behavior in simulation, and (2) which factor most strongly affect policy behavior. First, RoboLab enables human-authored and LLM-enabled generation of scenes and tasks in a robot- and policy-agnostic manner within a high-fidelity simulation environment. We introduce an accompanying RoboLab-120 benchmark, consisting of 120 tasks categorized into three competency axes: visual, procedural, relational, across three difficulty levels. Second, we introduce a systematic analysis of real-world policies that quantify both their performance and the sensitivity of their behavior to controlled perturbations, exposing significant performance gap in current state-of-the-art models. By providing granular metrics and a scalable toolset, RoboLab offers a scalable framework for evaluating the true generalization capabilities of task-generalist robotic policies. Project website: https://research.nvidia.com/labs/srl/projects/robolab/.
AMP1 tops the leaderboard with 87% Praxis-1 leads with 46.2%
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Praxis-1, at 46.2%. This view shows only models we track, each at its best.
Praxis-1 leads with 46.2%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 87.0 AMP1
- 57.1 SenseNova-RoboRSI
- 51.0 Lakeside
- 49.2 BiMind v0.5
- 46.2 Praxis-1
- 42.9 FLUX 3 Action
- 42.3 Cosmic
- 39.9 HiDream-O1-Embodied
- 39.7 Cosmos3-Nano-Policy
- 39.6 Atomic-WAM
- 39.0 OASIS WAM
- 34.4 Phoenix
- 33.3 BiMind v0.1
- 31.4 VoLo
- 28.8 Cosmos3-Edge-Policy
- 28.1 π0.5
- 25.7 DreamZero
- 15.5 π0-FAST
- 7.20
GR00T N1.6
- 5.50 paligemma-binning
- 5.00 π0
- 1 AMP1 specific instructions 87%
- 2 SenseNova-RoboRSI 57.1%
- 3 Lakeside 51%
- 4 BiMind v0.5 49.2%
- 5 Praxis-1 46.2%
- 6 FLUX 3 Action 42.9%
- 7 Cosmic 42.3%
- 8 HiDream-O1-Embodied 39.9%
- 9 Cosmos3-Nano-Policy specific instructions 39.7%
- 10 Atomic-WAM 39.6%
- 11 OASIS WAM 39%
- 12 Phoenix 34.4%
- 13 BiMind v0.1 33.3%
- 14 VoLo specific instructions 31.4%
- 15 Cosmos3-Edge-Policy specific instructions 28.8%
- 16 π0.5 specific instructions 28.1%
- 17 DreamZero 25.7%
- 18 π0-FAST 15.5%
- 19
GR00T N1.6 7.20%
- 20 paligemma-binning specific instructions 5.50%
- 21 π0 5.00%
Source: Official RoboLab leaderboard
- 1 Praxis-1 46.2%
- 2 FLUX 3 Action 42.9%
- 3 π0.5 specific instructions 28.1%
- 4 π0-FAST 15.5%
- 5
GR00T N1.6 7.20%
- 6 π0 5.00%
Source: Official RoboLab leaderboard
- 1 Praxis-1 46.2%
- 2 FLUX 3 Action 42.9%
- 3
Cosmos3-Nano-Policy-DROID 39.7%
- 4 π0.5 specific instructions 28.1%
- 5 π0-FAST 15.5%
- 6
GR00T N1.7 8.58%
- 7
GR00T N1.6 7.20%
- 8 π0 5.00%
Sources: Official RoboLab leaderboard and labs’ model cards
| # | Entry | Score ± 95% CI | Setting | Submitted by | Agent by | Released | Run on | Source |
|---|---|---|---|---|---|---|---|---|
| 1 | AMP1 open weights | 87% ±1.2 | specific instructions, Agentic LM | Apple | Apple | Oct 2026 | Official leaderboard | |
| 2 | SenseNova-RoboRSI open weights | 57.1% ±2.8 | Agentic LM | Nanyang Technological University | Nanyang Technological University | Oct 2026 | Official leaderboard | |
| 3 | Lakeside closed weights, 5B | 51% ±1.8 | LWM | PrI | PrI | Sep 2026 | Official leaderboard | |
| 4 | BiMind v0.5 closed weights, 8B | 49.2% ±2.9 | VLA | Bicameral Robotics | Bicameral Robotics | Oct 2026 | Official leaderboard | |
| 5 | Praxis-1 | 46.2% 43.4 to 49.1 | WAM | Runway | Runway | Nov 2026 | Nov 2026 | Official leaderboard |
| 6 | FLUX 3 Action | 42.9% ±2.8 | WAM | Black Forest Labs | Black Forest Labs | Sep 2026 | Sep 2026 | Official leaderboard |
| 7 | Cosmic closed weights, 0.4B | 42.3% ±2.8 | TAMP+FM | Geodesic | Geodesic | Oct 2026 | Official leaderboard | |
| 8 | HiDream-O1-Embodied closed weights, 6B | 39.9% ±2.8 | VLA | HiDream.ai | HiDream.ai | Sep 2026 | Official leaderboard | |
| 9 | Cosmos3-Nano-Policy 16B | 39.7% ±2.8 | specific instructions, WAM | NVIDIA Cosmos | May 2026 | Official leaderboard | ||
| 10 | Atomic-WAM open weights, 16.2B | 39.6% ±2.8 | VLM+WAM | ZJU-APRIL & ACE Robotics | ZJU-APRIL & ACE Robotics | Sep 2026 | Official leaderboard | |
| 11 | OASIS WAM closed weights | 39% ±2.8 | VLM+WAM | USTC & Synkrotron | USTC & Synkrotron | Sep 2026 | Official leaderboard | |
| 12 | Phoenix closed weights, 0.3B | 34.4% 31.7 to 37.2 | TAMP+FM | Geodesic | Geodesic | Aug 2026 | Official leaderboard | |
| 13 | BiMind v0.1 closed weights, 7B | 33.3% 30.7 to 36.1 | VLA | Bicameral Robotics | Bicameral Robotics | Aug 2026 | Official leaderboard | |
| 14 | VoLo open weights | 31.4% 28.8 to 34.1 | specific instructions, Agentic VLA | NVIDIA LPR | NVIDIA LPR | Sep 2026 | Official leaderboard | |
| 15 | Cosmos3-Edge-Policy 4B | 28.8% 26.3 to 31.5 | specific instructions, WAM | NVIDIA Cosmos | NVIDIA Cosmos | Jul 2026 | Official leaderboard | |
| 16 | π0.5 | 28.1% ±2.6 | specific instructions, VLA | Physical Intelligence | Apr 2025 | May 2026 | Official leaderboard | |
| 17 | DreamZero 14B | 25.7% ±2.5 | WAM | NVIDIA GEAR | May 2026 | Official leaderboard | ||
| 18 | π0-FAST | 15.5% 13.5 to 17.7 | VLA | Physical Intelligence | Jan 2025 | May 2026 | Official leaderboard | |
| 19 | | 7.20% 5.8 to 8.9 | VLA | NVIDIA GEAR | Dec 2025 | May 2026 | Official leaderboard | |
| 20 | paligemma-binning 3B | 5.50% 4.3 to 6.9 | specific instructions, VLA | Physical Intelligence | May 2026 | Official leaderboard | ||
| 21 | π0 | 5.00% 3.8 to 6.4 | VLA | Physical Intelligence | Oct 2024 | May 2026 | Official leaderboard |
| # | Model | Score ± 95% CI | Setting | Submitted by | Agent by | Released | Run on | Source |
|---|---|---|---|---|---|---|---|---|
| 1 | Praxis-1 | 46.2% 43.4 to 49.1 | WAM | Runway | Runway | Nov 2026 | Nov 2026 | Official leaderboard |
| 2 | FLUX 3 Action | 42.9% ±2.8 | WAM | Black Forest Labs | Black Forest Labs | Sep 2026 | Sep 2026 | Official leaderboard |
| 3 | π0.5 | 28.1% ±2.6 | specific instructions, VLA | Physical Intelligence | Apr 2025 | May 2026 | Official leaderboard | |
| 4 | π0-FAST | 15.5% 13.5 to 17.7 | VLA | Physical Intelligence | Jan 2025 | May 2026 | Official leaderboard | |
| 5 | | 7.20% 5.8 to 8.9 | VLA | NVIDIA GEAR | Dec 2025 | May 2026 | Official leaderboard | |
| 6 | π0 | 5.00% 3.8 to 6.4 | VLA | Physical Intelligence | Oct 2024 | May 2026 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Praxis-1 | 46.2% | WAM | Nov 2026 | Official leaderboard |
| 2 | FLUX 3 Action | 42.9% | WAM | Sep 2026 | Official leaderboard |
| 3 | 39.7% | May 2026 | Lab model card | ||
| 4 | π0.5 | 28.1% | specific instructions, VLA | Apr 2025 | Official leaderboard |
| 5 | π0-FAST | 15.5% | VLA | Jan 2025 | Official leaderboard |
| 6 | 8.58% | Jul 2026 | Lab model card | ||
| 7 | 7.20% | VLA | Dec 2025 | Official leaderboard | |
| 8 | π0 | 5.00% | VLA | Oct 2024 | Official leaderboard |
Labs report scores for 3 models in their cards. Switch the scores above to rank them together.
Rankings by part
We read 6 parts from the benchmark’s own leaderboard in 2 groups: difficulty and category. Each ranks models on one part of the test. Praxis-1 leads 4 of them, and FLUX 3 Action leads the other 2. Arrows show how far a model moved from its overall rank among the models we track.
Simple
6 models we track have a score here.
- 1 FLUX 3 Action ↑1 49.1%
- 2 Praxis-1 ↓1 46.9%
- 3 π0.5 29.8%
- 4 π0-FAST 20.2%
- 5
GR00T N1.6 8.80%
- 6 π0 7.20%
Moderate
6 models we track have a score here.
- 1 Praxis-1 47.9%
- 2 FLUX 3 Action 39.2%
- 3 π0.5 31.5%
- 4 π0-FAST 13.3%
- 5
GR00T N1.6 7.90%
- 6 π0 3.80%
Complex
6 models we track have a score here.
- 1 Praxis-1 40%
- 2 FLUX 3 Action 28.2%
- 3 π0.5 14.7%
- 4 π0-FAST 3.50%
- 5 π0 ↑1 0%
- 6
GR00T N1.6 ↓1 0%
Relational
6 models we track have a score here.
- 1 FLUX 3 Action ↑1 49.5%
- 2 Praxis-1 ↓1 49.3%
- 3 π0.5 33.8%
- 4 π0-FAST 23.3%
- 5
GR00T N1.6 10%
- 6 π0 6.40%
Visual
6 models we track have a score here.
- 1 Praxis-1 38.3%
- 2 FLUX 3 Action 35.6%
- 3 π0.5 25%
- 4 π0-FAST 10%
- 5
GR00T N1.6 5.00%
- 6 π0 1.90%
Procedural
6 models we track have a score here.
- 1 Praxis-1 40%
- 2 FLUX 3 Action 35.9%
- 3 π0.5 21.8%
- 4 π0-FAST 6.40%
- 5
GR00T N1.6 4.10%
- 6 π0 1.40%
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official RoboLab leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| Praxis-1 | WAM | 46.2% | |||
| FLUX 3 Action | WAM | 42.9% | |||
| π0.5 | specific instructions | VLA | 28.1% | ||
| π0.5 | VLA | 28% | |||
| π0-FAST | VLA | 15.5% | |||
| π0.5 | vague instructions | VLA | 15.2% | ||
| π0-FAST | specific instructions | VLA | 14.9% | ||
| π0-FAST | vague instructions | VLA | 9.20% | ||
| VLA | 7.20% | ||||
| vague instructions | VLA | 5.40% | |||
| specific instructions | VLA | 5.30% | |||
| π0 | VLA | 5.00% | |||
| π0 | specific instructions | VLA | 3.50% | ||
| π0 | vague instructions | VLA | 2.80% |
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: Official RoboLab leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| Praxis-1 | WAM | 46.2% | |||
| FLUX 3 Action | WAM | 42.9% | |||
| π0.5 | specific instructions | VLA | 28.1% | ||
| π0.5 | VLA | 28% | |||
| π0-FAST | VLA | 15.5% | |||
| π0.5 | vague instructions | VLA | 15.2% | ||
| π0-FAST | specific instructions | VLA | 14.9% | ||
| π0-FAST | vague instructions | VLA | 9.20% | ||
| VLA | 7.20% | ||||
| vague instructions | VLA | 5.40% | |||
| specific instructions | VLA | 5.30% | |||
| π0 | VLA | 5.00% | |||
| π0 | specific instructions | VLA | 3.50% | ||
| π0 | vague instructions | VLA | 2.80% |
3 models have held number one on the official leaderboard since its release in April 2026 4 models have held number one with claims included since its release in April 2026
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official RoboLab leaderboard and labs’ model cards
2 labs put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Nov 11, 2026 | Praxis-1 | 46.2 | RoboLab-120 Success |
| Jul 7, 2026 | 8.58 | RoboLab, zero-shot nvidia/GR00T-N1.7-DROID, 120 tasks, 40 episodes each, 412 of 4,800 successes. N1.6 reference 7.25% (examples/RoboLab README) |
One system card reports it
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| May 31, 2026 | | 39.7 | RoboLab-120 overall, specific language level, Cosmos3-Nano-Policy-DROID, Table 19 | Capabilities |
One lab claim next to the leaderboard
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.
More in Robotics
- ERQA In use 400 multiple-choice visual questions from robot and egocentric footage on spatial, trajectory, action and state reasoning. Accuracy.
Gemini Robotics ER 2 78.5%
- SimplerEnv In use Real-to-sim replicas of Google Robot and WidowX/Bridge setups. Policies scored by success rate on pick, move, drawer and Bridge tasks. Praxis-1 80%
- RoboDojo In use Generalist manipulation policies run 42 Isaac Sim tasks and 18 remote real-robot tasks. Scored by task score and success rate. GPT-6 Astra 27.6%
- Progress classification (Google DeepMind) In use Models assign each frame of a robot video to one of five task-progress bands from 0 to 100 percent. Scored by accuracy.
Gemini Robotics ER 2 57.4%
- Human proximity (Google DeepMind robotics) In use Models judge from robot camera views whether a human is within 1 meter, a spatial awareness check for safe robot behavior. Scored by accuracy.
Gemini Robotics ER 2 93%
- Moment finding (Google DeepMind) In use Models find the moment in a robot video when an event happens. Scored by accuracy and mean absolute time distance in seconds.
Gemini Robotics ER 2 91.3%
Data audit
All 11 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (21 entries), launch posts (2 scores) and system cards (1 score).
- Pass Models scored 8 models we track, plus 15 other entries on its leaderboard.
- Pass Newest score The newest is from November 2026, for Praxis-1.
- Note Frontier models None of its models are on the capabilities index.
- Pass Headroom The best is 46.2%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on research.nvidia.com: 14 rows for models we track and 15 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 2 pass
- Pass Lab claims The one pair agrees within 10 points.
- Pass Known issues No published audit found problems.