Benchmark Almanac

RoboLab

AMP1 87% tops its leaderboard Praxis-1 46.2% leads the models we track
Praxis-1 46.2% leads since November 2026
In use

Zero-shot DROID Franka policies in Isaac Lab on 120 tabletop tasks at three instruction levels, 10 episodes each. Success rate.

Best entries on November 1st, 2026: agentic LM harness AMP1 at 86.0% success, then SenseNova-RoboRSI 57.1%. Best VLA or world model is Lakeside at 51.0%. NVIDIA runs internal rows, including its own models.

Made by
NVIDIA
Released
April 2026
Kind
Simulated robots
Level
Everyday
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Images, Time series → Actions
Interaction
Agentic
Environment
Isaac Lab
Items from
Generated
Refresh
One fixed set
Run by
Anyone
In views
Agents, Vision, Robotics
Last in a launch post
November 2026
Last model added
November 2026
Scores refreshed
November 20th, 2026

In the authors’ words

The pursuit of general-purpose robotics has yielded impressive foundation models, yet simulation-based benchmarking remains a bottleneck due to rapid performance saturation and a lack of true generalization testing. Existing benchmarks often exhibit significant domain overlap between training and evaluation, trivializing success rates and obscuring insights into robustness. We introduce RoboLab, a simulation benchmarking framework designed to address these challenges. Concretely, our framework is designed to answer two questions: (1) to what extent can we understand the performance of a real-world policy by analyzing its behavior in simulation, and (2) which factor most strongly affect policy behavior. First, RoboLab enables human-authored and LLM-enabled generation of scenes and tasks in a robot- and policy-agnostic manner within a high-fidelity simulation environment. We introduce an accompanying RoboLab-120 benchmark, consisting of 120 tasks categorized into three competency axes: visual, procedural, relational, across three difficulty levels. Second, we introduce a systematic analysis of real-world policies that quantify both their performance and the sensitivity of their behavior to controlled perturbations, exposing significant performance gap in current state-of-the-art models. By providing granular metrics and a scalable toolset, RoboLab offers a scalable framework for evaluating the true generalization capabilities of task-generalist robotic policies. Project website: https://research.nvidia.com/labs/srl/projects/robolab/.

Abstract of RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies (arXiv:2604.09860, CC0)
21
Entries on its own leaderboard, 6 models we track
2
Launch posts list it, from 2 labs
1
System card reports it, from 1 lab

AMP1 tops the leaderboard with 87% Praxis-1 leads with 46.2%

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Praxis-1, at 46.2%. This view shows only models we track, each at its best.

Praxis-1 leads with 46.2%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

  1. 1 AMP1 specific instructions 87%
  2. 2 SenseNova-RoboRSI 57.1%
  3. 3 Lakeside 51%
  4. 4 BiMind v0.5 49.2%
  5. 5 Praxis-1 46.2%
  6. 6 FLUX 3 Action 42.9%
  7. 7 Cosmic 42.3%
  8. 8 HiDream-O1-Embodied 39.9%
  9. 9 Cosmos3-Nano-Policy specific instructions 39.7%
  10. 10 Atomic-WAM 39.6%
  11. 11 OASIS WAM 39%
  12. 12 Phoenix 34.4%
  13. 13 BiMind v0.1 33.3%
  14. 14 VoLo specific instructions 31.4%
  15. 15 Cosmos3-Edge-Policy specific instructions 28.8%
  16. 16 π0.5 specific instructions 28.1%
  17. 17 DreamZero 25.7%
  18. 18 π0-FAST 15.5%
  19. 19 GR00T N1.6 7.20%
  20. 20 paligemma-binning specific instructions 5.50%
  21. 21 π0 5.00%

Source: Official RoboLab leaderboard

Source: Official RoboLab leaderboard

Sources: Official RoboLab leaderboard and labs’ model cards

#EntryScore ± 95% CISettingSubmitted byAgent byReleasedRun onSource
1 AMP1 open weights 87% ±1.2 specific instructions, Agentic LM Apple Apple Oct 2026 Official leaderboard
2 SenseNova-RoboRSI open weights 57.1% ±2.8 Agentic LM Nanyang Technological University Nanyang Technological University Oct 2026 Official leaderboard
3 Lakeside closed weights, 5B 51% ±1.8 LWM PrI PrI Sep 2026 Official leaderboard
4 BiMind v0.5 closed weights, 8B 49.2% ±2.9 VLA Bicameral Robotics Bicameral Robotics Oct 2026 Official leaderboard
5 Praxis-1 46.2% 43.4 to 49.1 WAM Runway Runway Nov 2026 Nov 2026 Official leaderboard
6 FLUX 3 Action 42.9% ±2.8 WAM Black Forest Labs Black Forest Labs Sep 2026 Sep 2026 Official leaderboard
7 Cosmic closed weights, 0.4B 42.3% ±2.8 TAMP+FM Geodesic Geodesic Oct 2026 Official leaderboard
8 HiDream-O1-Embodied closed weights, 6B 39.9% ±2.8 VLA HiDream.ai HiDream.ai Sep 2026 Official leaderboard
9 Cosmos3-Nano-Policy 16B 39.7% ±2.8 specific instructions, WAM NVIDIA Cosmos May 2026 Official leaderboard
10 Atomic-WAM open weights, 16.2B 39.6% ±2.8 VLM+WAM ZJU-APRIL & ACE Robotics ZJU-APRIL & ACE Robotics Sep 2026 Official leaderboard
11 OASIS WAM closed weights 39% ±2.8 VLM+WAM USTC & Synkrotron USTC & Synkrotron Sep 2026 Official leaderboard
12 Phoenix closed weights, 0.3B 34.4% 31.7 to 37.2 TAMP+FM Geodesic Geodesic Aug 2026 Official leaderboard
13 BiMind v0.1 closed weights, 7B 33.3% 30.7 to 36.1 VLA Bicameral Robotics Bicameral Robotics Aug 2026 Official leaderboard
14 VoLo open weights 31.4% 28.8 to 34.1 specific instructions, Agentic VLA NVIDIA LPR NVIDIA LPR Sep 2026 Official leaderboard
15 Cosmos3-Edge-Policy 4B 28.8% 26.3 to 31.5 specific instructions, WAM NVIDIA Cosmos NVIDIA Cosmos Jul 2026 Official leaderboard
16 π0.5 28.1% ±2.6 specific instructions, VLA Physical Intelligence Apr 2025 May 2026 Official leaderboard
17 DreamZero 14B 25.7% ±2.5 WAM NVIDIA GEAR May 2026 Official leaderboard
18 π0-FAST 15.5% 13.5 to 17.7 VLA Physical Intelligence Jan 2025 May 2026 Official leaderboard
19 GR00T N1.6 7.20% 5.8 to 8.9 VLA NVIDIA GEAR Dec 2025 May 2026 Official leaderboard
20 paligemma-binning 3B 5.50% 4.3 to 6.9 specific instructions, VLA Physical Intelligence May 2026 Official leaderboard
21 π0 5.00% 3.8 to 6.4 VLA Physical Intelligence Oct 2024 May 2026 Official leaderboard
#ModelScore ± 95% CISettingSubmitted byAgent byReleasedRun onSource
1 Praxis-1 46.2% 43.4 to 49.1 WAM Runway Runway Nov 2026 Nov 2026 Official leaderboard
2 FLUX 3 Action 42.9% ±2.8 WAM Black Forest Labs Black Forest Labs Sep 2026 Sep 2026 Official leaderboard
3 π0.5 28.1% ±2.6 specific instructions, VLA Physical Intelligence Apr 2025 May 2026 Official leaderboard
4 π0-FAST 15.5% 13.5 to 17.7 VLA Physical Intelligence Jan 2025 May 2026 Official leaderboard
5 GR00T N1.6 7.20% 5.8 to 8.9 VLA NVIDIA GEAR Dec 2025 May 2026 Official leaderboard
6 π0 5.00% 3.8 to 6.4 VLA Physical Intelligence Oct 2024 May 2026 Official leaderboard
#ModelScoreSettingDateSource
1 Praxis-1 46.2% WAM Nov 2026 Official leaderboard
2 FLUX 3 Action 42.9% WAM Sep 2026 Official leaderboard
3 Cosmos3-Nano-Policy-DROID 39.7% May 2026 Lab model card
4 π0.5 28.1% specific instructions, VLA Apr 2025 Official leaderboard
5 π0-FAST 15.5% VLA Jan 2025 Official leaderboard
6 GR00T N1.7 8.58% Jul 2026 Lab model card
7 GR00T N1.6 7.20% VLA Dec 2025 Official leaderboard
8 π0 5.00% VLA Oct 2024 Official leaderboard

Labs report scores for 3 models in their cards. Switch the scores above to rank them together.

Rankings by part

We read 6 parts from the benchmark’s own leaderboard in 2 groups: difficulty and category. Each ranks models on one part of the test. Praxis-1 leads 4 of them, and FLUX 3 Action leads the other 2. Arrows show how far a model moved from its overall rank among the models we track.

Simple

6 models we track have a score here.

  1. 1 FLUX 3 Action ↑1 49.1%
  2. 2 Praxis-1 ↓1 46.9%
  3. 3 π0.5 29.8%
  4. 4 π0-FAST 20.2%
  5. 5 GR00T N1.6 8.80%
  6. 6 π0 7.20%

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
π0.5 VLA
vague instr… specific in…
28.1%
π0-FAST VLA
vague instr… specific in…
15.5%
GR00T N1.6 VLA
specific in…
7.20%
π0 VLA
vague instr…
5.00%

Source: Official RoboLab leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Praxis-1 WAM 46.2%
FLUX 3 Action WAM 42.9%
π0.5 specific instructions VLA 28.1%
π0.5 VLA 28%
π0-FAST VLA 15.5%
π0.5 vague instructions VLA 15.2%
π0-FAST specific instructions VLA 14.9%
π0-FAST vague instructions VLA 9.20%
GR00T N1.6 VLA 7.20%
GR00T N1.6 vague instructions VLA 5.40%
GR00T N1.6 specific instructions VLA 5.30%
π0 VLA 5.00%
π0 specific instructions VLA 3.50%
π0 vague instructions VLA 2.80%

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
π0.5 VLA
vague instr… specific in…
28.1%
π0-FAST VLA
vague instr… specific in…
15.5%
GR00T N1.6 VLA
specific in…
7.20%
π0 VLA
vague instr…
5.00%

Source: Official RoboLab leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Praxis-1 WAM 46.2%
FLUX 3 Action WAM 42.9%
π0.5 specific instructions VLA 28.1%
π0.5 VLA 28%
π0-FAST VLA 15.5%
π0.5 vague instructions VLA 15.2%
π0-FAST specific instructions VLA 14.9%
π0-FAST vague instructions VLA 9.20%
GR00T N1.6 VLA 7.20%
GR00T N1.6 vague instructions VLA 5.40%
GR00T N1.6 specific instructions VLA 5.30%
π0 VLA 5.00%
π0 specific instructions VLA 3.50%
π0 vague instructions VLA 2.80%

3 models have held number one on the official leaderboard since its release in April 2026 4 models have held number one with claims included since its release in April 2026

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Physical Intelligence Black Forest Labs Runway NVIDIA
Own leaderboard 3 leaders
Praxis-1 46.2%
Lab claims 2 leaders
Praxis-1 46.2%

Sources: Official RoboLab leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Praxis-1 Cosmos3-Nano-Policy-DROID π0.5 π0-FAST π0 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Praxis-1 Cosmos3-Nano-Policy-DROID

Sources: Official RoboLab leaderboard and labs’ model cards

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

NVIDIA
Runway
DateLaunch postScoreAs the post writes it
Nov 11, 2026 Praxis-1 46.2 RoboLab-120 Success
Jul 7, 2026 GR00T N1.7 8.58 RoboLab, zero-shot nvidia/GR00T-N1.7-DROID, 120 tasks, 40 episodes each, 412 of 4,800 successes. N1.6 reference 7.25% (examples/RoboLab README)

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
May 31, 2026 Cosmos3-Nano-Policy-DROID technical report 39.7 RoboLab-120 overall, specific language level, Cosmos3-Nano-Policy-DROID, Table 19 Capabilities

One lab claim next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Praxis-1 46.2% RoboLab-120 Success 46.2% Leaderboard, WAM Same

More in Robotics

All 172 in Robotics
  • ERQA In use 400 multiple-choice visual questions from robot and egocentric footage on spatial, trajectory, action and state reasoning. Accuracy. Gemini Robotics ER 2 78.5%
  • SimplerEnv In use Real-to-sim replicas of Google Robot and WidowX/Bridge setups. Policies scored by success rate on pick, move, drawer and Bridge tasks. Praxis-1 80%
  • RoboDojo In use Generalist manipulation policies run 42 Isaac Sim tasks and 18 remote real-robot tasks. Scored by task score and success rate. GPT-6 Astra 27.6%
  • Progress classification (Google DeepMind) In use Models assign each frame of a robot video to one of five task-progress bands from 0 to 100 percent. Scored by accuracy. Gemini Robotics ER 2 57.4%
  • Human proximity (Google DeepMind robotics) In use Models judge from robot camera views whether a human is within 1 meter, a spatial awareness check for safe robot behavior. Scored by accuracy. Gemini Robotics ER 2 93%
  • Moment finding (Google DeepMind) In use Models find the moment in a robot video when an event happens. Scored by accuracy and mean absolute time distance in seconds. Gemini Robotics ER 2 91.3%

Data audit

All 11 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (21 entries), launch posts (2 scores) and system cards (1 score).
  • Pass Models scored 8 models we track, plus 15 other entries on its leaderboard.
  • Pass Newest score The newest is from November 2026, for Praxis-1.
  • Note Frontier models None of its models are on the capabilities index.
  • Pass Headroom The best is 46.2%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on research.nvidia.com: 14 rows for models we track and 15 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 2 pass
  • Pass Lab claims The one pair agrees within 10 points.
  • Pass Known issues No published audit found problems.

How we check