RoboEval

MolmoAct 2 44.3%

π0.5 40.5% leads

MolmoAct 2 44.3% leads

Fading

Labs’ own numbers All independent runs Include claimed scores

Robot policies perform eight bimanual manipulation tasks with controlled variations in simulation. Scores add behavior and progress metrics, such as coordination and efficiency, to task success.

The paper ships over three thousand expert demonstrations. Version 2 of the paper came out on May 4th, 2026. Ai2 reports a 0 to 1 score for MolmoAct 2.

Made by: University of Washington and collaborators (Wang et al.)
Released: July 2025
Kind: Simulated robots
Level: Everyday
Scored: Checked automatically
Answers: Agent task
Tasks: All public

Last in a launch post: May 2026, dropped before it saturated
Last model added: May 2026
Scores refreshed: November 12th, 2026

In the authors’ words

We introduce RoboEval, a structured evaluation framework and benchmark for robotic manipulation that augments binary success with principled behavioral and outcome metrics. Existing evaluations often collapse performance into outcome counts, masking differences in execution quality and obscuring failure structure. RoboEval provides eight bimanual tasks with systematically controlled variations, more than three thousand expert demonstrations, and a modular simulation platform for reproducible experimentation. All tasks are instrumented with standardized metrics that quantify efficiency, coordination, and safety/stability, as well as outcome measures that trace stagewise progress and localize failure modes. Through extensive experiments with state-of-the-art visuomotor policies, we validate these metrics by analyzing their stability under variation, discriminative power across policies with similar success rates, and correlation with task success. Project Page: https://robo-eval.github.io

Abstract of RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation (arXiv:2507.00435, CC0)

Model scored in lab cards

Launch post lists it, from 1 lab

System card reports it, from 1 lab

MolmoAct 2 leads with 44.3%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

π0.5 leads with 40.5% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

MolmoAct 2 leads with 44.3%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

Bars Table

44.3 MolmoAct 2

1 MolmoAct 2 44.3%

Source: Labs’ model cards

40.5 π0.5

1 π0.5 40.5%

Source: Official RoboEval leaderboard

1 MolmoAct 2 44.3%
2 π0.5 40.5%

Sources: Labs’ model cards and official RoboEval leaderboard

#	Model	Score	Setting	Date	Source
1	MolmoAct 2	44.3%		May 2026	Lab model card

#	Model	Score	Setting	Date	Source
1	π0.5	40.5%		Apr 2025	Official leaderboard

#	Model	Score	Setting	Date	Source
1	MolmoAct 2	44.3%		May 2026	Lab model card
2	π0.5	40.5%		Apr 2025	Official leaderboard

The benchmark's own leaderboard has 6 entries, 1 of them models we track. Switch the scores above to rank them together.

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far

Sources: Official RoboEval leaderboard and labs’ model cards

Labs headline it less often now, last in May 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Ai2

Date	Launch post	Score	As the post writes it
May 5, 2026	MolmoAct 2	0.443	RoboEval (0 to 1, higher is better)

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

Date	Document	Score	As the document writes it	Section
May 4, 2026	MolmoAct 2 technical report	44.3	RoboEval success rate averaged over 8 bimanual tasks, fine-tuned (6.3 text and Figure 6)	Capabilities