LBM Eval

Fading

Bimanual robot policies run 49 manipulation tasks in the Drake simulator from random starting conditions. Scored by success rate per task.

About 125 GitHub stars. Version 1.1.0 (fall 2025) is newer than the snapshot the LBM 1.0 paper used, and the README says it may not reproduce the paper. TRI's VLA Foundry cards score 16 seen and 3 unseen tasks.

Made by: Toyota Research Institute
Released: May 2025
Kind: Simulated robots
Level: Everyday
Scored: Checked automatically
Answers: Agent task
Tasks: All public

Last in a launch post: April 2026, dropped before it saturated
Last model added: April 2026

Models scored in lab cards

Launch posts list it, from 1 lab

Foundry-Qwen3VLA-2.1B leads with 72.4%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Bars Table

1 Foundry-Qwen3VLA-2.1B 72.4%
2 Foundry-VLA-1.7B 34.1%

Source: Labs’ model cards

#	Model	Score	Setting	Date	Source
1	Foundry-Qwen3VLA-2.1B	72.4%		Apr 2026	Lab model card
2	Foundry-VLA-1.7B	34.1%		Apr 2026	Lab model card

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card

Source: Labs’ model cards

Labs headline it less often now, last in April 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Toyota Research Institute

Date	Launch post	Score	As the post writes it
Apr 21, 2026	Foundry-VLA-1.7B	34.1	OSS simulator, success rate on 16 seen tasks, 200 rollouts per task. Unseen (3 tasks) 7.5% (Evaluation Results table)
Apr 21, 2026	Foundry-Qwen3VLA-2.1B	72.4	OSS simulator, success rate on 16 seen tasks, 200 rollouts per task. Unseen (3 tasks) 13.5% (Evaluation Results table)

Foundry-Qwen3VLA-2.1B leads with 72.4%

Scores over time

Labs headline it less often now, last in April 2026

More in Robotics