Benchmark Almanac

ERQA

Gemini Robotics ER 2 78.5% leads since July 2026
Gemini 2.5 Pro Jun 2025 55.6% leads since June 2025
Gemini Robotics ER 2 78.5% leads since July 2026
In use

400 multiple-choice visual questions from robot and egocentric footage on spatial, trajectory, action and state reasoning. Accuracy.

Not saturated: best 78.5 (Gemini Robotics ER 2, DeepMind chart), chance ~25%. Source tables (DeepMind, Unitree, Embodied Arena) use different setups. Do not pool.

Made by
Google DeepMind
Released
March 2025
Kind
Embodied reasoning
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Strongest model tested
Qwen 3.8 Max #29 on the capability index
Last in a launch post
September 2026
Last model added
October 2026
Scores refreshed
November 12th, 2026
36
Models scored in lab cards
28
Launch posts list it, from 7 labs
15
System cards report it, from 7 labs

Gemini Robotics ER 2 leads with 78.5%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Gemini 2.5 Pro leads with 55.6% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Gemini Robotics ER 2 leads with 78.5%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

36 models. Scroll sideways for the rest.

  1. 78.5 Gemini Robotics ER 2
  2. 78.3 Qwen3.8 Max (0902)
  3. 77.8 Qwen 3.8 Max
  4. 72.3 Qwen3.8 Flash
  5. 72.0 Seed2.1 Pro
  6. 71.5 Seed2.0 Lite (0428)
  7. 71.0 Qwen3.8-Omni-Flash
  8. 69.8 Qwen3.7-Plus
  9. 68.5 Seed2.0 Pro
  10. 67.5 Qwen3.5 397B-A17B
  11. 65.7 GPT-5
  12. 65.7 Qwen 3.6 Plus
  13. 65.5 Qwen 3.8 27B
  14. 64.8 Qwen3.5-35B-A3B
  15. 64.7 Muse Spark
  16. 62.9 GPT-5 mini
  17. 62.5 Qwen3.6 27B
  18. 62.0 Qwen3.5-122B-A10B
  19. 60.5 Qwen3.5-27B
  20. 58.8 Seed1.8
  21. 57.5 Nemotron 3.5 Super VL
  22. 55.5 Qwen3.5-9B
  23. 54.8 gemini-robotics-er-1.5-preview
  24. 54.8 Qwen3.5-Omni-Plus
  25. 54.0 Qwen3.5-4B
  26. 52.5 Qwen3-VL-235B-A22B
  27. 50.1 GPT-5 nano
  28. 50.0 GLM-4.5V
  29. 48.9 STEP3-VL-10B
  30. 47.8 GLM-4.6V
  31. 46.8 InternVL3.5-241B-A28B
  32. 45.8 GLM-4.1V-9B-Thinking
  33. 44.0 Cosmos3-Nano
  34. 43.8 Qwen3.5-2B
  35. 42.0 Cosmos3-Edge
  36. 34.5 Qwen3.5-0.8B
  1. 1 Gemini Robotics ER 2 78.5%
  2. 2 Qwen3.8 Max (0902) 78.3%
  3. 3 Qwen 3.8 Max 77.8%
  4. 4 Qwen3.8 Flash 72.3%
  5. 5 Seed2.1 Pro 72%
  6. 6 Seed2.0 Lite (0428) 71.5%
  7. 7 Qwen3.8-Omni-Flash 71%
  8. 8 Qwen3.7-Plus 69.8%
  9. 9 Seed2.0 Pro 68.5%
  10. 10 Qwen3.5 397B-A17B 67.5%
  11. 11 GPT-5 65.7%
  12. 12 Qwen 3.6 Plus 65.7%
  13. 13 Qwen 3.8 27B 65.5%
  14. 14 Qwen3.5-35B-A3B 64.8%
  15. 15 Muse Spark 64.7%
  16. 16 GPT-5 mini 62.9%
  17. 17 Qwen3.6 27B 62.5%
  18. 18 Qwen3.5-122B-A10B 62%
  19. 19 Qwen3.5-27B 60.5%
  20. 20 Seed1.8 58.8%
  21. 21 Nemotron 3.5 Super VL 57.5%
  22. 22 Qwen3.5-9B 55.5%
  23. 23 gemini-robotics-er-1.5-preview 54.8%
  24. 24 Qwen3.5-Omni-Plus 54.8%
  25. 25 Qwen3.5-4B 54%
  26. 26 Qwen3-VL-235B-A22B 52.5%
  27. 27 GPT-5 nano 50.1%
  28. 28 GLM-4.5V 50%
  29. 29 STEP3-VL-10B 48.9%
  30. 30 GLM-4.6V 47.8%
  31. 31 InternVL3.5-241B-A28B 46.8%
  32. 32 GLM-4.1V-9B-Thinking 45.8%
  33. 33 Cosmos3-Nano 44%
  34. 34 Qwen3.5-2B 43.8%
  35. 35 Cosmos3-Edge 42%
  36. 36 Qwen3.5-0.8B 34.5%

Source: Labs’ model cards

Source: Official ERQA leaderboard

53 models. Scroll sideways for the rest.

  1. 78.5 Gemini Robotics ER 2
  2. 78.3 Qwen3.8 Max (0902)
  3. 77.8 Qwen 3.8 Max
  4. 72.3 Qwen3.8 Flash
  5. 72.0 Seed2.1 Pro
  6. 71.5 Seed2.0 Lite (0428)
  7. 71.0 Qwen3.8-Omni-Flash
  8. 69.8 Qwen3.7-Plus
  9. 68.5 Seed2.0 Pro
  10. 67.5 Qwen3.5 397B-A17B
  11. 65.7 GPT-5
  12. 65.7 Qwen 3.6 Plus
  13. 65.5 Qwen 3.8 27B
  14. 64.8 Qwen3.5-35B-A3B
  15. 64.7 Muse Spark
  16. 62.9 GPT-5 mini
  17. 62.5 Qwen3.6 27B
  18. 62.0 Qwen3.5-122B-A10B
  19. 60.5 Qwen3.5-27B
  20. 58.8 Seed1.8
  21. 57.5 Nemotron 3.5 Super VL
  22. 55.6 Gemini 2.5 Pro
  23. 55.5 Qwen3.5-9B
  24. 54.8 gemini-robotics-er-1.5-preview
  25. 54.8 Qwen3.5-Omni-Plus
  26. 54.0 Qwen3.5-4B
  27. 52.5 Qwen3-VL-235B-A22B
  28. 50.1 GPT-5 nano
  29. 50.0 GLM-4.5V
  30. 48.9 STEP3-VL-10B
  31. 47.8 GLM-4.6V
  32. 46.8 InternVL3.5-241B-A28B
  33. 45.8 GLM-4.1V-9B-Thinking
  34. 44.0 Cosmos3-Nano
  35. 43.8 Qwen3.5-2B
  36. 42.0 Cosmos3-Edge
  37. 41.7 InternVL3-38B
  38. 41.5 o3
  39. 41.5 GPT-5.2
  40. 40.2 Gemini 2.0 Flash
  41. 39.3 Qwen-VL-Max
  42. 38.8 InternVL3-14B
  43. 37.1 InternVL3-8B
  44. 36.3 InternVL3.5-8B
  45. 34.8 GPT-4o
  46. 34.5 Qwen3.5-0.8B
  47. 34.4 LLaVA-OneVision 7B
  48. 34.4 Qwen2-VL-7B-Instruct
  49. 34.4 Qwen2.5-VL-7B
  50. 33.6 VILA1.5-13B
  51. 32.6 Qwen2.5-VL-3B
  52. 30.7 o4-mini
  53. 14.3 Claude 3.7 Sonnet
  1. 1 Gemini Robotics ER 2 78.5%
  2. 2 Qwen3.8 Max (0902) 78.3%
  3. 3 Qwen 3.8 Max 77.8%
  4. 4 Qwen3.8 Flash 72.3%
  5. 5 Seed2.1 Pro 72%
  6. 6 Seed2.0 Lite (0428) 71.5%
  7. 7 Qwen3.8-Omni-Flash 71%
  8. 8 Qwen3.7-Plus 69.8%
  9. 9 Seed2.0 Pro 68.5%
  10. 10 Qwen3.5 397B-A17B 67.5%
  11. 11 GPT-5 65.7%
  12. 12 Qwen 3.6 Plus 65.7%
  13. 13 Qwen 3.8 27B 65.5%
  14. 14 Qwen3.5-35B-A3B 64.8%
  15. 15 Muse Spark 64.7%
  16. 16 GPT-5 mini 62.9%
  17. 17 Qwen3.6 27B 62.5%
  18. 18 Qwen3.5-122B-A10B 62%
  19. 19 Qwen3.5-27B 60.5%
  20. 20 Seed1.8 58.8%
  21. 21 Nemotron 3.5 Super VL 57.5%
  22. 22 Gemini 2.5 Pro Jun 2025 55.6%
  23. 23 Qwen3.5-9B 55.5%
  24. 24 gemini-robotics-er-1.5-preview 54.8%
  25. 25 Qwen3.5-Omni-Plus 54.8%
  26. 26 Qwen3.5-4B 54%
  27. 27 Qwen3-VL-235B-A22B 52.5%
  28. 28 GPT-5 nano 50.1%
  29. 29 GLM-4.5V 50%
  30. 30 STEP3-VL-10B 48.9%
  31. 31 GLM-4.6V 47.8%
  32. 32 InternVL3.5-241B-A28B 46.8%
  33. 33 GLM-4.1V-9B-Thinking 45.8%
  34. 34 Cosmos3-Nano 44%
  35. 35 Qwen3.5-2B 43.8%
  36. 36 Cosmos3-Edge 42%
  37. 37 InternVL3-38B 41.7%
  38. 38 o3 41.5%
  39. 39 GPT-5.2 41.5%
  40. 40 Gemini 2.0 Flash Feb 2025 40.2%
  41. 41 Qwen-VL-Max 39.3%
  42. 42 InternVL3-14B 38.8%
  43. 43 InternVL3-8B 37.1%
  44. 44 InternVL3.5-8B 36.3%
  45. 45 GPT-4o Mar 2025 34.8%
  46. 46 Qwen3.5-0.8B 34.5%
  47. 47 LLaVA-OneVision 7B 34.4%
  48. 48 Qwen2-VL-7B-Instruct 34.4%
  49. 49 Qwen2.5-VL-7B 34.4%
  50. 50 VILA1.5-13B 33.6%
  51. 51 Qwen2.5-VL-3B 32.6%
  52. 52 o4-mini 30.7%
  53. 53 Claude 3.7 Sonnet 14.3%

Sources: Labs’ model cards and official ERQA leaderboard

#ModelScoreSettingDateSource
1 Gemini Robotics ER 2 78.5% Jul 2026 Lab model card
2 Qwen3.8 Max (0902) 78.3% Sep 2026 Lab model card
3 Qwen 3.8 Max 77.8% Aug 2026 Lab model card
4 Qwen3.8 Flash 72.3% Aug 2026 Lab model card
5 Seed2.1 Pro 72% Jun 2026 Lab model card
6 Seed2.0 Lite (0428) 71.5% Apr 2026 Lab model card
7 Qwen3.8-Omni-Flash 71% Sep 2026 Lab model card
8 Qwen3.7-Plus 69.8% Jun 2026 Lab model card
9 Seed2.0 Pro 68.5% Feb 2026 Lab model card
10 Qwen3.5 397B-A17B 67.5% Feb 2026 Lab model card
11 GPT-5 65.7% Aug 2025 Lab model card
12 Qwen 3.6 Plus 65.7% Apr 2026 Lab model card
13 Qwen 3.8 27B 65.5% Aug 2026 Lab model card
14 Qwen3.5-35B-A3B 64.8% Feb 2026 Lab model card
15 Muse Spark 64.7% Apr 2026 Lab model card
16 GPT-5 mini 62.9% Aug 2025 Lab model card
17 Qwen3.6 27B 62.5% Apr 2026 Lab model card
18 Qwen3.5-122B-A10B 62% Feb 2026 Lab model card
19 Qwen3.5-27B 60.5% Feb 2026 Lab model card
20 Seed1.8 58.8% Dec 2025 Lab model card
21 Nemotron 3.5 Super VL 57.5% Oct 2026 Lab model card
22 Qwen3.5-9B 55.5% Mar 2026 Lab model card
23 gemini-robotics-er-1.5-preview 54.8% Sep 2025 Lab model card
24 Qwen3.5-Omni-Plus 54.8% Mar 2026 Lab model card
25 Qwen3.5-4B 54% Mar 2026 Lab model card
26 Qwen3-VL-235B-A22B 52.5% Sep 2025 Lab model card
27 GPT-5 nano 50.1% Aug 2025 Lab model card
28 GLM-4.5V 50% Aug 2025 Lab model card
29 STEP3-VL-10B 48.9% Jan 2026 Lab model card
30 GLM-4.6V 47.8% Jan 2026 Lab model card
31 InternVL3.5-241B-A28B 46.8% Aug 2025 Lab model card
32 GLM-4.1V-9B-Thinking 45.8% Jul 2025 Lab model card
33 Cosmos3-Nano 44% May 2026 Lab model card
34 Qwen3.5-2B 43.8% Mar 2026 Lab model card
35 Cosmos3-Edge 42% May 2026 Lab model card
36 Qwen3.5-0.8B 34.5% Mar 2026 Lab model card
#ModelScoreSettingDateSource
1 Gemini 2.5 Pro Jun 2025 55.6% Jun 2025 Official leaderboard
2 InternVL3-38B 41.7% Apr 2025 Official leaderboard
3 o3 41.5% Apr 2025 Official leaderboard
4 GPT-5.2 41.5% Dec 2025 Official leaderboard
5 Gemini 2.0 Flash Feb 2025 40.2% Feb 2025 Official leaderboard
6 Qwen-VL-Max 39.3% Jan 2024 Official leaderboard
7 InternVL3-14B 38.8% Apr 2025 Official leaderboard
8 InternVL3-8B 37.1% Apr 2025 Official leaderboard
9 InternVL3.5-8B 36.3% Aug 2025 Official leaderboard
10 GPT-4o Mar 2025 34.8% Mar 2025 Official leaderboard
11 LLaVA-OneVision 7B 34.4% Aug 2024 Official leaderboard
12 Qwen2-VL-7B-Instruct 34.4% Aug 2024 Official leaderboard
13 Qwen2.5-VL-7B 34.4% Jan 2025 Official leaderboard
14 VILA1.5-13B 33.6% Apr 2024 Official leaderboard
15 Qwen2.5-VL-3B 32.6% Jan 2025 Official leaderboard
16 o4-mini 30.7% Apr 2025 Official leaderboard
17 Claude 3.7 Sonnet 14.3% Feb 2025 Official leaderboard
#ModelScoreSettingDateSource
1 Gemini Robotics ER 2 78.5% Jul 2026 Lab model card
2 Qwen3.8 Max (0902) 78.3% Sep 2026 Lab model card
3 Qwen 3.8 Max 77.8% Aug 2026 Lab model card
4 Qwen3.8 Flash 72.3% Aug 2026 Lab model card
5 Seed2.1 Pro 72% Jun 2026 Lab model card
6 Seed2.0 Lite (0428) 71.5% Apr 2026 Lab model card
7 Qwen3.8-Omni-Flash 71% Sep 2026 Lab model card
8 Qwen3.7-Plus 69.8% Jun 2026 Lab model card
9 Seed2.0 Pro 68.5% Feb 2026 Lab model card
10 Qwen3.5 397B-A17B 67.5% Feb 2026 Lab model card
11 GPT-5 65.7% Aug 2025 Lab model card
12 Qwen 3.6 Plus 65.7% Apr 2026 Lab model card
13 Qwen 3.8 27B 65.5% Aug 2026 Lab model card
14 Qwen3.5-35B-A3B 64.8% Feb 2026 Lab model card
15 Muse Spark 64.7% Apr 2026 Lab model card
16 GPT-5 mini 62.9% Aug 2025 Lab model card
17 Qwen3.6 27B 62.5% Apr 2026 Lab model card
18 Qwen3.5-122B-A10B 62% Feb 2026 Lab model card
19 Qwen3.5-27B 60.5% Feb 2026 Lab model card
20 Seed1.8 58.8% Dec 2025 Lab model card
21 Nemotron 3.5 Super VL 57.5% Oct 2026 Lab model card
22 Gemini 2.5 Pro Jun 2025 55.6% Jun 2025 Official leaderboard
23 Qwen3.5-9B 55.5% Mar 2026 Lab model card
24 gemini-robotics-er-1.5-preview 54.8% Sep 2025 Lab model card
25 Qwen3.5-Omni-Plus 54.8% Mar 2026 Lab model card
26 Qwen3.5-4B 54% Mar 2026 Lab model card
27 Qwen3-VL-235B-A22B 52.5% Sep 2025 Lab model card
28 GPT-5 nano 50.1% Aug 2025 Lab model card
29 GLM-4.5V 50% Aug 2025 Lab model card
30 STEP3-VL-10B 48.9% Jan 2026 Lab model card
31 GLM-4.6V 47.8% Jan 2026 Lab model card
32 InternVL3.5-241B-A28B 46.8% Aug 2025 Lab model card
33 GLM-4.1V-9B-Thinking 45.8% Jul 2025 Lab model card
34 Cosmos3-Nano 44% May 2026 Lab model card
35 Qwen3.5-2B 43.8% Mar 2026 Lab model card
36 Cosmos3-Edge 42% May 2026 Lab model card
37 InternVL3-38B 41.7% Apr 2025 Official leaderboard
38 o3 41.5% Apr 2025 Official leaderboard
39 GPT-5.2 41.5% Dec 2025 Official leaderboard
40 Gemini 2.0 Flash Feb 2025 40.2% Feb 2025 Official leaderboard
41 Qwen-VL-Max 39.3% Jan 2024 Official leaderboard
42 InternVL3-14B 38.8% Apr 2025 Official leaderboard
43 InternVL3-8B 37.1% Apr 2025 Official leaderboard
44 InternVL3.5-8B 36.3% Aug 2025 Official leaderboard
45 GPT-4o Mar 2025 34.8% Mar 2025 Official leaderboard
46 Qwen3.5-0.8B 34.5% Mar 2026 Lab model card
47 LLaVA-OneVision 7B 34.4% Aug 2024 Official leaderboard
48 Qwen2-VL-7B-Instruct 34.4% Aug 2024 Official leaderboard
49 Qwen2.5-VL-7B 34.4% Jan 2025 Official leaderboard
50 VILA1.5-13B 33.6% Apr 2024 Official leaderboard
51 Qwen2.5-VL-3B 32.6% Jan 2025 Official leaderboard
52 o4-mini 30.7% Apr 2025 Official leaderboard
53 Claude 3.7 Sonnet 14.3% Feb 2025 Official leaderboard

The benchmark's own leaderboard has 40 entries, 17 of them models we track. Switch the scores above to rank them together.

6 models have held number one in labs’ own numbers since its release in March 2025 3 models have held number one across all independent runs since its release in March 2025 8 models have held number one with claims included since its release in March 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Google DeepMind Shanghai AI Lab Z.ai OpenAI ByteDance Seed
Own leaderboard Not tested since Dec 2025
Gemini 2.5 Pro 55.6%
Lab claims 6 leaders
Gemini Robotics ER 2 78.5%

Sources: Official ERQA leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemini Robotics ER 2 Seed2.1 Pro GPT-5 Gemini 2.5 Pro InternVL3-38B Qwen-VL-Max 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemini Robotics ER 2 Seed2.1 Pro

Sources: Official ERQA leaderboard and labs’ model cards

3 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

OpenAI
Z.ai
Shanghai AI Lab
Qwen
ByteDance Seed
Meta
Google DeepMind
DateLaunch postScoreAs the post writes it
Sep 18, 2026 Qwen3.8-Omni-Flash 71 ERQA
Sep 2, 2026 Qwen3.8 Max (0902) 78.3 Embodied intelligence, ERQA
Aug 26, 2026 Qwen3.8 Flash 72.3 Embodied intelligence ERQA
Aug 14, 2026 Qwen 3.8 27B 65.5 Embodied intelligence ERQA
Aug 3, 2026 Qwen 3.8 Max 77.8 ERQA
Jul 30, 2026 Gemini Robotics ER 2 78.5 Question answering (ERQA)
Jun 23, 2026 Seed2.1 Pro 72 ERQA
Jun 1, 2026 Qwen3.7-Plus 69.8 ERQA
Apr 28, 2026 Seed2.0 Lite (0428) 71.5 ERQA
Apr 22, 2026 Qwen3.6 27B 62.5 ERQA
Apr 8, 2026 Muse Spark 64.7 ERQA (Embodied Reasoning)
Apr 2, 2026 Qwen 3.6 Plus 65.7 ERQA
Mar 30, 2026 Qwen3.5-Omni-Plus 54.8 ERQA
Mar 2, 2026 Qwen3.5-0.8B 34.5 ERQA 34.5/23.8 (with CI / without CI)
Mar 2, 2026 Qwen3.5-2B 43.8 ERQA 43.8/33.0 (with CI / without CI)
Mar 2, 2026 Qwen3.5-4B 54 ERQA
Mar 2, 2026 Qwen3.5-9B 55.5 ERQA
Feb 24, 2026 Qwen3.5-35B-A3B 64.8 ERQA
Feb 24, 2026 Qwen3.5-27B 60.5 ERQA
Feb 24, 2026 Qwen3.5-122B-A10B 62 ERQA
Feb 16, 2026 Qwen3.5 397B-A17B 67.5 ERQA
Dec 18, 2025 Seed1.8 58.8 ERQA
Sep 23, 2025 Qwen3-VL-235B-A22B 52.5 ERQA
Aug 25, 2025 InternVL3.5-241B-A28B 46.8 ERQA
Aug 11, 2025 GLM-4.5V 50 ERQA
Aug 7, 2025 GPT-5 nano 50.1 ERQA
Aug 7, 2025 GPT-5 mini 62.9 ERQA
Aug 7, 2025 GPT-5 65.7 ERQA Multimodal spatial reasoning

15 system cards report it, from 7 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 15, 2026 Nemotron 3.5 Super VL technical report 57.5 ERQA, SpatialClaw harness, with agentic vision training, Table 7 Capabilities
Sep 22, 2026 Qwen3.8-Omni-Flash technical report 71 ERQA (Table 3) Capabilities
Jun 23, 2026 Seed2.1 Pro model card 72 ERQA (Table 9) Capabilities
May 31, 2026 Cosmos3-Nano technical report 44 ERQA, Cosmos3-Nano 8B column, Table 10 Capabilities
May 31, 2026 Cosmos3-Edge technical report 42 ERQA, Cosmos3-Edge 2B column, Table 10 Capabilities
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 54.8 ERQA (Table 6) Capabilities
Mar 21, 2026 Seed1.8 model card 58.8 ERQA (Table 2) Capabilities
Feb 14, 2026 Seed2.0 Pro model card 68.5 ERQA (Table 8) Capabilities
Jan 14, 2026 STEP3-VL-10B technical report 48.9 ERQA, SeRe (Tables 1 and 3). PaCoRe 51.75 Capabilities
Jan 1, 2026 GLM-4.6V technical report 47.8 ERQA, GLM-4.6V thinking (Table 2) Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report 52.5 ERQA (Table 2) Capabilities
Sep 25, 2025 gemini-robotics-er-1.5-preview technical report 54.8 ERQA (Table 19, thinking on). Thinking off 47.0 Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 46.8 ERQA (Table 2, Table 11) Capabilities
Aug 13, 2025 GLM-4.5V technical report 50 ERQA, GLM-4.5V thinking (Table 2) Capabilities
Jul 1, 2025 GLM-4.1V-9B-Thinking technical report 45.8 ERQA, GLM-4.1V-9B-Thinking thinking (Table 2) Capabilities

More in Robotics

All 103 in Robotics
  • RoboDojo In use Generalist manipulation policies run 42 Isaac Sim tasks and 18 remote real-robot tasks. Scored by task score and success rate. GPT-6 Astra 29%
  • RoboLab In use Zero-shot DROID Franka policies in Isaac Lab on 120 tabletop tasks at three instruction levels, 10 episodes each. Success rate. FLUX 3 Action 42.9%
  • Progress classification (Google DeepMind) In use Models assign each frame of a robot video to one of five task-progress bands from 0 to 100 percent. Scored by accuracy. Gemini Robotics ER 2 57.4%
  • Human proximity (Google DeepMind robotics) In use Models judge from robot camera views whether a human is within 1 meter, a spatial awareness check for safe robot behavior. Scored by accuracy. Gemini Robotics ER 2 93%
  • Moment finding (Google DeepMind) In use Models find the moment in a robot video when an event happens. Scored by accuracy and mean absolute time distance in seconds. Gemini Robotics ER 2 91.3%
  • LIBERO Fading Simulated Franka tabletop tasks in four 10-task suites. Policies fine-tuned on 50 demos per task. Success rate over ~50 trials. MolmoAct 2 97.2%