| MLVU Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. | Long videos | Jun 2024 | 90.8% | In use |
| RefSpatial-Bench Real images where the model points to an object or free space defined by multi-step spatial relations. Scored by mask hit. | Embodied reasoning | Jun 2025 | 73.6% | Fading |
| MeasureBench Vision-language models read values from images of measuring instruments, such as gauges and rulers, in real and synthesized photos. Scored by matching the reading to the key. | General visual questions | Oct 2025 | 62.9% | In use |
| Open FlagEval VLM Leaderboard Models answer questions about images. The leaderboard compares their scores across visual understanding tasks. | General visual questions | Feb 2025 | 70.7 | Not in cards |
| Open Chinese LLM Leaderboard Models answer Chinese language tasks. The leaderboard compares model results across its evaluation suite. | Knowledge in many languages | May 2024 | 75.7 | Not in cards |
| TRUE (Text Recognition and Understanding) Models recognize text in images and answer questions about its content. | Documents, charts and text in images | | 81.6 | Not in cards |
| ROME Models solve reasoning problems selected to limit contamination. | Language reasoning | Sep 2025 | 69.4 | Not in cards |
| MeasureBench Real-world Models read values and units from real instrument images. The score is the published overall measurement accuracy. | General visual questions | Oct 2025 | 41.7% | Not in cards |
| MeasureBench Synthetic Models read values and units from generated instrument images. The score is the published overall measurement accuracy. | General visual questions | Oct 2025 | 39.2% | Not in cards |
| LiveOCRVQA Models read stylized text in images and answer questions about it. | Documents, charts and text in images | | 80.5 | Not in cards |
| EmbodiedVerse-Bench Models answer questions about spatial relationships and actions from visual observations. The leaderboard compares models across embodied reasoning tasks. | Embodied reasoning | | 50.7 | Not in cards |
| Video-SafetyBench Video-language models receive 2,264 synthesized videos paired with a harmful or a benign-looking query across 13 unsafe categories and 48 subcategories. An LLM judge scores harm with a confidence-weighted metric called RJScore. | Jailbreaks and prompt injection | May 2025 | 1.30 | Not in cards |
| HalluDial Models detect hallucinations in dialogue. | Hallucination rates | Jun 2024 | 80.2 | Not in cards |
| AgentSWE Coding agents build, edit, or optimize agents across 25 software-engineering commissions. Frozen deliveries run on held-out cases, where execution gates and task-specific model-judged rubrics score behavior. Optimization tasks measure improvement against each source benchmark objective. | Machine learning engineering | Oct 2026 | 19.1 | Not in cards |
| CMMU Models answer Chinese multimodal questions across seven school subjects. | General visual questions | Jan 2024 | 30.9 | Not in cards |
| ATOM-Bench Robot policies combine manipulation skills on single arm and dual arm platforms. | Real robots | Jun 2026 | 48.2% | Not in cards |
| ChildMandarin Models transcribe Mandarin speech from children ages three to five or verify speakers. | Speech recognition and translation | Sep 2024 | 11.5% | Not in cards |
| CORAL Models retrieve passages, answer questions, and cite supporting evidence across conversations. Retrieval metrics, response-overlap metrics, model judgments, and citation precision and recall score different parts of the task. | Questions that need search | Oct 2024 | 16.3 | Not in cards |
| EmotionTalk Models infer emotions and sentiment from Chinese conversations. | Audio understanding and voice | May 2025 | 50.9 | Not in cards |
| SeniorTalk Models transcribe conversations with older Mandarin speakers and identify who speaks. | Speech recognition and translation | Mar 2025 | 23.8% | Not in cards |
| Carena-Hard-Auto Models answer Chinese prompts. A judge compares pairs of answers, and the evaluation uses a Bayesian ranking model. | Chat and instructions in many languages | | | Not in cards |
| Chinese Linguistics and Cognition Challenge Models answer open questions about Chinese language and cognition. Human graders or the publisher’s judge model assess the answers. | Reasoning in many languages | | | Not in cards |
| HM3D-MFMON Agents navigate across floors and find a sequence of objects. | Simulated robots | Aug 2026 | | Not in cards |
| Massive Visualized IR Benchmark Models retrieve screenshots that match a search query. The benchmark compares performance across four retrieval task groups. | Documents, charts and text in images | Feb 2025 | | Not in cards |
| RefSpatial-Expand-Bench Models point to an object or free space described by multi-step spatial relations, in more indoor scenes plus new outdoor streets and parking lots. Scored by mask hit. | Embodied reasoning | Oct 2025 | | Not in cards |
| RefSpatial-Expand-Bench Location Models solve spatial location questions in the RefSpatial expansion set. The score is the published overall accuracy. | Embodied reasoning | Oct 2025 | | Not in cards |
| RefSpatial-Expand-Bench Placement Models solve spatial placement questions in the RefSpatial expansion set. The score is the published overall accuracy. | Embodied reasoning | Oct 2025 | | Not in cards |
| TraceSpatial-Bench Models plan an object-centric 3D trace, an ordered list of metric 3D waypoints, from language instructions in real cluttered indoor scenes with depth. | Embodied reasoning | Dec 2025 | | Not in cards |