| EmbSpatialBench Models answer multiple-choice questions on six spatial relations between objects in egocentric embodied scenes. Scored by accuracy. | Spatial reasoning and counting | Jun 2024 | 85.4% | Fading |
| AutoLogi Models solve open-ended logic puzzles in English and Chinese. Generated verifier programs check each answer against the puzzle constraints. | Abstract puzzles and logic | Feb 2025 | 89.5% | Dropped |
| GAOKAO-Bench Models answer Chinese national college entrance exam questions from 2010 to 2022 across nine subjects, with objective and subjective questions. | Exam questions | May 2023 | 72.2% | Dropped |
| CL-bench Life Models solve 405 tasks that require learning from messy real-life context such as group chats. A model judge checks rubrics to give a solve rate. | Learning from the context | Apr 2026 | 22.2% | Fading |
| InstructTTSEval Text-to-speech models follow natural-language style instructions in English and Chinese across acoustic parameter, descriptive style and role-play tasks. Gemini judges each clip. | Speech recognition and translation | Jun 2025 | 83% | Fading |
| UniGenBench Text-to-image models draw images for 600 prompts across 5 themes and 20 subthemes, in English and Chinese and in short and long forms. A multimodal model judge checks each image on 10 primary and 27 secondary semantic dimensions, such as attributes, actions, relationships, layout, text and logical reasoning. | Image generation and editing | Aug 2025 | 95.8% | System cards only |
| CELLO Models follow real-world complex instructions with multiple tasks, noisy inputs and format or length constraints. Rule-based checks score each answer against the instruction features. | Following precise instructions | Sep 2023 | 76.4% | Dropped |
| AncientDoc Models transcribe historical Chinese pages, translate passages into modern Chinese, and answer questions about meaning, knowledge, and literary style. | Documents, charts and text in images | Aug 2025 | 5.77 | Not in cards |
| Code2Video-Compass Multimodal models write programs that render complete videos from text, an image or a video. The default Overall board ranks models by their mean rank across quality metrics, so a lower score is better. | Video generation | Oct 2026 | 1.07 | Not in cards |
| RNG-Bench Agents play Matching Pairs and navigate 3D mazes using remembered observations. Scores measure matched pairs and combine maze success, efficiency, and exploration. | Puzzles and chess | Jun 2026 | 49.9% | Not in cards |
| PunEval Models recognize, explain and generate English puns. The evaluation checks recognition against labels and compares explanations with human annotations. | Language reasoning | Apr 2024 | 1.00 | Not in cards |
| CroSS-MR Models read a scenario from a novel and select the reason for a character’s decision. The score is the percentage of correct choices. | Reading comprehension | Apr 2024 | 54.6% | Not in cards |
| TaskBench Models decompose requests, choose tools and dependencies, and predict tool parameters. Text metrics score decomposition, while graph and parameter F1 scores measure the remaining stages. | Function calling | Nov 2023 | 93.5 | Not in cards |
| UniREditBench Image editing models carry out 2,700 edits that need reasoning, in real-world scenes and in game worlds with human-defined rules, across 8 primary and 18 secondary dimensions. A multimodal judge scores each edit against both a text reference and a ground-truth image. | Image generation and editing | Nov 2025 | 57% | Not in cards |
| VidScribe Models write, copy, preserve, or edit visible text in videos. The evaluator tracks spelling, glyph shapes, placement, and stability across frames. | Video generation | Sep 2026 | 79.0 | Not in cards |
| LIBERO-Plus LIBERO policies run 10,030 perturbed tasks across seven dimensions: layout, camera, robot start state, language, light, background and sensor noise. Success rate. | Simulated robots | Oct 2025 | 69.6% | Not in cards |
| PRMBench Models identify errors and redundant steps in mathematical solutions. PRMScore combines positive and negative F1 scores across simplicity, soundness, and sensitivity tests. | Word problems | Jan 2025 | 68.8 | Not in cards |
| VLABench 100 language-conditioned manipulation task categories needing world knowledge, implicit intent and long-horizon reasoning. Evaluates VLAs and VLMs. | Simulated robots | Dec 2024 | 9.84% | Not in cards |
| LifeChoice Models select the decision a character makes in a novel. The authors score the percentage of correct choices. | Reading comprehension | Apr 2024 | 65.3% | Not in cards |
| CauGym Models infer causal effects from descriptions and probabilities. Test questions vary the wording, add irrelevant facts, or remove information needed to answer. | Language reasoning | Feb 2026 | | Not in cards |
| ClassEval Models generate whole Python classes for 100 hand-built tasks. Tests check each method and the class as a whole. | Function-level problems | Aug 2023 | | Not in cards |
| LiveMACE Agents trade in live financial markets, and the benchmark scores the process capabilities behind their decisions, not only returns. | Finance | Oct 2026 | | Not in cards |
| SocioHack RL-trained models act in 72 sandbox environments that copy institutional reward rules, such as credit card points or school grading. The test measures whether they find compliant loopholes that defeat each rule's purpose. | Deception and alignment | Jun 2026 | | Not in cards |
| TaxoBench Agents find research papers and organize them into a taxonomy. The evaluator compares their groupings and hierarchy with taxonomies from 72 published surveys. | Deep research | Jan 2026 | | Not in cards |
| TravelPlanner Agents plan multi-day trips from a user request, using tools to search nearly four million records of flights, hotels, restaurants and attractions. Each plan must meet the user's hard constraints, such as budget, and commonsense rules. The headline final pass rate counts a plan only when it meets every constraint. | General assistants | Feb 2024 | | Not in cards |