| Mol-Instructions Models answer molecule-oriented, protein-oriented and biomolecular text instructions, such as reaction prediction, molecule description and protein function questions. Scored per task with task metrics. | Genomics and life sciences | Jun 2023 | 54% | Fading |
| AIR-Bench (Audio Instruction Benchmark) Audio language models answer questions about speech, natural sounds and music. A foundation set has 19 tasks with about 19k single-choice questions, and a chat set has 2k open-ended questions scored by GPT-4. | Audio understanding and voice | Feb 2024 | 6.98 | System cards only |
| AIR-Bench Foundation Models answer foundational audio understanding questions. The score is the official average accuracy across nineteen tasks. | Audio understanding and voice | Feb 2024 | 67.3% | In use |
| SPBench Spatial perception and reasoning questions built from ScanNet scenes. A single-image set covers distance, size and direction, and a multi-view set adds counting across views. Answers are numbers or multiple choice. | Spatial reasoning and counting | Oct 2025 | 63.8% | System cards only |
| General-Bench Models perform multimodal understanding and generation tasks. General-Level averages task scores and measures performance above specialist baselines at progressively broader levels. | General visual questions | May 2025 | 8.41 | Not in cards |
| GSM8K-V Vision-language models solve GSM8K word problems that are redrawn as multi-image visual scenes, so the facts must be read from pictures. | Visual math and diagrams | Sep 2025 | 59.1% | Not in cards |
| S-Eval Models respond to Chinese and English harmful prompts and attack variants. A trained safety judge measures the safe-response fraction and attack success rate. | Refusing harmful requests | May 2024 | 88.8% | Not in cards |
| MemGUI-Bench Agents remember details while working across Android apps. Tasks include recording audio and creating calendar events. A model checks the actions and results across repeated attempts. | Phones | Feb 2026 | 73.4% | Not in cards |
| BlenderWorld Visual agents turn a text, image or video brief into an editable Blender scene with rendered media. Hidden binary rubrics and eight visual quality dimensions score each project. | CAD models | Oct 2026 | 79.9 | Not in cards |
| DeskCraft Desktop agents complete 538 Ubuntu tasks, including 152 tasks with later user clarifications or interruptions. Programmatic checks inspect final application state, saved files, and exported artifacts. | Desktop apps | Jun 2026 | 29.8% | Not in cards |
| MatCha Models answer multiple-choice questions about material structure, shape and properties using images from materials characterization instruments. | Lab and data work | Sep 2025 | 68.1% | Not in cards |
| RoughCut-Bench Models select and order video clips or mark speech to remove from a transcript. The evaluation compares cut plans with professional editors’ annotations. | Video understanding | Jul 2026 | 0.53 | Not in cards |
| TVRBench Agents move through simulated indoor scenes to reproduce a target image's viewpoint. Success requires stopping at exactly the target pose on the simulator's discrete grid. The evaluation split has 500 tasks across 80 scenes. | Simulated robots | May 2026 | 12% | Not in cards |
| ChJGBench Models draft the reasoning and orders in Chinese civil, criminal, and administrative judgments. The evaluator compares their work with 3,462 real judgments. | Law | Nov 2026 | 0.74 | Not in cards |
| KnowU-Bench Agents use Android apps such as Mattermost and Taodian. They infer preferences, ask for missing details, and decide when to act or stay silent. | Phones | Apr 2026 | 60.4% | Not in cards |
| ProOPF-B Models translate operational requirements into executable optimal power flow programs. They change grid parameters, objectives, and constraints. The evaluator checks implementations against expert references. | Scientific code | Feb 2026 | 37.2% | Not in cards |
| TaskBench Models decompose requests, choose tools and dependencies, and predict tool parameters. Text metrics score decomposition, while graph and parameter F1 scores measure the remaining stages. | Function calling | Nov 2023 | 93.5 | Not in cards |
| RSCC Models describe disaster damage from satellite images taken before and after an event. The authors reserve 988 image pairs from 19 events for testing. | Detection and captioning | Apr 2025 | 79.2 | Not in cards |
| MemTrapBench Models with memory are tested on cases where correct, relevant memories still distort their reasoning or beliefs and lower task performance. | Learning from the context | Aug 2026 | 85.2% | Not in cards |
| OceanGym Agents interpret underwater images and sonar, then navigate simulated search and inspection tasks. The simulator scores how closely they reach target locations. | Simulated robots | Sep 2025 | 14.6 | Not in cards |
| Aether-Bench Models retouch photographs automatically, match a requested style, or apply specified adjustments. The style test measures fidelity on 100 image pairs from unseen presets, including peak signal-to-noise ratio. | Image generation and editing | Apr 2026 | 79.7% | Not in cards |
| Agents4PLC v2.0 Models write Structured Text control programs against formal requirements. The expanded benchmark adds 70 medium tasks and three hard tasks to the original 23. | Electronics | Aug 2025 | | Not in cards |
| Mol-Instructions Chemical Entity Recognition Models identify chemical entities in biomedical text. The score is test-set F1. | Genomics and life sciences | Jun 2023 | | Not in cards |
| Mol-Instructions Chemical-disease Interaction Extraction Models extract chemical and disease interactions from text. The score is test-set F1. | Genomics and life sciences | Jun 2023 | | Not in cards |
| Mol-Instructions Chemical-protein Interaction Extraction Models extract chemical and protein interactions from text. The score is test-set F1. | Genomics and life sciences | Jun 2023 | | Not in cards |
| Mol-Instructions Description-guided Molecule Design Models generate molecules from descriptions. The score is exact-match accuracy on the molecule design test set. | Genomics and life sciences | Jun 2023 | | Not in cards |
| Mol-Instructions Forward Reaction Prediction Models predict reaction products from reactants. The score is exact-match accuracy on the forward reaction test set. | Genomics and life sciences | Jun 2023 | | Not in cards |
| Mol-Instructions Multi-choice Models answer biomedical multiple-choice questions. The score is test-set accuracy. | Genomics and life sciences | Jun 2023 | | Not in cards |
| Mol-Instructions Reagent Prediction Models predict reaction reagents. The score is exact-match accuracy on the reagent prediction test set. | Genomics and life sciences | Jun 2023 | | Not in cards |
| Mol-Instructions Retrosynthesis Models predict reactants for target products. The score is exact-match accuracy on the retrosynthesis test set. | Genomics and life sciences | Jun 2023 | | Not in cards |
| Mol-Instructions True or False Models answer biomedical true or false questions. The score is test-set accuracy. | Genomics and life sciences | Jun 2023 | | Not in cards |
| MSRVTT-QA Models answer short open-ended questions about web video clips from the MSR-VTT captioning dataset. The question and answer pairs were generated automatically from the clip descriptions. Scored by answer accuracy. | Video understanding | Oct 2017 | | Not in cards |
| WorldOlympiad Models generate long videos for gaming, robotics, and everyday scenes. The evaluator checks physical rules, reconstructed geometry, and instruction following across video segments. | Video world models | Jun 2026 | | Not in cards |