| LVBench Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. | Long videos | Jun 2024 | 91.7% | In use |
| LongBench v2 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy. | Reasoning over long documents | Dec 2024 | 70.6% | In use |
| AlignBench Models answer 683 real Chinese user queries across eight categories. A GPT-4 judge rates each answer from 1 to 10 against a reference. | Overall chat quality | Nov 2023 | 8.94 | Dropped |
| MotionBench Models answer questions about fine-grained motion in video clips across six motion question types. Scoring is accuracy. | Video understanding | Jan 2025 | 75.2% | Fading |
| Vision2Web Agents build websites from 918 prototype images across 193 tasks, from static pages to full-stack apps. A GUI agent and a VLM judge score them. | Web and app front ends | Mar 2026 | 82.3% | In use |
| LongBench-Chat Models follow real user instructions over contexts of 10K to 100K tokens. GPT-4 scores each answer from 1 to 10 against references. | Reasoning over long documents | Jan 2024 | 8.76 | Dropped |
| ComplexFuncBench Models make function calls for 1,000 travel booking style samples that need several steps, user constraints, implicit parameter reasoning, long parameter values or 128K context. | Function calling | Jan 2025 | 61% | Dropped |
| NaturalCodeBench Models solve 402 Python and Java problems taken from real user queries to online coding services in 6 domains. Semi-automatically built tests check each solution. | Function-level problems | May 2024 | 52.5% | Dropped |
| CC-Bench-V2 Backend Agents change 85 real backend codebases in six languages. A task passes only when all of its unit tests pass. | Fixing real repositories | Feb 2026 | 25.8% | Fading |
| CC-Bench-V2 Large Repo Exploration Agents locate a hidden target file in a large repository from a user-level question. A question counts as solved when the agent reads the file. | Fixing real repositories | Feb 2026 | 72.2% | Fading |
| CC-Bench-V2 Frontend Agents build web projects in HTML, React, Vue, Svelte and Next.js. A GUI judge agent tests each check item in the running app. | Web and app front ends | Feb 2026 | 74.8% | Fading |
| Novel logical reasoning problems expert evaluation (Z.ai) Human experts score model answers to new multi-step logic problems with a fixed scoring standard for each question. | Abstract puzzles and logic | Aug 2025 | 62.0 | System cards only |
| CVTG-2K Image models render text in images across regions and languages. CVTG-2K reports normalized edit distance, CLIPScore and word accuracy. | Image generation and editing | Jan 2026 | 0.91 | Fading |
| LongText-Bench Image models render long text in images. The benchmark reports separate English and Chinese scores. | Image generation and editing | Jan 2026 | 0.97 | Fading |
| OneIG-Bench Image models generate images from prompts with English and Chinese evaluation splits. | Image generation and editing | Jan 2026 | 0.53 | Fading |
| TIIF-Bench Image models generate short and long images scored on overall image quality. | Image generation and editing | Jan 2026 | 81.0 | Fading |
| Z.ai Code Bench Zhipu in-house coding benchmark used for GLM-5.3. Reports pass rate and output tokens per task. Method unpublished. | Fixing real repositories | Aug 2026 | 39.5% | In use |
| CC-Bench-V2 Multi-Step Chained Tasks Agents complete chains of tasks mined from merged pull requests, where each step changes the codebase for the next. Tests check every step. | Fixing real repositories | Feb 2026 | 52.3% | System cards only |
| GLM-4 All Tools information seeking (Z.ai) Models use a web browser to find information for user questions. The ChatGLM report gives accuracy but does not describe the test set. | Questions that need search | Jan 2024 | 78.1% | System cards only |
| Challenging translation human evaluation (Z.ai) People blindly score translations of 100 real cases with slang, nicknames and memes on a 0 to 3 scale. | Translation | Aug 2025 | 1.71 | System cards only |
| IF-Badcase Models follow 450 multi-constraint instructions drawn from real user failure reports. A checklist checks each constraint. | Following precise instructions | Feb 2026 | 83.2% | System cards only |
| ImageMining Multimodal agents retrieve and reason over image information for image mining tasks. | Deep research | Apr 2026 | 30.7% | Fading |
| LVBench (with memory) Models answer multiple-choice questions about videos up to two hours long, aided by Qwen's external video memory system. Scored by accuracy. | Long videos | Aug 2026 | 85.6% | In use |
| MENT-SNS Models translate English and Chinese sentences with slang, wordplay and idioms. A GPT-4.1 judge compares each translation with a baseline. | Translation | Feb 2026 | 1013.0 | System cards only |
| MMSearch Multimodal agents search and answer questions that require visual and textual evidence. | Deep research | Apr 2026 | 72.9% | Fading |
| Slide generation human evaluation (Z.ai) People compare HTML slide decks from two models on content, layout and visual quality. Metric is overall win rate. | Taste and style | Feb 2026 | 67.5% | System cards only |
| ToolCall-Badcase Models choose a tool and its arguments for 200 cases drawn from real user failure reports. Each case has a ground-truth call. | Function calling | Feb 2026 | 95.8% | System cards only |
| Translation Human Scoring (Z.ai) People score model translations. The GLM-5 report does not describe the test set or the scale. | Translation | Feb 2026 | 3.12 | System cards only |
| ZClawBench Agents do end-to-end OpenClaw workflows such as environment setup, development, information retrieval, data analysis and content creation, drawn from real OpenClaw use. | General assistants | Mar 2026 | 57.6% | Fading |
| ZMultiDialBench Human annotators score model replies to 141 multilingual dialogue prompts on a 1 to 10 scale with category criteria. | Chat and instructions in many languages | Feb 2026 | 8.57 | System cards only |
| ZMultiTransBench Models translate 1,220 real Chinese source texts into seven languages. A GPT-4.1 judge compares each translation with a fixed baseline. | Translation | Feb 2026 | 1050.0 | System cards only |
| Real-scenario prompts human evaluation (Z.ai) One human evaluator scores replies to 660 curated real user prompts in English, Chinese and other languages on a 0 to 10 scale. | Overall chat quality | Aug 2025 | | System cards only |
| ComplexBench Models answer 1,150 hand-written Chinese instructions that compose several constraints by chaining, selection and nesting. LLM evaluators with rules check each constraint, and scores follow the dependency structure. | Following precise instructions | Jul 2024 | 80% | Not in cards |
| Vision2Web S2-2026 Agents build websites from 918 prototype images across 193 tasks, from static pages to full-stack apps. A GUI agent and a VLM judge score them. | Web and app front ends | Mar 2026 | 43.5% | Not in cards |
| Vision2Web S3-2026 Agents build websites from 918 prototype images across 193 tasks, from static pages to full-stack apps. A GUI agent and a VLM judge score them. | Web and app front ends | Jun 2026 | 74.6% | Not in cards |
| Vision2Web S1-2026 Agents build websites from 918 prototype images across 193 tasks, from static pages to full-stack apps. A GUI agent and a VLM judge score them. | Web and app front ends | Mar 2026 | 52.7% | Not in cards |
| VisualAgentBench Large multimodal models act as agents in embodied, GUI and visual design environments, including mobile, WebArena-based web and Minecraft tasks. | Web browsing | Aug 2024 | 36.2% | Not in cards |
| Terminal-Bench 2.0 Verified (Z.ai) Agents do the Terminal-Bench 2.0 tasks after Z.ai fixed environments for 89 tasks and instructions for 11. A further 2026-08-18 round fixed tests and instructions in 6 more tasks. | Terminal tasks | Feb 2026 | 61.8% | Not in cards |
| AndroidLab Mobile agents complete 138 tasks across nine apps on predefined Android virtual devices, in text-only and multimodal modes. Checks on the final device state score each task. | Phones | Oct 2024 | 31.2% | Not in cards |
| AISE-Bench Agents answer academic information-seeking questions through multi-step knowledge-graph APIs and source-grounded responses. Planning, execution and answer-grounding metrics score each trajectory. | Deep research | Jul 2026 | | Not in cards |
| DataSciBench Agents complete open-ended data-science tasks that require data cleaning, analysis, visualization and modeling. Component checks and overall task success score the work. | Data science and SQL | Apr 2026 | | Not in cards |
| Video-MonetBench Models rank two videos generated for the same prompt. The test compares each ranking with a human preference, including ties. | Video understanding | Dec 2024 | | Not in cards |