Image, video and 3D generation
Models make images, video or 3D models. We count 401 benchmarks here, 66 of them arenas. A benchmark belongs here when output includes images, video or 3D.
How fast it grows
116 were released since July, more than the 63 in the first half of 2026.
How much is still open
No model has passed 85% on 50 of the 59 benchmarks here that score in percent, 85% of them.
Every benchmark in this view
Newest first. Facets come from each benchmark’s own documents, and our methodology says how we tag them.
| Benchmark | Level | Released | Best score | Status |
|---|---|---|---|---|
| 3DArena-Modeling Agents create 3D models for 120 tasks covering precision and appearance. Tencent converts Elo ratings from blind comparisons into percentages. | University | Nov 2026 | Not in cards | |
| 3DSceneBench Agents reconstruct interactive MuJoCo scenes from images and task descriptions. Verifiers measure appearance, geometry, and physical functionality, then average the three scores. | Expert | Nov 2026 | 59.5 | Not in cards |
| AudioVideoBench Agents produce and edit audio and video in containers. The tasks include compositing, mixing, repairing media, reframing shots, and transcribing recordings. | University | Nov 2026 | Not in cards | |
| Construction AI Leaderboard Models answer construction questions, inspect drawings, edit building models, and generate validation rules. ONESTRUCTION averages the representative scores from five evaluations. | University | Nov 2026 | 73.7 | Not in cards |
| EVF-Bench Models generate product videos with visual effects. The evaluator checks motion, product appearance, text stability, and whether the effects fit the request. | University | Nov 2026 | | Not in cards |
| Frontier Design Benchmark Professional designers vote blind between two models’ designs for the same real client brief from Contra’s marketplace, such as a marketing site. Models are ranked by Elo from the votes. | Expert | Nov 2026 | 1,713 | Not in cards |
| Frontier3DBench Agents create editable Blender objects and scenes or executable Unreal Engine worlds across 145 tasks and nine task families. Model judgments and native project checks assess appearance, geometry, requirements, and interactions. The overall score weights object, scene, and world scores by 30%, 30%, and 40%. | Expert | Nov 2026 | 66.0 | Not in cards |
| HandWorldBench Models generate videos of human or robotic hands moving objects. The benchmark combines 17 measures of hand motion, object state, and video quality. | Everyday | Nov 2026 | | Not in cards |
| InfiHall-Bench Models generate images from 411 prompts that combine visual requirements. The evaluator checks details such as object parts, actions, and spatial relations separately. | Everyday | Nov 2026 | 67.1 | Not in cards |
| LongBanana Models synthesize images from many reference images and textual constraints. Atomic rubric checks score content fidelity, relationships, and global constraints across 90 tasks. | Everyday | Nov 2026 | | Not in cards |
| MedHEB Models retrieve related medical images, reports, and image regions. The authors evaluate how models rank relevant evidence across 107 retrieval tasks. | University | Nov 2026 | | Not in cards |
| MKA Agents complete professional tasks whose work is visual, 200 tasks across 10 fields and 83 occupations, each about three to eight hours of expert work. An agentic grader checks each deliverable against an expert rubric of about 70 yes or no checks, weighted 75% on factual accuracy and 25% on visual design. | Expert | Nov 2026 | | In use |
| PhyRubric-Bench Models generate videos of physical events. A judge checks caption-specific requirements against evidence at particular times in each clip. | School | Nov 2026 | | Not in cards |
| Pixel2Perspective Models locate and segment objects from aerial and ground views. Tests vary the camera angle, weather, and use of visible or infrared images. | Everyday | Nov 2026 | Not in cards | |
| Rapidata Multi-Step Image Editing (Identity Drift) Image editors apply five sequential changes to 199 photographs, and people compare the final faces for identity drift. | Expert | Nov 2026 | 1,866 | Not in cards |
| Real2Sim Coding agents get a robot recording from DROID (three camera videos, stereo calibration, joint and gripper commands and the task instruction) and build an interactive simulation of the scene. Each scene is scored on how closely it matches the recording, on replaying the recorded actions, and on whether a fixed robot policy completes the task in it. | Expert | Nov 2026 | 0.60 | Not in cards |
| SolidWorks Bench Agents model the custom parts of 73 real machines in SolidWorks and assemble them from raster drawing packages, through screenshots and the SolidWorks API. A grader checks the geometry against the reference and reads mates, motion and feature trees from the native files. | Expert | Nov 2026 | 0.57 | Not in cards |
| T2I-NSI Models generate images from paired positive and negative prompts. The index measures whether removing a requested concept also removes it from the image. | Everyday | Nov 2026 | 0.1 | Not in cards |
| VAHBench Agents turn short drama briefs into finished videos with specified characters, dialogue, and durations. The evaluator checks completeness, quality, reliability, and accuracy of production reports. | University | Nov 2026 | 51.8 | Not in cards |
| VibePrevisBench Coding agents write Blender Python that turns a script, photos, an opening frame or reference footage into a graybox 4D previs scene and render it. Rule checks score the exported scene against hidden facts, and a VLM judge checks that those facts are visible in the render. | Expert | Nov 2026 | 77.7 | Not in cards |
| WorkflowArena Agents write executable workflows that coordinate other agents on professional tasks. Hidden verifiers check the completed deliverables. | Expert | Nov 2026 | 37% | Not in cards |
| AA-3D Agents in an offline sandbox turn a written brief into an interactive three.js scene. A recruited human panel votes on blind pairs, and votes become Elo ratings. | Expert | Oct 2026 | 1,318 | In use |
| ABACUS Models add, remove, or change a specified number of objects in an image. The evaluator checks the count and preservation of the surrounding scene. | School | Oct 2026 | | Not in cards |
| AdSpark-Bench Models generate advertisement videos from product images and prompts. The benchmark checks whether products remain consistent and whether each shot conveys the requested selling point. | University | Oct 2026 | | Not in cards |
| BenchCAD 2.0 Models reconstruct parts, assemblies, and circuit boards from drawings and images. The Core set contains 100 problems with partial credit. | Expert | Oct 2026 | 59% | Not in cards |
| BenchCAD 2.1 Models reconstruct parts, assemblies, and circuit boards from drawings and images. The Core set contains 100 problems with partial credit. | Expert | Oct 2026 | 83.5% | In use |
| BenchCAD 2.1-beta Models reconstruct parts, assemblies, and circuit boards from drawings and images. The Core set contains 100 problems with partial credit. | Expert | Oct 2026 | Not in cards | |
| BlenderWorld Visual agents turn a text, image or video brief into an editable Blender scene with rendered media. Hidden binary rubrics and eight visual quality dimensions score each project. | Expert | Oct 2026 | 79.9 | Not in cards |
| CineRecon Agents use tools to rebuild reference videos from film, real footage and games as editable dynamic 3D scenes. The set has 50 videos and 360 evaluation clips. | Expert | Oct 2026 | 76.6 | Not in cards |
| Clipping Bench Agents run a simulated clip farm for about a month. They join pay-per-view campaigns, cut clips from licensed streams, post them and compete with rival clippers for budgets. | Expert | Oct 2026 | $9,514 | Not in cards |
| Design Bench (Moonshot) Models make graphic designs from direct prompts or few-shot reference images. An automated grader rates the results as Elo scores. | University | Oct 2026 | 957 | In use |
| Dual Exam Multimodal agents are tested under fixed environment responses, and world models under fixed actions. The agent track has 1,088 items and the world model track 400 cases. | Everyday | Oct 2026 | Not in cards | |
| Ego2Act Models generate first-person video showing how to complete a task from an initial image. A judge checks the steps and their physical plausibility. | Everyday | Oct 2026 | | Not in cards |
| FrontierCAD Agents build 39 machined parts from engineering drawings and place parts into 16 assemblies, in their own coding harnesses. A verifier that tolerates rotation and checks hole positions scores the geometry. | Expert | Oct 2026 | 0.67 | Not in cards |
| FrontierCAD Assemblies Agents place supplied CAD parts into 16 assemblies. The score is the published mean assembly geometry score. | Expert | Oct 2026 | 0.71 | Not in cards |
| FrontierCAD Parts Agents build 39 CAD parts from engineering drawings. The score is the published mean geometry score. | Expert | Oct 2026 | 0.63 | Not in cards |
| GeoFlip Models generate videos from paired scenes where a geometry change reverses the expected physical outcome. Model judges check the event and the scene structure. | School | Oct 2026 | | Not in cards |
| HealBench-Eval Models generate images intended to provide psychological comfort. The authors compare 14 image generators and prompt strategies on 1,095 cases. | Everyday | Oct 2026 | | Not in cards |
| Infographic factuality (Google) Image generation systems produce infographics and an automated rater scores factuality. | Everyday | Oct 2026 | | In use |
| Kaggle Sort Viz Arena Models use Manim to animate sorting algorithms and explain their time complexity. A submission must produce a valid video under the file size limit. Votes determine the arena rating. | University | Oct 2026 | 1,650 | Not in cards |
| LightGenBench Models predict which parts of a 3D surface emit light and what color they emit. The authors compare generated emission with reference assets. | University | Oct 2026 | Not in cards | |
| LMBuild Language model agents design 3D structures as parts, joints, materials and assembly steps. Scoring checks that the structures can be built and work as intended. | Expert | Oct 2026 | 66.5% | Not in cards |
| LogicVal Models generate images from prompts with formal constraints on objects, attributes, counts, and relationships. The evaluator checks which constraints the image satisfies. | School | Oct 2026 | | Not in cards |
| MarathonDocBench Models parse 100 documents spanning 103 to 300 pages. Evaluation scores content, cross-page continuations, figures, reading order, and heading hierarchy. | Expert | Oct 2026 | | In use |
| MoI-Bench Models generate and edit images for mobile users. The authors compare image quality across 64 subtasks and measure the cost of running models on phones. | Everyday | Oct 2026 | 0.97 | Not in cards |
| Physical-Engineering's Last Exam Agents produce billable engineering deliverables such as calculation packages, drawings and design reviews in 85 tasks across electrical, civil, structural and mechanical engineering. Automatic graders check constraints, safety limits and the design objective. | Expert | Oct 2026 | 0.49 | Not in cards |
| PROWBench Programmable world models must render video that follows the rules and interactions their program specifies. The test checks visual fidelity to fine-grained rules. | Everyday | Oct 2026 | | Not in cards |
| RGraph-Eval Models generate robot manipulation videos. The evaluator compares objects, changing relationships, and three-dimensional motion paths with a reference demonstration. | Everyday | Oct 2026 | | Not in cards |
| RISEBench++ Image editing models follow 1,000 bilingual instructions over six reasoning types, adding counterfactual and hybrid reasoning. It covers single-image, multi-image and multi-turn editing across 65 task types. | University | Oct 2026 | 56.6% | Not in cards |
| RWE-bench Agents do mechanical, electrical and aerospace design tasks, such as a crankshaft, dense PCB routing or turbine-vane cooling, with CAD and CAE tools. A deterministic verifier runs physics solvers and passes a design only if it meets every requirement. | Expert | Oct 2026 | 62% | Not in cards |
| ScribbleEdit Models edit images using a scribble as the only instruction. The evaluator checks whether the change matches the intended operation and location. | Everyday | Oct 2026 | | Not in cards |
| Short Form Bench Agents act as growth lead of a small brand for a simulated week. They post short videos, run the store and answer customers, sponsors and the founder. | Expert | Oct 2026 | $2,949 | Not in cards |
| SimReadyBench NV100 Agents reconstruct an articulated 3D asset from one image and nominal dimensions. The verifier grades formal, visual, and kinematic fidelity separately across 100 assets. | Expert | Oct 2026 | 100% | Not in cards |
| SimRepairBench Agents find and repair modeling errors in 3D OpenUSD assets for robot simulation, from 25 damaged objects in NVIDIA's SimReady collection, with partial credit from verifiers. | Expert | Oct 2026 | Not in cards | |
| SnorkelManufacturing Agents make engineering, production, quality, supplier and safety decisions in a simulated discrete-parts plant across 200 tasks, a third of them CAD tasks. | Expert | Oct 2026 | | Not in cards |
| SplatBench A model rebuilds an object in 3D from eight photos by placing up to 300 Gaussians itself through tool calls, with no optimizer. Half of the 30 tasks start from an empty scene, and half ask it to reassemble a scrambled fit. SplatScore compares renders from eight angles the model never sees, with an empty scene at 0 and gradient descent at 100. | University | Oct 2026 | 49.6 | Not in cards |
| StreamEval Streaming video generation models handle 300 scripted cases of 15 to 300 seconds. Requests arrive over time across avatar dialogue, performance, world generation and editing. | Everyday | Oct 2026 | Not in cards | |
| TextVBench Models render requested text in video or preserve text from a reference image. The score tracks spelling, placement, and consistency across frames. | Everyday | Oct 2026 | | Not in cards |
| UltraText Bench Image generators render dense bilingual text in many regions from 432 prompts. Scores check string accuracy, placement and legibility. | Everyday | Oct 2026 | 99.3 | Not in cards |
| VibeVideoBench Agents write code to render a finished video with sound, with no image or video generation model, in their native harness within one hour. Program checks, task rubrics and a judge panel score the result. | Expert | Oct 2026 | 4.11 | Not in cards |
| VTR-Bench Models generate videos containing visible text. The evaluation checks the rendered words and how well the text persists across frames. | Everyday | Oct 2026 | | Not in cards |
| WeGenEditBench Models edit images from instructions that can include text or multiple references. A shared checklist scores each requested change and the details that should stay intact. | Everyday | Oct 2026 | | Not in cards |
| WorldState-Bench Video models generate two clips from the same image and action. A hidden state changes which ending the rule requires. A vision model checks both clips. | Everyday | Oct 2026 | | Not in cards |
| AV-SafetyBench Models generate videos with sound from written prompts. Judges review each clip with sound, in silence, and as audio alone to identify unsafe outputs. | Everyday | Sep 2026 | 49.4% | Not in cards |
| BlenderBench (Refresh) Agents edit Blender scenes across 20 tasks. Each task grades the submitted scene using its own scoring configuration. | University | Sep 2026 | 0.69 | Not in cards |
| BVB Multimodal agents rebuild real-world videos as animated Blender scenes by writing code. The renders are compared with the source video under a shared cost limit. | Expert | Sep 2026 | 76.4 | In use |
| CAD Arena Agents rebuild licensed engineering drawings as editable CAD parts through the scripting APIs of NX, SolidWorks, Onshape, Fusion or Build123d. A verifier scores geometric match. | Expert | Sep 2026 | | Not in cards |
| Code2Video Benchmark 168 product-video briefs. Agent writes HTML motion-graphics code rendered by HyperFrames. A trained pairwise judge on five axes produces Elo. | Expert | Sep 2026 | 1,585 | Not in cards |
| Code4Scene Agents build and edit Unreal Engine scenes. The evaluator inspects the scene itself for correct edits, intact assets, and physical validity. | University | Sep 2026 | 0.62 | Not in cards |
| CutCraft Benchmark Models generate multi-shot videos under editing instructions. The evaluator checks transitions, shot structure, and the timing of audio relative to cuts. | University | Sep 2026 | | Not in cards |
| EngiWorld Agents work through the full industrial engineering design loop on 1,301 expert-curated tasks that need reasoning over geometric and physical constraints across software. | Expert | Sep 2026 | 44.7% | Not in cards |
| GenPuzzle Models generate images that solve visual puzzles such as mazes, tangrams, and Sudoku. The authors select an automated judge using human reference scores. | School | Sep 2026 | | Not in cards |
| GenScale Models generate or edit images containing objects with known relative sizes. A judge calibrated against human ratings scores the relative scale of object pairs. | Everyday | Sep 2026 | Not in cards | |
| HappyWorld Embodied The HappyWorld arena publishes a separate embodied vote pool and Elo ranking. | Everyday | Sep 2026 | | Not in cards |
| HappyWorld Spatial Reconstruction The HappyWorld arena publishes a separate spatial reconstruction vote pool and Elo ranking. | Everyday | Sep 2026 | 1,252 | Not in cards |
| HappyWorld-Bench World models are tested on the quality of the worlds they generate and on staying consistent and responsive while agents explore, interact and modify them. | Everyday | Sep 2026 | Not in cards | |
| MindWorldBench Models animate a scene to show how a person would act given their beliefs or intentions. Paired prompts change mental states while holding the scene fixed. | Everyday | Sep 2026 | Not in cards | |
| OmniVBench Video generators make videos that follow many kinds of reference inputs at once. Scoring checks that each reference factor is kept, separated and placed correctly. | Everyday | Sep 2026 | | Not in cards |
| ORAV Bench Models combine reference images, sounds, and videos into a requested audiovisual scene. A judge compares pairs of outputs against each reference's assigned role. | Everyday | Sep 2026 | | Not in cards |
| OSWorld-Science Computer-use agents do 146 scientific software tasks, such as molecular drawing, across several domains. Produced artifacts are checked to score each task. | Expert | Sep 2026 | 73.7% | Not in cards |
| Parametric CAD Bench Agents create and edit parametric FreeCAD models from text and engineering drawings. Programmatic checks award continuous task rewards for geometry and specification consistency. | Expert | Sep 2026 | 61.0 | Not in cards |
| Runway Model Router Benchmark Video generation systems produce clips for 250 image-to-video prompts across ten production categories and are compared by usable rate and cost. | Expert | Sep 2026 | Not in cards | |
| SolveEdit Models edit an image to solve a visual problem. The evaluator checks whether the goal holds and whether unrelated scene details survive the edit. | School | Sep 2026 | 57% | Not in cards |
| StateBench (video continuation) Models continue a video while tracking what happened in earlier segments. A judge checks whether visible objects, hidden processes, and later state changes remain consistent. | Everyday | Sep 2026 | | Not in cards |
| T2LSC-Bench Models render text at a specified location in an image. Judges check the lettering and whether its meaning alters other objects or the background. | Everyday | Sep 2026 | | Not in cards |
| TimelineBench Agents turn raw footage and a client brief into a finished video. The edit must pass delivery checks, meet the brief, and clear a quality threshold. | University | Sep 2026 | 32.1% | Not in cards |
| TrafficImag Models edit roadside traffic videos to show a different sequence of events. The evaluator checks routes, interactions, and preservation of the surrounding scene. | Everyday | Sep 2026 | 90.0 | Not in cards |
| TriWorldBench World models generate synchronized robot-view videos from head and wrist cameras. Nineteen signals measure cross-view agreement, task alignment, physical coherence, motion, temporal stability, and visual quality. | Everyday | Sep 2026 | Not in cards | |
| UFO-Bench Models generate customized images from text and visual references. The evaluator breaks each request into individual requirements and checks how they interact. | Everyday | Sep 2026 | | Not in cards |
| VidScribe Models write, copy, preserve, or edit visible text in videos. The evaluator tracks spelling, glyph shapes, placement, and stability across frames. | Everyday | Sep 2026 | | Not in cards |
| VIF-Bench Models generate images from multiple references and visual instructions. Judges check whether the result follows controls such as pose or lighting without copying unwanted features. | Everyday | Sep 2026 | 7.60 | Not in cards |
| ADVANCE audio and image retrieval (MTEB) Models match field recordings and aerial images from 5,075 locations. The correct recording and image come from the same place. | Everyday | Aug 2026 | Not in cards | |
| AEC-Geometric-Bench Models locate doors, windows, and fixtures on construction drawings and trace walls and enclosed spaces. The scorer matches objects and compares wall and area masks. | University | Aug 2026 | | Not in cards |
| Brennan Center election image generation audit Image generators receive requests for misleading election scenes. Researchers count how often each system produces an image, including images that omit requested details. | Everyday | Aug 2026 | 100% | Not in cards |
| ConceptEdit-Bench Models edit images according to 1,000 instructions across six categories. A vision-language judge scores instruction following, visual quality, and consistency in unedited regions from 0 to 10. The per-case score averages those three dimensions. | Everyday | Aug 2026 | | Not in cards |
| Cyc3D Models generate 3D assets from images. The evaluator checks mesh quality and measures how much shape and identity drift across repeated rendering and reconstruction. | University | Aug 2026 | Not in cards | |
| H2R-Bench Models turn first-person demonstrations into videos of a specified robot performing the task. The benchmark checks contacts, object responses, and completion of the goal. | Everyday | Aug 2026 | | Not in cards |
| HarnessEval-W Video and world models generate rollouts from image and action prompts. An evaluator agent plans checks, spawns sub-agents, gathers spatiotemporal evidence and scores observation, transition and persistence with written reasoning. | Expert | Aug 2026 | | Not in cards |
| OmniCAD Models choose parts from a component library and predict their assembly positions, orientations, and connections. The evaluator measures pose accuracy and how well connections match. | University | Aug 2026 | | Not in cards |
| OmniPhys Models solve Chinese physics problems from middle school through university and draw diagrams. The evaluator grades answers, reasoning, and generated figures. | University | Aug 2026 | | Not in cards |
| PAWBench Video generators are run many times on the same start and action, and the outcomes become a distribution over physical behaviors. The test checks whether it matches reference probabilities and covers every valid behavior. | Everyday | Aug 2026 | Not in cards | |
| PersonaShot Models generate multi-shot videos about a person. The evaluator checks character continuity, emotional changes, and camera choices within and between shots. | Everyday | Aug 2026 | 0.67 | Not in cards |
| PlayWorld An agent player pursues objectives inside video world models. Gemini 3.1 Pro applies weighted rubrics for geometry, interaction, and state evolution across the resulting videos. | Everyday | Aug 2026 | Not in cards | |
| QwenBlenderBench Models complete Blender 3D modeling tasks in Qwen's internal visual agent and coding test. Qwen has not described its scoring. | University | Aug 2026 | | In use |
| SemComp-Bench Video generation models must complete an intended task outcome from a reference image while keeping the high-level semantics of the reference. | Everyday | Aug 2026 | | Not in cards |
| VGI-Bench Video generation models solve 27 visual reasoning tasks with 810 instances through generated frames. Scoring checks that the process is valid, not only the final frame. | Everyday | Aug 2026 | | Not in cards |
| VWE-Bench Multimodal agents build interactive 3D worlds from a user request. They infer intent, plan the scene layout, call 3D tools for asset retrieval, editing and rendering, and revise from rendered feedback. The benchmark has 2,616 3D assets, 323 human-annotated seed worlds and 6,828 queries. Verified queries have ground truth, and the rest are scored with rubrics. | Expert | Aug 2026 | 57.3% | Not in cards |
| WildArtifactBench Agents produce real deliverables in any format, from audio analysis and medical video reads to 3D meshes and kitchen layouts. Human and agent judges compare them with a baseline agent's work to give win rates and Elo. | Expert | Aug 2026 | Not in cards | |
| WorldExam Controllable video generators are tested as world models. Levels run from visual appearance to whether the depicted world reacts plausibly beyond what the prompt states. | Everyday | Aug 2026 | | Not in cards |
| AutoCAD Bench Computer-use agents turn a dimensioned reference drawing into a correct, editable DWG in AutoCAD through the screen alone, with mouse and keyboard. 50 tasks span 2D drafting and 3D modeling at four difficulty levels. | Expert | Jul 2026 | 46% | Not in cards |
| Blind-Spots-Bench Models answer 235 text and visual problems chosen to expose failures on tasks that people find simple. A Gemini 3 Flash grader compares each response with a structured reference solution and gives binary credit. The grader can execute code to check exact constraints. | Everyday | Jul 2026 | Not in cards | |
| EmbodiedCLUE-World 2.0 World models predict outcomes in embodied settings under the second evaluation protocol. | Everyday | Jul 2026 | | Not in cards |
| KeyFrame-Compass Video generators must reproduce a given sequence of keyframes while keeping overall video quality. The set has 386 samples over three domains and four keyframe densities. | Everyday | Jul 2026 | | Not in cards |
| MPIE-Bench Image editing models place several named people into shared contact actions such as an embrace or a carry. Scoring targets fused limbs, extra extremities and bodies passing through each other. | Everyday | Jul 2026 | | Not in cards |
| MultiRef-Compass Models generate audio-video content from several visual references and text instructions across 350 samples. Fourteen metrics assess quality, reference fidelity, audio-visual consistency, and instruction following. The evaluator combines automatic measurements with multimodal model judgments. | Everyday | Jul 2026 | Not in cards | |
| SoundingEarth audio and image retrieval (MTEB) Models match a field recording to an aerial image of the same location, or retrieve the recording from the image. Each direction has 1,000 queries and 2,048 candidates. | Everyday | Jul 2026 | Not in cards | |
| Arena-T2I Hard Models generate images from detailed user prompts. A judge checks whether the images satisfy individual requirements and their dependencies. | University | Jun 2026 | | Not in cards |
| BIM-Edit Agents create, change, or delete elements in IFC building models. The evaluator checks the resulting geometry, attributes, and relationships. | University | Jun 2026 | | Not in cards |
| BuildArena 2.0 Construction Challenge Agents build and test machines in Besiege for seasonal engineering challenges. The organizers score each entry using simulation results and the rules for its track. | Competition | Jun 2026 | Not in cards | |
| CoVEBench Video editing models follow compositional instructions with several coupled edits to subjects, actions and camera, while unrelated content must stay unchanged. | Everyday | Jun 2026 | 55.2 | Not in cards |
| Image2FloorPlan Models study several real photos of a home and draw its floor plan. Scored by an average quality score. | Everyday | Jun 2026 | | In use |
| Physics-IQ Verified Video models continue 198 real filmed physics experiments from their first frames. Generated videos are compared with the real outcome, with every sample and metric weighted equally. | Everyday | Jun 2026 | 10.9 | Not in cards |
| TailOR Models generate images and videos of regular, unconventional, and impossible tool interactions. Annotators score instruction adherence, interaction accuracy, physical realism, and perceptual quality. | Everyday | Jun 2026 | Not in cards | |
| WorldOlympiad Models generate long videos for gaming, robotics, and everyday scenes. The evaluator checks physical rules, reconstructed geometry, and instruction following across video segments. | Everyday | Jun 2026 | Not in cards | |
| 3DCodeArena Models generate 3D assets through code. Human preference votes determine the published Bradley-Terry Elo rating. | Expert | May 2026 | 1,230 | Not in cards |
| 3DCodeBench Agentic procedural 3D modeling via code: author engine-ready 3D assets through software APIs. Graded for executability and shape fidelity. | May 2026 | | In use | |
| AgenticVBench Agents complete 100 video post-production tasks in 4 task families built from real workflows by 20 industry experts. Programmatic verifiers and expert rubrics score the results. | Expert | May 2026 | 38.4% | In use |
| Blueprint-Bench 2 Agents turn about 20 interior photos of each of 50 apartments into 2D floor plans. Scores compare room connectivity with ground truth, normalized against random. | Everyday | May 2026 | | In use |
| CADGenBench Agents construct CAD models from design prompts. The generation split is graded for executability and geometric correctness. | University | May 2026 | 45.1% | System cards only |
| EduVideoBench Models generate classroom videos intended to teach facts, skills, or attitudes. Experts assess instructional accuracy, presentation, and educational safety. | University | May 2026 | | Not in cards |
| EmbodiedCLUE-World World models predict outcomes in embodied settings. | Everyday | May 2026 | | Not in cards |
| EvalVerse Video models generate cinematic clips from prompts and reference material derived from professional videos. Film experts compare outputs across visual design, acting, cinematography, aesthetics, emotion, shot sequences, and sound. A calibrated vision-language evaluator also estimates these judgments. | Expert | May 2026 | 0.73 | Not in cards |
| LongAV-Compass Models generate minute-long videos with sound from text, an image or a video. Scoring covers identity consistency, narrative coherence and audio-visual alignment over time. | Everyday | May 2026 | | Not in cards |
| MDMR-Bench Models generate one image from several references. A model judge checks whether the output preserves requested elements, follows instructions, and combines them correctly. | Everyday | May 2026 | 88.8% | Not in cards |
| MPDocBench-Parse Models parse complete multipage documents while preserving content and cross-page structure. Evaluation scores text, formulas, tables, figure extraction, reading order, and heading hierarchy. | Expert | May 2026 | | In use |
| MultiMedia-TerminalBench Agents inspect and edit media in persistent terminal workspaces. The 105 tasks require verifiable output files across five areas of professional multimedia work. | Expert | May 2026 | 16.2% | Not in cards |
| PaintBench Image editing models do precise single-answer edits in 20 operations such as geometric transforms, color changes and symbolic edits. Procedural generation gives deterministic checks. | Everyday | May 2026 | | Not in cards |
| Parametric CAD Bench V1 Agents write FreeCAD scripts that build editable parametric parts from plain-English descriptions and dimension tables. Each of 100 tasks scores the harmonic mean of geometry similarity and spec consistency. | Expert | May 2026 | In use | |
| PRISM-Bench (audio and video generation) Models generate speech, music, or environmental sounds with video. A judge compares outputs with reference clips across 35 criteria, including synchronization and prompt following. | Everyday | May 2026 | | Not in cards |
| Qwen-Image-Bench Text-to-image models are tested on professional creative work, from faithful real-world reconstruction to creative expression. A released judge model scores the images. | Everyday | May 2026 | 64.7 | Not in cards |
| Tricky Edits of Collected and Curated Images Models edit a source image to follow a written instruction. People and a model judge instruction following, preservation of unaffected content, and visual quality. | Everyday | May 2026 | | Not in cards |
| WBench Interactive video world models run 289 multi-turn cases with navigation, subject actions, event edits and perspective switches. 22 metrics cover quality, adherence, consistency and physics. | Everyday | May 2026 | | Not in cards |
| WorldArena 2.0 Extends WorldArena to visuotactile prediction, world models as RL environments, and simulated plus real multi-embodiment robot platforms. | May 2026 | | Not in cards | |
| Aether-Bench Models retouch photographs automatically, match a requested style, or apply specified adjustments. The style test measures fidelity on 100 image pairs from unseen presets, including peak signal-to-noise ratio. | University | Apr 2026 | | Not in cards |
| AVGen-Bench Models generate videos with audio from prompts across eleven real-world categories. Specialist models and multimodal judges measure perceptual quality, synchronization, and adherence to detailed visual, speech, music, and physical requirements. | University | Apr 2026 | | Not in cards |
| DataSciBench Agents complete open-ended data-science tasks that require data cleaning, analysis, visualization and modeling. Component checks and overall task success score the work. | Expert | Apr 2026 | Not in cards | |
| GraphicDesignBench Models analyze and produce graphic designs in 50 tasks spanning layout, typography, infographics, design semantics, and animation. The suite uses professional layered design templates. | Expert | Apr 2026 | 0.94 | Not in cards |
| neuralCAD-Edit Agents edit existing CAD models from text or recorded instructions. CAD experts rate whether they would accept each edit. Software also compares the resulting geometry. | University | Apr 2026 | 0.25 | Not in cards |
| WISE Verified Models generate images from the revised WISE prompts. The verified evaluation changes about 200 prompts and the judge, so original scores remain separate. | School | Apr 2026 | 0.82 | Not in cards |
| BIBIMBAP Agents read and edit IFC building files. Executable tests compare their changes and answers with reference results. | University | Mar 2026 | 50.2% | Not in cards |
| BioVITA retrieval benchmark Models match animal images, audio recordings, and descriptions. The benchmark checks whether the retrieved items match the reference species, genus, or family. | University | Mar 2026 | Not in cards | |
| DesignJanus People rate generated objects from six viewpoints and mark inconsistencies in shape, materials, and identity. The harness compares human ratings with automatic measures. | Everyday | Mar 2026 | Not in cards | |
| Teaching Monster Challenge Agents generate instructional videos for a specified topic and learner. A model screens submissions, viewers compare videos in pairs, and an expert panel sets the final ranking. | University | Mar 2026 | Not in cards | |
| TextEdit Image generation models edit text inside 2,148 real and synthetic images across 18 subcategories. OCR, image-fidelity metrics and a multimodal LLM judge score target accuracy, text preservation and scene integrity. | Everyday | Mar 2026 | | Not in cards |
| TextEdit Real Models edit text in real images in the full TextEdit test set. The score is the published judged average. | Everyday | Mar 2026 | 0.90 | Not in cards |
| TextEdit Virtual Models edit text in virtual images in the full TextEdit test set. The score is the published judged average. | Everyday | Mar 2026 | 0.90 | Not in cards |
| AIDABench Agents analyze documents and spreadsheets to answer questions, draw charts, and generate files across 603 tasks. They can execute Python in a sandbox. Separate model evaluators check answer correctness, chart correctness and readability, and spreadsheet structure and content. | Expert | Feb 2026 | 59.4% | Not in cards |
| GEBench Image generation models create interactive GUI states from prompts. A judge scores goal, logic, consistency, interface and quality across five task types. | Everyday | Feb 2026 | 63.2 | Not in cards |
| Human SxS image editing Elo (Google) Google compares edited images side by side and converts human preferences into Elo ratings. | Everyday | Feb 2026 | | Fading |
| Human SxS image generation Elo (Google) Google compares generated images side by side and converts human preferences into Elo ratings. | Everyday | Feb 2026 | | In use |
| HY3D-Bench Models generate textured 3D meshes from text and images, with semantic and geometric quality evaluated across categories. | Expert | Feb 2026 | Not in cards | |
| MarsRetrieval Models match Mars images to captions, retrieve examples of named landforms, and locate features in a global mosaic of the planet. | University | Feb 2026 | | Not in cards |
| PhyWorldBench Video generators create scenes that follow physical laws across object motion, energy conservation, rigid interactions and human or animal motion. | Expert | Feb 2026 | Not in cards | |
| SA-1B Hard human pose evaluation Models locate body and foot joints in photographs with unusual viewpoints, poses, and occlusions. Meta measures the fraction of correctly placed joints. | School | Feb 2026 | Not in cards | |
| SAM 3D Body synthetic pose evaluation Models estimate human pose and shape from synthetic images of individual people. Meta compares predicted joints and mesh vertices with the rendering parameters. | School | Feb 2026 | Not in cards | |
| VBVR-Bench Video generation models solve visual reasoning tasks by generating a video from a starting image and instruction. Rule-based scorers check the result instead of a judge model. Tasks cover abstraction, knowledge, perception, spatial reasoning and transformation, with in-domain and out-of-domain splits. | Everyday | Feb 2026 | | Not in cards |
| VBVR-Pro The current official results table evaluates the Pro release. Original VBVR results must remain separate. | Everyday | Feb 2026 | | Not in cards |
| WorldArena Embodied world models generate RoboTwin 2.0 task videos from first frame plus text or actions. 15 video metrics (EWMScore) plus utility tracks. | Feb 2026 | | Not in cards | |
| CVTG-2K Image models render text in images across regions and languages. CVTG-2K reports normalized edit distance, CLIPScore and word accuracy. | Everyday | Jan 2026 | 0.91 | Fading |
| LMArena Image-to-Video Arena Video generation models animate the same input image with the same prompt, anonymously and side by side. Votes give Elo-style ratings. | Everyday | Jan 2026 | | Not in cards |
| LMArena Text-to-Video Arena Video generation models make clips from the same text prompt, anonymously and side by side. Votes from logged-in users give Elo-style ratings. | Everyday | Jan 2026 | | Not in cards |
| LongText-Bench Image models render long text in images. The benchmark reports separate English and Chinese scores. | Everyday | Jan 2026 | 0.97 | Fading |
| MineBench Models turn a prompt into a Minecraft-style voxel build as raw 3D coordinates or tool calls. Users compare builds blind, and votes become rankings. | Everyday | Jan 2026 | 2,169 | Not in cards |
| OneIG-Bench Image models generate images from prompts with English and Chinese evaluation splits. | Everyday | Jan 2026 | 0.53 | Fading |
| RBench 650 image+text prompts for robot video generation (tasks and embodiments). MLLM judge scores task completion, physical plausibility, stability. | Jan 2026 | | System cards only | |
| SpatialGenEval Text-to-image models generate scenes from 1,230 prompts, and vision-language judges answer 12,300 spatial questions about the resulting images. | University | Jan 2026 | Not in cards | |
| TIIF-Bench Image models generate short and long images scored on overall image quality. | Everyday | Jan 2026 | 81.0 | Fading |
| VDEBench Image editing models make local text and table edits to 3,506 Chinese and English document images. Scores check the edit and that the rest of the page is preserved. | Everyday | Jan 2026 | 0.77 | Not in cards |
| Video Browser Showdown 2026 marine video search Participants use interactive retrieval systems to find five underwater clips after seeing each target. Each search has a five-minute limit. | Everyday | Jan 2026 | Not in cards | |
| GenEval 2 Models generate images from compositional prompts. A visual question answering model checks whether each requested object, attribute, count, and relation appears. | Everyday | Dec 2025 | Not in cards | |
| PAI-Bench Physical-AI video generation from 1,044 prompts in six domains, VLM-QA plus quality metrics. Companion control and understanding tracks. | Dec 2025 | | System cards only | |
| TraceSpatial-Bench Models plan an object-centric 3D trace, an ordered list of metric 3D waypoints, from language instructions in real cluttered indoor scenes with depth. | Everyday | Dec 2025 | Not in cards | |
| VIST Image models generate a new image in a reference brand style. Human evaluators compare pairs of outputs. | University | Dec 2025 | Not in cards | |
| ArtEdit-Bench-Lr Models retouch photos from English or Chinese instructions. GPT-4o scores instruction following and visual quality. Pixel errors measure fidelity to reference edits. | University | Nov 2025 | | Not in cards |
| AssurAI Models respond to Korean prompts across 35 risk categories. Model judges rate the safety of text, image, audio, and video outputs from 1 to 5. | Everyday | Nov 2025 | Not in cards | |
| GEO-Bench-2 Models interpret Earth imagery across 19 tasks, including field boundaries, forest biomass and tree species. The authors normalize task scores to compare model capabilities. | University | Nov 2025 | Not in cards | |
| SAM 3D Artist Objects Models reconstruct an object's shape and placement from a single photograph. The benchmark compares predictions with meshes created by 3D artists. | School | Nov 2025 | Not in cards | |
| SAM 3D Objects human preference evaluation Models reconstruct objects from images of real scenes. People compare the shape and texture of competing reconstructions, including objects seen from an egocentric viewpoint. | Everyday | Nov 2025 | Not in cards | |
| Segment Anything with Concepts Models find every instance of a concept named in a prompt. The benchmark compares image masks and video tracks with human annotations. | Everyday | Nov 2025 | Not in cards | |
| UniREditBench Image editing models carry out 2,700 edits that need reasoning, in real-world scenes and in game worlds with human-defined rules, across 8 primary and 18 secondary dimensions. A multimodal judge scores each edit against both a text reference and a ground-truth image. | Everyday | Nov 2025 | | Not in cards |
| LeWitt-Bench Image models interpret instructions for Sol LeWitt wall drawings. People compare the resulting pictures. | Everyday | Oct 2025 | Not in cards | |
| MathCanvas-Bench Models solve 3,000 math problems that call for interleaved visual and textual solutions, such as drawing auxiliary lines in geometry. | School | Oct 2025 | | Fading |
| MBench Models generate human motion from 450 prompts. The evaluation measures motion quality and uses a vision-language model to judge prompt matching and generalization. | Everyday | Oct 2025 | Not in cards | |
| PICABench Models edit images while preserving physical effects in optics, mechanics, and state transitions. A vision-language judge answers region-specific questions to measure physical correctness. | Everyday | Oct 2025 | | Not in cards |
| SuperCLUE-Edit Models edit an input image according to a Chinese instruction. The evaluation measures whether the requested change preserves relevant details. | Everyday | Oct 2025 | 82.5 | Not in cards |
| UWBench Models answer questions about underwater scenes, write captions, and locate objects from descriptions. Questions cover visibility, materials, and spatial relationships. | Everyday | Oct 2025 | Not in cards | |
| Arti4D Models watch first-person recordings of people opening furniture. They predict whether parts rotate or slide and estimate the axis of motion. | School | Sep 2025 | Not in cards | |
| Blueprint-Bench Models turn about 20 interior photos of each of 50 apartments into a 2D floor plan. Scored by similarity of room connectivity and size ranking to the true plan. | Everyday | Sep 2025 | 0.43 | Not in cards |
| MapAnything dense multi-view reconstruction Models reconstruct a scene from multiple images. The benchmark compares point geometry, metric scale, depth, and camera poses with reference measurements. | School | Sep 2025 | Not in cards | |
| MapAnything single-view calibration Models infer camera intrinsics from a single image. The benchmark measures angular error between predicted camera rays and reference rays. | School | Sep 2025 | Not in cards | |
| PRISM-Bench (image generation) Models generate images from English or Chinese prompts. Vision-language judges score visual appeal and prompt alignment across seven tracks, including composition, lettering, and style. | Everyday | Sep 2025 | 86.3 | Not in cards |
| VoxelBench Language models turn text or image prompts into voxel structures. Voters compare anonymous builds from the same prompt, and Glicko-2 ratings rank the models. | Everyday | Sep 2025 | 2,607 | Not in cards |
| VoxelBench Image Models turn image prompts into voxel structures. Human comparisons determine Glicko-2 ratings in a separate image-prompt vote pool. | Everyday | Sep 2025 | 2,215 | Not in cards |
| 7Bench Layout-guided image models generate 224 text-and-layout samples across seven scenarios, scored for semantic and spatial alignment. | University | Aug 2025 | Not in cards | |
| MOSEv2 Models track and segment objects in videos with complex scenes. Objects may disappear, reappear, or blend into a crowd. | Everyday | Aug 2025 | Not in cards | |
| SatLidar canopy height evaluation Models predict canopy height from satellite images. Meta compares height estimates with airborne lidar measurements and reports errors on held-out regions. | University | Aug 2025 | Not in cards | |
| UniGenBench Text-to-image models draw images for 600 prompts across 5 themes and 20 subthemes, in English and Chinese and in short and long forms. A multimodal model judge checks each image on 10 primary and 27 secondary semantic dimensions, such as attributes, actions, relationships, layout, text and logical reasoning. | Everyday | Aug 2025 | 95.8% | System cards only |
| BM-Bench Image editing models perform instruction-guided non-rigid image manipulations on the ByteMorph benchmark dataset. | Expert | Jun 2025 | Not in cards | |
| Kontext Bench Image editing models get 1,026 image and instruction pairs across local edits, global edits, text editing, style reference and character consistency. | Everyday | Jun 2025 | Not in cards | |
| LMArena Image Edit Arena Image editing models apply the same user edit request to the same image, anonymously and side by side. Community votes give Elo-style ratings. | Everyday | Jun 2025 | 1,519 | Not in cards |
| MMArt-Bench Models retouch 200 photos from editing instructions. Gemini scores visual quality and instruction following, and image comparisons measure how closely edits match reference photos. | University | Jun 2025 | | Not in cards |
| PST50 Models transfer a reference look to a photograph. The paired test compares 50 outputs with reference grades that professionals refined in DaVinci Resolve. | University | Jun 2025 | Not in cards | |
| DreamGen Bench Video generation benchmark measuring how well video models adapt to a target robot embodiment. Part of NVIDIA's DreamGen pipeline. | May 2025 | Not in cards | ||
| EWMBench Embodied world models generate robot manipulation videos from language. Scored on scene consistency, motion correctness, semantic alignment. | May 2025 | Not in cards | ||
| General-Bench Models perform multimodal understanding and generation tasks. General-Level averages task scores and measures performance above specialist baselines at progressively broader levels. | University | May 2025 | | Not in cards |
| GIE-Bench Image-editing models follow text instructions while preserving unedited regions. GPT-4o question answers and masked image metrics score functional correctness and preservation. | University | May 2025 | Not in cards | |
| ImgEdit-Bench Image editing models carry out edit instructions such as add, remove, replace, style transfer and background change. A GPT-4.1 judge scores each result from 1 to 5. | Everyday | May 2025 | | System cards only |
| IntelligentBench Image editing models complete 350 free-form edits that need reasoning and world knowledge. GPT-4o scores each output from 0 to 2 against a reference image, normalized to 100. | Everyday | May 2025 | | Dropped |
| KRIS-Bench Image editing models follow 1,267 instructions that need factual, conceptual or procedural knowledge across 22 tasks. A GPT-4o judge scores the edits. | School | May 2025 | | System cards only |
| NarrLV Models generate videos from prompts that describe changes in a scene, an object, or an action. A multimodal judge checks what appears and in what order. | Everyday | May 2025 | | Not in cards |
| ViStoryBench Models turn stories and character references into coherent image sequences. Model judgments and image features measure prompt alignment, character consistency, style, and image quality. | University | May 2025 | | Not in cards |
| 3D Animation Arena Models reconstruct a moving 3D human from a video. Voters compare motion fidelity, ground contact, smoothness, and overall appearance. | Everyday | Apr 2025 | Not in cards | |
| GEdit-Bench Image editing models follow real-world user edit instructions. A model judge scores semantic consistency, perceptual quality and an overall score. | Everyday | Apr 2025 | | Dropped |
| RISEBench Image editing models follow 360 instructions that need temporal, causal, spatial or logical reasoning. An LMM judge scores instruction reasoning, appearance consistency and visual plausibility. | University | Apr 2025 | 51.4% | System cards only |
| WorldScore 3,000 next-scene generation tasks with camera-trajectory layouts. Scores 3D/4D and video world generators on controllability, quality, dynamics. | Apr 2025 | Not in cards | ||
| 3DGen-Bench People compare generated 3D objects in pairs. The authors use their judgments to evaluate automated scoring methods for text-to-3D and image-to-3D generation. | Everyday | Mar 2025 | Not in cards | |
| AutoAdvExBench Agents inspect adversarial-example defenses and produce attacks that perturb images within a fixed bound. The benchmark measures attack success against 51 real defense implementations. It reports defense accuracies and separate results on pedagogical and real-world subsets. | Expert | Mar 2025 | 22% | Not in cards |
| ETVA Text-to-video systems are scored on fine-grained prompt alignment through generated atomic questions and video-language answers. | Expert | Mar 2025 | Not in cards | |
| FiVE-Bench Models edit objects in videos from written instructions. A vision language model checks whether each edit succeeded, and other metrics measure changes to the background. | University | Mar 2025 | Not in cards | |
| FunGraph3D Models identify objects and their controls in indoor scans. They predict which part operates which object, such as a switch that controls a light. | Everyday | Mar 2025 | Not in cards | |
| LEGO-Puzzles Vision-language models answer LEGO assembly questions, from basic spatial understanding to planning several assembly steps. The elementary set has 1,100 questions over 11 tasks. | Everyday | Mar 2025 | 72% | Not in cards |
| OpenLex3D Models label points and retrieve objects in indoor 3D scans. Annotators distinguish an object’s synonyms from pictures on its surface and visually similar objects. | Everyday | Mar 2025 | Not in cards | |
| VBench-2.0 Video generation models are scored on intrinsic faithfulness, such as human fidelity, controllability, creativity, physics and commonsense, beyond pixel quality and basic prompt match. | Everyday | Mar 2025 | | Not in cards |
| VideoPhy-2 200 action-centric prompts. Generated videos human-rated for semantic adherence, physical commonsense and physical-rule grounding. | Mar 2025 | | System cards only | |
| WISE Text-to-image models draw 1,000 prompts that need world knowledge in culture, space, time and natural science. A vision model judge gives the WiScore. | School | Mar 2025 | 0.82 | System cards only |
| AGI-Eval Community Text to Image Models generate images from text. The board reports objective, subjective and combined scores. | Everyday | Feb 2025 | | Not in cards |
| AGI-Eval Community Text to Video Models generate videos from text. The board reports objective, subjective and combined scores. | Everyday | Feb 2025 | | Not in cards |
| Massive Visualized IR Benchmark Models retrieve screenshots that match a search query. The benchmark compares performance across four retrieval task groups. | Everyday | Feb 2025 | Not in cards | |
| WorldModelBench Video generators judged as world models on instruction following and physics adherence in application domains. Human labels and a fine-tuned judge. | Feb 2025 | Not in cards | ||
| Physics-IQ Video models continue 396 filmed physical events from a frame or 3 s clip. Motion compared with the real continuation. | Jan 2025 | | System cards only | |
| ViGiL3D Models locate objects in indoor 3D scans from English descriptions. A description can match several objects or none. Models must return the matching set. | Everyday | Jan 2025 | Not in cards | |
| Articulate3D Models locate movable parts and handles in scanned rooms. They predict how each part moves, including its axis and the location of its hinge. | School | Dec 2024 | Not in cards | |
| Gray Swan Jailbreak Arena Human attackers try to jailbreak models for violent, self-harm and malicious code outputs. Scored as attack success rate per category. | Expert | Dec 2024 | 3.60% | System cards only |
| Labelbox Image Generation v2 Models generate images from text prompts. Human raters compare how well the outputs meet the request. | Everyday | Dec 2024 | 1,069 | Not in cards |
| Labelbox Video Generation v2 Models generate videos from text prompts. Human raters compare the outputs for visual quality and adherence to the prompt. | Everyday | Dec 2024 | | Not in cards |
| LMArena Text-to-Image Arena Image generation models answer the same user prompt side by side and anonymously. Community votes between the two images give Elo-style ratings. | Everyday | Dec 2024 | 1,428 | Not in cards |
| ScanNet++ v2 Models reconstruct indoor scenes from images of real scans. The second release adds scene types and updates the annotations and camera poses. | School | Dec 2024 | Not in cards | |
| GEOBench-VLM Models interpret satellite and aerial images, identify land use, and track changes across observations. Some tasks ask models to locate objects or outline regions. | University | Nov 2024 | 0.41 | Not in cards |
| VBench image-to-video Models animate input images. Automatic checks score image conditioning and video quality across the complete image-to-video suite. | Everyday | Nov 2024 | | Not in cards |
| Harmony4D Models recover the body shapes and poses of people interacting at close range. The benchmark includes sports where people occlude each other. | School | Oct 2024 | Not in cards | |
| LEVIR-CDC Models describe changes between aerial images and mark changed buildings. The authors paired LEVIR-CC captions with binary building masks. | Everyday | Oct 2024 | Not in cards | |
| MMIE Models interpret and generate interleaved text and images. MMIE-Score, a fine-tuned InternVL-2-4B model, grades the resulting multimodal responses. | University | Oct 2024 | Not in cards | |
| Movie Gen Video Bench Video generation models make clips from a fixed prompt set, and people compare them. Meta released non-cherry-picked Movie Gen videos for comparison. | Everyday | Oct 2024 | Not in cards | |
| OBI-Bench Models locate inscriptions, rejoin bone fragments, classify characters, retrieve matches, and explain glyphs. The authors publish separate scores for each task. | Expert | Oct 2024 | 0.61 | Not in cards |
| QAG-360K Models compare two satellite images, answer questions about land cover, and mark the relevant pixels. The authors score answer accuracy and mask overlap separately. | School | Oct 2024 | Not in cards | |
| TaxaBench-8k Models identify species or match observations across photographs, sounds, and other field data. The test contains 8,813 observations paired across six types of data. | Expert | Oct 2024 | Not in cards | |
| WHU-CDC Models compare aerial photos of Christchurch, describe what changed, and mark changed pixels. Reference captions describe reconstruction after the earthquake. | Everyday | Oct 2024 | Not in cards | |
| WorldSimBench Dual evaluation of video world simulators: human-preference perceptual scoring plus action-level embodied tasks in games, driving, manipulation. | Oct 2024 | Not in cards | ||
| Labelbox Image Generation (original) Models generate images from text prompts. Human raters compare how well the outputs meet the request. | Everyday | Sep 2024 | Not in cards | |
| Labelbox Video Generation (original) Models generate videos from text prompts. Human raters compare the outputs for visual quality and adherence to the prompt. | Everyday | Sep 2024 | Not in cards | |
| Text2CAD Models generate parametric CAD designs from text prompts at different levels of detail. The authors compare generated geometry with the reference and count invalid designs. | University | Sep 2024 | Not in cards | |
| GeoSound Models match environmental recordings to satellite images of their locations and retrieve recordings from images. The test holds out geographic areas and contains 9,931 examples. | Everyday | Aug 2024 | Not in cards | |
| K-Sort Arena Users rank outputs from K image or video generation models at once instead of a pair. Bayesian updates turn the rankings into ratings. | Everyday | Aug 2024 | 31 | Not in cards |
| OpenCanopy Models estimate tree canopy height from satellite images. The evaluation compares predicted height maps with measurements from airborne laser scans. | University | Jul 2024 | Not in cards | |
| ReasonVOS Models segment and track objects in videos from explicit referring queries and implicit reasoning queries. The score is overall J&F on ReVOS. | Everyday | Jul 2024 | Not in cards | |
| SA-37 interactive segmentation Models refine an object mask after a user clicks on an image. The benchmark averages overlap across datasets at different numbers of clicks. | Everyday | Jul 2024 | Not in cards | |
| Segment Anything Video Models predict masks for prompted objects throughout a video. The evaluation compares predicted boundaries and regions with annotated object tracks. | Everyday | Jul 2024 | Not in cards | |
| 1X World Model Challenge Predict future egocentric frames of 1X's EVE humanoid from video and state. Compression scored by token cross-entropy, Sampling by PSNR on private tests. | Jun 2024 | Not in cards | ||
| ChronoMagic-Bench Video models generate time-lapse scenes from 1,649 prompts covering biological, human-made, weather, and physical changes. Automatic metrics measure the amount of change and temporal coherence, with a separate GPT-4o assessment of change. | Everyday | Jun 2024 | Not in cards | |
| GeckoNum Models generate images from prompts about quantities. Human raters check exact counts, approximate quantities, and reasoning about numerical concepts. | Everyday | Jun 2024 | 47.6% | Not in cards |
| GenAI-Arena Users compare outputs from two anonymous models for image generation, image editing and video generation, and vote. Votes give Elo ratings. | Everyday | Jun 2024 | 1,133 | Not in cards |
| GenAI-Arena Image Editing Models edit images from user instructions. Anonymous human comparisons determine Elo in this independent vote pool. | Everyday | Jun 2024 | Not in cards | |
| GenAI-Arena Image Generation Models generate images from text prompts. Anonymous human comparisons determine Elo in this independent vote pool. | Everyday | Jun 2024 | 1,133 | Not in cards |
| GenAI-Arena Video Generation Models generate videos from text prompts. Anonymous human comparisons determine Elo in this independent vote pool. | Everyday | Jun 2024 | Not in cards | |
| SuperCLUE-Image Models generate images from Chinese prompts. The benchmark scores text rendering, visual quality, prompt adherence and scene composition. | Everyday | Jun 2024 | 94.3 | Not in cards |
| VideoPhy Text-to-video models generate clips from captions of physical interactions between solids and fluids. Human raters judge caption match and physical commonsense. | Everyday | Jun 2024 | Not in cards | |
| Long-term Video Object Segmentation v2 Models track object masks through long videos. The evaluation tests whether they can recover an object's identity after an extended absence. | Everyday | Apr 2024 | Not in cards | |
| DPG-Bench Text-to-image models draw 1,000 dense prompts with many objects, attributes and relations. A VQA model checks each element of the prompt graph in the image. | Everyday | Mar 2024 | | Fading |
| SceneFun3D Models identify movable parts in indoor scans, locate the part needed for a written task, and estimate how it moves. | Everyday | Mar 2024 | Not in cards | |
| TGVE+ Models follow video editing instructions on the original 76 TGVE videos. The extension adds object removal, object addition, and texture changes. | University | Mar 2024 | Not in cards | |
| ChartX Vision-language models do 7 chart tasks, such as question answering, summarization and redrawing, over 18 chart types and 22 topics. | Everyday | Feb 2024 | 20.9 | Not in cards |
| SuperCLUE-Video Models generate videos from Chinese prompts. A judging model examines video frames and scores them against criteria for the prompt. | School | Feb 2024 | | Not in cards |
| Ego-Exo4D Models estimate human pose during skilled activities filmed from first-person and external viewpoints. Meta reports joint and mesh errors for physical and procedural activities. | School | Dec 2023 | Not in cards | |
| HEIM Text-to-image models are scored on 12 aspects, such as alignment, quality, aesthetics, originality, reasoning, knowledge, bias and toxicity, with human raters and automatic metrics. | Everyday | Nov 2023 | Not in cards | |
| VBench Video generation models make clips from prompts, and automatic checks score 16 dimensions of quality and prompt adherence. | Everyday | Nov 2023 | | Fading |
| EvalCrafter Text-to-video models generate clips for 700 prompts drawn from real user requests. The clips are scored with 17 automatic metrics for visual quality, content quality, motion and text alignment. Weights fitted to human ratings combine the metrics into one score. | Everyday | Oct 2023 | Not in cards | |
| GenEval Text-to-image models draw prompts about object count, color, position and co-occurrence, and an object detector checks each image. Scored by the share of correct images. | Everyday | Oct 2023 | | Fading |
| Ref-ZOM Models mark the pixels described by a phrase in a COCO image. A phrase can refer to zero, one or several objects. | Everyday | Oct 2023 | Not in cards | |
| Code Interpreter Benchmark Models use a Python code interpreter for math, data visualization, and general tasks. Scoring covers result accuracy and the share of code that runs. | School | Sep 2023 | | System cards only |
| EMDB Models reconstruct human pose and body shape in everyday scenes. The benchmark compares joint positions with electromagnetic measurements. | School | Aug 2023 | Not in cards | |
| MeViS Models segment the objects in a video that a sentence picks out by their motion, not by static looks. Scored by J&F. | Everyday | Aug 2023 | Not in cards | |
| ReasonSeg Models infer which object an instruction describes and return its mask. Instructions can require reasoning about the image. | Everyday | Aug 2023 | Not in cards | |
| OpenSUN3D Open-Vocabulary 3D Object Instance Search Models read a description and mark every matching object in a scanned room. The organizers rank predictions by mean average precision across 25 test scenes. | Everyday | Jul 2023 | Not in cards | |
| PUMaVOS Models track masks for object parts across video frames. A prompt can identify a small region, such as one cheek of a person's face. | Everyday | Jul 2023 | Not in cards | |
| Aria Digital Twin Models infer the layout of objects from first-person images. The SAM 3D Objects evaluation compares predicted positions and orientations with reference scene geometry. | School | Jun 2023 | Not in cards | |
| LOVEU-TGVE-2023 Models edit 76 videos with four instructions per video. People compare each submission with Tune-A-Video on text alignment, structure, and visual quality. | University | May 2023 | Not in cards | |
| Large-Vocabulary Video Instance Segmentation Models detect, segment, and track object instances across video frames. The benchmark includes categories outside the training vocabulary. | Everyday | Apr 2023 | Not in cards | |
| Hi4D Models reconstruct the bodies of people interacting at close range. The SAM 3D Body evaluation compares joint and mesh errors with motion capture references. | School | Mar 2023 | Not in cards | |
| Event-based Segmentation Dataset Models segment objects in cluttered indoor scenes filmed from a moving robot arm. The benchmark provides reference masks for comparison. | Everyday | Feb 2023 | Not in cards | |
| MOSE Models follow and segment objects in videos with crowds, occlusions, and objects leaving the screen. The benchmark compares masks against human annotations. | Everyday | Feb 2023 | Not in cards | |
| Video Object Segmentation under Transformations Models follow objects whose appearance changes as people manipulate them. The benchmark compares predicted masks with annotations through these transformations. | Everyday | Dec 2022 | Not in cards | |
| Inappropriate Image Prompts Models generate images from prompts drawn from real users. The evaluation measures how often the outputs contain inappropriate material. | Everyday | Nov 2022 | Not in cards | |
| Common Syntactic Processes in Text-to-Image Generation Models generate images from sentences with different grammatical structures. Evaluators check whether each image preserves the relationships expressed in the sentence. | School | Oct 2022 | Not in cards | |
| BURST Models recognize, segment, and track objects across video frames. The benchmark tests how well they associate the same object across time. | Everyday | Sep 2022 | Not in cards | |
| Robust Multi-view Depth Models estimate depth from multiple photographs of a scene. The benchmark compares relative depth errors across datasets from different environments. | School | Sep 2022 | Not in cards | |
| EPIC-KITCHENS VISOR Hand Object Segmentation Models outline each hand and the object it touches in a kitchen image. They also identify which hand appears and whether it touches anything. | Everyday | Aug 2022 | Not in cards | |
| EPIC-KITCHENS VISOR Semi-Supervised Video Object Segmentation Models receive masks for hands and objects in the first frame of a kitchen video. They track those objects through later frames, including after occlusion. | Everyday | Aug 2022 | Not in cards | |
| Omni3D A large benchmark for 3D object detection from a single image, built by unifying indoor and outdoor datasets such as SUN RGB-D, ARKitScenes, Hypersim, Objectron, KITTI and nuScenes. Vision-language models answer its 3D grounding questions. | Everyday | Jul 2022 | | System cards only |
| Testing Relational Understanding in Text-Guided Image Generation Models depict relationships between named entities. Human participants judge whether the generated scenes match each caption. | Everyday | Jul 2022 | Not in cards | |
| PartiPrompts Models generate images from English prompts spanning subjects and difficulty types. Human raters compare image realism and fit to the prompt. | Everyday | Jun 2022 | Not in cards | |
| RICH Models reconstruct people interacting with their surroundings. The evaluation compares body joints and mesh vertices with the reference geometry. | School | Jun 2022 | Not in cards | |
| Crossmodal-3600 Models describe images in different languages or retrieve an image from a description. | School | May 2022 | | Not in cards |
| DrawBench Models generate images from text prompts that test counting, composition, and other skills. Human raters compare image quality and prompt alignment. | Everyday | May 2022 | Not in cards | |
| Google Scanned Objects Models synthesize an object from a new viewpoint. Researchers compare the rendered result with reference views of scanned household items. | School | Apr 2022 | Not in cards | |
| HierText Models locate and recognize text arranged in lines and paragraphs in real scenes. Google also uses the annotations to compare text and image retrieval. | School | Mar 2022 | Not in cards | |
| PaintSkills Models generate images of specified objects, counts, and spatial relationships. Detectors check whether an image satisfies the requested composition. | Everyday | Feb 2022 | Not in cards | |
| ARKitScenes Models locate objects in indoor RGB-D scenes captured on iPads and output 3D bounding boxes. Scored by 3D detection average precision. | Everyday | Nov 2021 | | Dropped |
| Ego4D State Change Object Detection Models locate the object that changes in a first-person video. The official test scores its bounding box in the frame where the change begins. | Everyday | Oct 2021 | Not in cards | |
| LoveDA Models label land cover in aerial images of urban and rural areas. The benchmark supports comparisons across these two settings. | University | Oct 2021 | Not in cards | |
| TREK-150 Models follow a marked object through first-person kitchen videos. In the one-pass test, they receive its bounding box in the first frame and track it to the end. | Everyday | Aug 2021 | Not in cards | |
| Unidentified Video Objects Models segment and track objects in video without a fixed list of object categories. The evaluation compares predicted masks and tracks with annotated instances. | Everyday | Apr 2021 | Not in cards | |
| PPR10K Models retouch groups of portraits against three expert retouchers’ Adobe Camera Raw edits. The test measures accuracy on people and color consistency across each group. | University | Mar 2021 | Not in cards | |
| Occluded Video Instance Segmentation Models track and segment objects that overlap or disappear behind other objects. The evaluation compares predicted masks and identities with annotated video instances. | Everyday | Feb 2021 | Not in cards | |
| Objectron Models predict 3D bounding boxes of objects in short object-centric videos across nine categories. Scored by 3D intersection over union. | Everyday | Dec 2020 | Dropped | |
| Hypersim Models localize named objects as 3D boxes in synthetic indoor Hypersim images, using the Omni3D test split. Scored by mAP at IoU 0.15. | Everyday | Nov 2020 | | Fading |
| MedICaT Models locate panels within a biomedical figure and match each panel to the relevant words in its caption. The original test scores word overlap. | University | Oct 2020 | Not in cards | |
| Ref-YouTube-VOS Models segment and track the object that a sentence describes in YouTube-VOS videos. Scored by J&F on a server-graded validation split. | Everyday | Aug 2020 | Not in cards | |
| Virtual KITTI 2 Models interpret simulated driving videos with reference geometry and pixel labels. Meta uses the sequences to compare video object segmentation models. | Everyday | Jan 2020 | Not in cards | |
| ARC-AGI Few-shot grid puzzles: infer a hidden transformation from about three examples and produce the output grid. Semi-private sets verified by ARC Prize. | Everyday | Nov 2019 | 98.5% | In use |
| ScanNet-1500 Models match features between pairs of indoor views and estimate relative camera poses. The evaluation measures pose accuracy on 1,500 image pairs. | School | Nov 2019 | Not in cards | |
| FreiHAND Models recover a hand's joint positions and surface from a color image. The evaluation compares predictions with reference hand geometry. | School | Sep 2019 | Not in cards | |
| DIODE Models estimate depth in indoor and outdoor scenes. The dataset provides depth measurements from a laser scanner for comparison. | School | Aug 2019 | Not in cards | |
| LVIS Models locate objects and outline each instance across a large vocabulary of categories. Meta reports box and mask average precision. | Everyday | Aug 2019 | Not in cards | |
| iSAID Models detect and segment individual objects in aerial images. The benchmark measures how well predicted masks match annotated object instances. | University | May 2019 | Not in cards | |
| YouTube-VIS Models detect, segment, and track object instances in YouTube videos. The SAM 3 report evaluates the 2021 edition. | Everyday | May 2019 | Not in cards | |
| Nuscene Models localize named objects as 3D boxes in nuScenes driving camera images. Scored by mAP. | Everyday | Mar 2019 | | Fading |
| nuScenes Detection Models detect 3D objects in nuScenes sensor data. The official test leaderboard ranks submissions by nuScenes Detection Score. | Everyday | Mar 2019 | Not in cards | |
| 3DPW Models recover a person's three-dimensional pose in outdoor video. Reference poses come from wearable sensors and a moving camera. | School | Sep 2018 | Not in cards | |
| YouCook2 retrieval Models retrieve cooking videos from text queries in the YouCook2 collection. | University | Sep 2018 | | Fading |
| YouTube-VOS Models segment and track objects across frames in YouTube videos. The evaluation distinguishes categories seen during training from previously unseen categories. | Everyday | Sep 2018 | Not in cards | |
| Places Audio Captions (Hindi) Models retrieve photographs from spoken Hindi descriptions or retrieve descriptions from photographs. The score counts how often the correct match appears among the top results. | Everyday | Apr 2018 | Not in cards | |
| Ref-DAVIS17 Models segment and track the object that a sentence describes through each frame of a DAVIS 2017 video. Scored by J&F, the mean of mask overlap and contour accuracy. | Everyday | Mar 2018 | Not in cards | |
| DAVIS 2017 Models track and segment individual objects throughout a video. The evaluation measures region overlap and boundary accuracy. | Everyday | Apr 2017 | Not in cards | |
| ScanNet Models infer indoor scene geometry from images. Meta evaluates depth predictions against measurements from scanned indoor environments. | School | Feb 2017 | Not in cards | |
| ADE20K Models label the pixels in a scene by object or region. Meta measures overlap between predicted and reference segments. | Everyday | Aug 2016 | Not in cards | |
| Cityscapes Models assign a class to each pixel in street scenes filmed from a vehicle. The benchmark compares predicted segments with annotated road scenes. | Everyday | Apr 2016 | Not in cards | |
| SUNRGBD Models detect objects in about 10,000 indoor RGB-D images and predict their 3D bounding boxes. Scored by average precision. | Everyday | Jun 2015 | | Fading |
| PASCAL-Context Models assign a semantic class to each pixel in a photograph. The benchmark includes objects and background regions. | Everyday | Jun 2014 | Not in cards | |
| PASCAL VOC 2012 Models locate objects and label their pixels in photographs. Meta reports detection and segmentation results on the 2012 challenge. | Everyday | May 2012 | Not in cards | |
| Leeds Sports Pose Extended Models locate body joints in photographs of athletes. Meta compares the fraction of predicted joints within a distance threshold. | School | Jun 2011 | Not in cards | |
| MIT-Adobe FiveK Models adjust photographic tone to match human edits. Five photographers each edited 5,000 raw photos in Lightroom. The original evaluation compares luminance with each reference edit. | University | Jun 2011 | Not in cards | |
| 3D Arena Users compare two 3D models generated from the same input and vote for the better one. A live leaderboard ranks the generators. | Everyday | Not in cards | ||
| 3D Worlds Arena Models generate explorable 3D scenes from an image and caption. Voters inspect two anonymous scenes from the same input and choose a favorite. | Everyday | Not in cards | ||
| AA-Image-Editing Validated private evaluators compare two image edits made from the same instruction and vote blind. Votes become Elo ratings. | Everyday | Not in cards | ||
| AA-Video-Editing Voters compare two edits of the same video made from one instruction and pick the better one. Votes become Elo ratings. | Everyday | Not in cards | ||
| Artificial Analysis Image Arena Voters compare images from two text-to-image models for the same prompt and pick the better one. Votes become Elo ratings. | Everyday | Not in cards | ||
| Artificial Analysis Video Arena Voters compare videos from two text-to-video models for the same prompt and pick the better one. Votes become Elo ratings. | Everyday | Not in cards | ||
| Chart Arena Models create charts. Pairwise votes determine the published Arena rating. | Everyday | 1,615 | Not in cards | |
| COCO-Stuff 164K Models label each pixel with an object or background category. The benchmark adds annotations for regions such as sky, grass, and pavement. | Everyday | Not in cards | ||
| Common Objects in 3D v2 Models estimate camera poses from images of objects viewed from different angles. Meta compares predicted poses with the reference poses. | School | Not in cards | ||
| DesignArena 3D Design Models build an interactive 3D scene that runs in the browser as a single HTML file. Users explore the scenes and vote for the better one. | Everyday | 1,535 | In use | |
| DesignArena Blender Agents build a 3D scene in Blender from a user prompt. Users compare the rendered scenes and vote for the better one. | Everyday | 1,381 | Not in cards | |
| DesignArena Code to Video Agents write code that renders a video from a user prompt. Users compare the two videos and vote for the better one. | Everyday | 1,454 | Not in cards | |
| DesignArena Graphic Design Image models make a graphic design piece from a user prompt. Users compare two anonymous designs and vote for the better one. | Everyday | 1,436 | Not in cards | |
| DesignArena Image Image models generate a picture from a user prompt. Users compare two anonymous images and vote for the better one. | Everyday | 1,381 | Not in cards | |
| DesignArena Image Editing Image models edit a user image following a text instruction. Users compare the two edits and vote for the better one. | Everyday | 1,368 | Not in cards | |
| DesignArena Image to Video Video models animate a user image into a short video. Users compare two anonymous videos and vote for the better one. | Everyday | | Not in cards | |
| DesignArena Logo Image models design a logo from a user prompt. Users compare two anonymous logos and vote for the better one. | Everyday | 1,365 | Not in cards | |
| DesignArena Multi-Image to Video Video models compose one video from several reference images. Users compare two anonymous videos and vote for the better one. | Everyday | | Not in cards | |
| DesignArena Multimodal to Video Video models make a video from a mix of text, images and other inputs. Users compare two anonymous videos and vote for the better one. | Everyday | | Not in cards | |
| DesignArena Video Arena Users compare videos from anonymous video generation models for the same prompt and vote for the better one. Votes become Elo ratings. | Everyday | | Not in cards | |
| DesignArena Video Editing Video models edit a user video following a text instruction. Users compare the two results and vote for the better one. | Everyday | | Not in cards | |
| DTU Multi-View Stereo Models reconstruct objects from multiple camera views. The benchmark measures the accuracy and completeness of reconstructed surfaces against scanned reference geometry. | School | Not in cards | ||
| EndoVis 2018 Robotic Scene Segmentation Models segment structures and surgical instruments in video from robot-assisted operations. The evaluation compares predicted masks with reference labels. | Everyday | Not in cards | ||
| ETH3D Models reconstruct scene geometry from images. Meta evaluates depth predictions against reference measurements on the ETH3D test scenes. | School | Not in cards | ||
| Floorplan Arena Models furnish hand-drawn room sketches. People compare the layouts and vote for the better floorplan. | Everyday | | Not in cards | |
| GameAdsBench Models generate video ads for games. Viewers watch two anonymous ads and choose the one they prefer. | University | | Not in cards | |
| GenAI-Bench Image generation systems are compared by human side-by-side preference on text-to-image prompts. | Everyday | | Fading | |
| GenExam Image generation systems are evaluated on subject-focused educational tasks with expert ratings. | University | Not in cards | ||
| Goliath human mesh recovery Models reconstruct a person's body from images of studio performances. Meta measures joint and surface errors against captured three-dimensional motion. | School | Not in cards | ||
| Image Matching Challenge Models match photographs of landmarks taken from different viewpoints. The evaluation measures the accuracy of camera poses recovered from the matches. | School | Not in cards | ||
| KITTI depth prediction Models estimate scene depth from road images. The evaluation compares model predictions with depth measurements collected from a moving vehicle. | School | Not in cards | ||
| LMArena Video Edit Arena Video editing models apply the same edit request to the same clip, anonymously and side by side. Community votes give Elo-style ratings. | Everyday | | Not in cards | |
| NYU Depth V2 Models estimate depth from indoor photographs. The benchmark compares predictions with depth measurements captured by a sensor. | School | Not in cards | ||
| OpenDesign Arena Models create design artifacts in the OpenDesign tool, and the board reports an evaluation score out of 100 with cost and speed per artifact. | Everyday | 91.0 | Not in cards | |
| RealEstate10K Models estimate relative camera poses across frames from real estate videos. The benchmark supplies camera trajectories for comparison. | School | Not in cards | ||
| StudyArena Image Models produce image responses to user prompts. Anonymous human comparisons determine the published rating in this separate vote pool. | Everyday | 1,284 | Not in cards | |
| SuperCLUE Arena Image Editing Models edit images in response to instructions. The arena ranks systems from pairwise comparisons. | University | 1,308 | Not in cards | |
| SuperCLUE Arena Image to Video Models animate images. The arena ranks systems from pairwise comparisons. | University | Not in cards | ||
| SuperCLUE Arena Reference to Video Models generate videos from references. The arena ranks systems from pairwise comparisons. | University | Not in cards | ||
| SuperCLUE Arena Text to Image Models generate images from text. The arena ranks systems from pairwise comparisons. | University | 1,458 | Not in cards | |
| SuperCLUE Arena Text to Video Models generate videos from text. The arena ranks systems from pairwise comparisons. | University | Not in cards | ||
| SuperCLUE-ComicShorts Models generate short animated comics. | University | | Not in cards | |
| SuperCLUE-I2V Models generate videos from still images. | University | | Not in cards | |
| SuperCLUE-R2V Models generate videos conditioned on reference material. | University | 64.9 | Not in cards | |
| SuperCLUE-VideoEdit Models edit videos in response to Chinese instructions. | University | | Not in cards | |
| SuperCLUE-World World models generate visual sequences. | University | | Not in cards | |
| TartanAir v2 Models reconstruct camera motion and scene geometry from simulated trajectories. The evaluation compares predictions with known poses and depths. | School | Not in cards | ||
| VIPSeg Models label video pixels and keep instance identities consistent across frames. The benchmark includes both countable objects and background regions. | Everyday | Not in cards | ||
| WMArena Video Arena Models generate a video from a starting image and an action. Viewers compare anonymous clips and vote for the more convincing result. | University | | Not in cards | |
| Wojak-Bench 3D Models turn the same drawing into a three-dimensional figure. Voters compare two anonymous prints or renders and choose which better matches the drawing. | University | 1,273 | Not in cards |