Benchmark Almanac

Nanjing University

We track 13 benchmarks that Nanjing University made or helped make.

1 in use 10 with public code 11 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
ScreenSpot Model locates a described UI element in mobile, desktop and web screenshots. Scored by click-point accuracy. Finding things on screen Jan 2024 95.4% Fading
OmniVideoBench Models answer 1,000 questions about videos that need audio and visual evidence together. Each question has annotated reasoning steps. Video understanding Oct 2025 61.8% Fading
TimeLens-Bench Models find the start and end time of a described event in a video. The benchmark re-annotates Charades-STA, ActivityNet Captions and QVHighlights with strict quality rules, and scores by mean IoU. Video understanding Dec 2025 62.9% Fading
OmniCap-IF Models write captions for videos with audio while following detailed user constraints. Captions are scored on format and content compliance. Video captioning Jun 2026 80.7% In use
BenchMAX Models follow instructions, write code, reason, use tools, translate, and retrieve information across 17 languages. Ten tasks use rule checks, model-judged wins, code tests, answer matching, or translation scores. Reasoning in many languages Feb 2025 85.2 Not in cards
OPT-BENCH Agents iteratively improve solutions to 20 machine-learning tasks and 10 combinatorial optimization problems using execution feedback. Win counts, invalid-solution rates, ranks, and improvement rates measure optimization performance. Scientific code Jun 2025 0.69 Not in cards
TestPrism Coding agents write tests for 300 tasks. A test passes only if it fails on the starting code, accepts every valid implementation and rejects every invalid one. Function-level problems Oct 2026 33.3% Not in cards
ContextBench Coding agents resolve 1,136 issues from 66 repositories in eight programming languages, and the benchmark checks which code they look at along the way. Each task has human-annotated gold context, and the tracker scores context recall, precision and efficiency over the agent's trajectory. It adds these process measures to the usual issue-resolution result. Fixing real repositories Feb 2026 60.6% Not in cards
CodeCap-Bench Models watch a structured 2D video and write code that re-renders it. The rendered video is compared with the original on visual similarity, object timing and placement, and retained detail. Video captioning Oct 2026 73.2 Not in cards
MM-AlignBench Models answer open-ended questions about 252 human-annotated images. GPT-4o compares their answers with Claude-Sonnet-3 reference answers to assess alignment with human preferences. General visual questions Feb 2025 82.5% Not in cards
LHRS-Bench Models answer 690 single-choice questions about 108 remote sensing images. The questions cover land use, object locations, image resolution, and other properties. General visual questions Mar 2024 39.1% Not in cards
CoVEBench Video editing models follow compositional instructions with several coupled edits to subjects, actions and camera, while unrelated content must stay unchanged. Video generation Jun 2026 55.2 Not in cards
MultiRef-Compass Models generate audio-video content from several visual references and text instructions across 350 samples. Fourteen metrics assess quality, reference fidelity, audio-visual consistency, and instruction following. The evaluator combines automatic measurements with multimodal model judgments. Video generation Jul 2026 Not in cards