Benchmark Almanac

OpenDataLab

We track 5 benchmarks that OpenDataLab made or helped make.

0 in use 5 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Earth-Bench V2 Agents analyze prepared satellite observations. The corrected release measures final answer accuracy, tool selection, argument accuracy, and execution efficiency. Apps and MCP servers Oct 2026 80% Not in cards
CiteVQA Multimodal models answer questions about long PDFs and must cite the page region that supports each answer with a bounding box. The set has 1,897 questions over 711 documents in seven domains and two languages, averaging 40.6 pages per document. Strict Attributed Accuracy counts an answer only when both the answer and the cited region are correct. Documents, charts and text in images May 2026 76% Not in cards
LOKI Models distinguish synthetic content from real content across images, videos, audio, text, and 3D data. The benchmark checks real-or-synthetic labels and annotated abnormal details, with separate tasks for explaining anomalies. General visual questions Oct 2024 60.8% Not in cards
Earth-Bench Agents analyze satellite observations with Earth science tools. The evaluator checks the final answer and compares tool calls with a reference sequence. Apps and MCP servers Sep 2025 66% Not in cards
Earth-Bench-Pro Agents complete Earth observation workflows under three conditions. They follow instructions, plan from supplied observations, or obtain observations at runtime. Apps and MCP servers Sep 2026 Not in cards