Benchmark Almanac

Mohamed bin Zayed University of Artificial Intelligence

We track 21 benchmarks that Mohamed bin Zayed University of Artificial Intelligence made or helped make. Its models are on its lab page.

0 in use 11 with public code 13 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
CMMLU Chinese multiple-choice questions across 67 subjects including China-specific topics. Accuracy. Exam questions Jun 2023 90.8% Fading
IndoMMLU Models answer 14,981 multiple-choice questions from Indonesian school and university entrance exams, including nine local languages. Scored by accuracy. Knowledge in many languages Oct 2023 35.8% Dropped
ArabicMMLU Models answer 14,575 multiple-choice questions in Modern Standard Arabic across 40 tasks, taken from school exams in North Africa, the Levant and the Gulf. Knowledge in many languages Feb 2024 71.4% Dropped
MadinahQA Models answer Arabic school exam questions on general Arabic language and grammar. Knowledge in many languages Sep 2024 69.4% System cards only
ArabCulture Models answer commonsense questions in Modern Standard Arabic about daily life in 13 Arab countries. Native speakers wrote and checked the questions across 12 domains. Reasoning in many languages Feb 2025 80.1% System cards only
CVQA Vision-language models answer culturally specific visual questions written by native speakers from many countries and languages. Knowledge in many languages Jun 2024 74.1% System cards only
LongShOTBench Models answer single-turn and multi-turn questions about long videos using visual, spoken, and ambient audio evidence. Weighted rubrics award partial credit. Long videos Dec 2025 64.2% Not in cards
AraGen Models answer Arabic questions in four tasks: question answering, reasoning, orthographic and grammatical analysis, and safety. An LLM judge scores each answer on the 3C3H measure: correctness, completeness, conciseness, helpfulness, honesty and harmlessness. Only correct answers get credit on the other dimensions. Chat and instructions in many languages Dec 2024 84.3% Not in cards
ArbMMLU-HT Models answer a human translation into Arabic of the 57 MMLU subjects. Knowledge in many languages Sep 2024 72.2% Not in cards
ThinkGeo Agents use remote sensing tools to interpret aerial and satellite images. The authors compare individual tool calls and use GPT-4o mini to grade final answers. Apps and MCP servers May 2025 100% Not in cards
Kinayat Models interpret Egyptian Arabic idioms, select appropriate contexts and identify connotations. The evaluation includes separate generation and multiple-choice tasks. Reasoning in many languages Oct 2025 95.3% Not in cards
SF20K-Test Models answer questions about the story in a short film. The authors check multiple-choice answers against a key and grade free responses with GPT-3.5. Long videos Jun 2024 62.8% Not in cards
Ego2Act Models generate first-person video showing how to complete a task from an initial image. A judge checks the steps and their physical plausibility. Video generation Oct 2026 83.3 Not in cards
OpenEarthAgent Agents use tools such as QGIS to measure features in satellite images and maps. The authors grade tool decisions and final answers. Apps and MCP servers Feb 2026 44.5% Not in cards
ArabCulture-Dialogue Models pick the most culturally fitting reply in dialogues from 13 Arab countries, in Modern Standard Arabic and each local dialect. It also has translation and dialect-steering tasks. Chat and instructions in many languages Apr 2026 95% Not in cards
GEOBench-VLM Models interpret satellite and aerial images, identify land use, and track changes across observations. Some tasks ask models to locate objects or outline regions. General visual questions Nov 2024 0.41 Not in cards
Alharthi LibriTTS Naturalness Listening Test Models estimate how natural synthesized speech sounds. Ten listeners rated each sample on a five-point scale for comparison. Audio understanding and voice Oct 2023 Not in cards
Grade Like a Human OS Dataset Models grade undergraduate operating systems answers. The authors measure absolute error against the mean grade from three teaching assistants per answer. Other professions May 2024 Not in cards
Jawaher Models translate Arabic proverbs into English and explain their meanings in English and Arabic. Human raters score translations and explanations on five-point scales. Translation Feb 2025 Not in cards
JEEM Vision-language models caption images and answer visual questions in four Arabic dialects. The evaluation compares outputs with reference annotations. Knowledge in many languages Apr 2025 Not in cards
PosterBench Agents turn research papers into posters. Programmatic checks and visual model judgments score seven content and design dimensions, with ceilings for serious rendering or layout failures. Documents and spreadsheets Aug 2026 Not in cards