Benchmark Almanac

AGI-Eval

We track 18 benchmarks and 7 arenas that AGI-Eval made or helped make.

1 in use 16 with public code 20 with a leaderboard here

Arena leaderboards

Top five in each of its 5 arenas with three or more rated models, the most rated first. We don't have a leaderboard yet for 2 others, listed below.

Benchmarks

Name Kind Released Best score Status
RWSearch Agents search the live web to answer 200 expert-written Chinese questions that need many retrieval steps, filtering and arithmetic. Each question has one objective answer. Deep research Jan 2026 82% Fading
FORTE Agents complete 180 office tasks across 15 corporate professions inside a workspace of xlsx, docx, pptx, pdf and other files. An LLM judge checks the output files against expert rubrics, and a task passes only if every rubric item passes. Documents and spreadsheets Jun 2026 77.8% In use
OIBench Models write programs for 250 original olympiad in informatics problems. Hidden tests check correctness, and time and space completion curves compare efficiency with reference solutions. Competitive programming Jun 2025 36.4% Fading
AGI-Eval Community Instruction-Tuned Language Models Models answer Chinese and English questions across reasoning, knowledge, learning, interaction and instruction following. Reasoning in many languages Mar 2025 0.86 Not in cards
AGI-Eval Community Image and Video Understanding Models interpret images and videos. The board reports visual question accuracy and separate comparison win rates. General visual questions Feb 2025 86.5 Not in cards
AGI-Eval Community Base Language Models Base models answer Chinese and English questions across seven skill areas. The board reports objective accuracy. Reasoning in many languages Feb 2025 0.71 Not in cards
AGI-Eval Community Text to Image Models generate images from text. The board reports objective, subjective and combined scores. Image generation and editing Feb 2025 0.64 Not in cards
AMemGym Assistants hold long simulated conversations with users whose details and preferences change over time. Simulated users react on policy, and the test checks whether the assistant remembers and applies the right facts. Learning from the context Jan 2026 55.6% Not in cards
CoreCodeBench Models complete over 1,500 repository-level Python tasks covering development, bug fixing and test-driven work, with single-function and multi-function variants. The CorePipe pipeline builds tasks from repositories at set difficulty levels. Fixing real repositories Jul 2025 44.4% Not in cards
UniGUI-Bench Agents interpret and operate graphical interfaces. The benchmark tests 11 capabilities and includes separate system evaluations. Finding things on screen May 2026 80.9% Not in cards
FORTE-MM Agents complete multimodal office tasks across 14 professions. A run passes only when the agent's output satisfies every task rubric. Documents and spreadsheets Nov 2026 74.5% Not in cards
AGI-Eval Community Text to Video Models generate videos from text. The board reports objective, subjective and combined scores. Video generation Feb 2025 62.5 Not in cards
DailyReport Search agents answer practical daily information requests. Evaluators assess the resulting reports across dimensions of user satisfaction. Deep research Jun 2026 2.89 Not in cards
PRDBench Code agents build 50 real-world Python projects across 20 domains from product requirement documents. A trained judge agent checks each project against structured criteria. Building whole projects Oct 2025 69% Not in cards
AgentEscapeBench Agents solve procedural escape rooms using unfamiliar tools. The evaluator checks whether they complete dependent steps and escape. Puzzles and chess May 2026 76.9% Not in cards
AGI-Eval Community Speech Synthesis Models synthesize speech. The board ranks models on how human the voices sound. Audio understanding and voice May 2025 13.0 Not in cards
AGI-Eval Community Voice Interaction Models converse by voice. The board scores naturalness, problem solving, learning assistance, leisure and empathy. Audio understanding and voice May 2025 0.60 Not in cards
SIGHT (Search and Intent Gauge for Hospitality and Tourism) Agents recommend restaurants, hotels and attractions from user preferences and search results. Evaluators check constraints, evidence and privacy. Apps and MCP servers May 2026 Not in cards

Arenas without a leaderboard here

Name Kind Released Best score Status
CATArena Pommerman Code agents compete in Pommerman. The public competition board reports Elo Rating for each agent configuration. Building whole projects Oct 2025 Not in cards
CATArena Tetris Code agents compete in Tetris. The public competition board reports Score for each agent configuration. Building whole projects Oct 2025 Not in cards