Benchmark Almanac

Kaggle

We track 17 benchmarks and 34 arenas that Kaggle made or helped make.

0 in use 21 with public code 47 with a leaderboard here

Arena leaderboards

Top five in each of its 34 arenas with three or more rated models, the most rated first.

Benchmarks

Name Kind Released Best score Status
FACTS Grounding Long-form responses must be fully grounded in a provided document. Judged by LLM panels for grounding and eligibility. Faithful to sources Jan 2025 88.5% Dropped
FACTS Benchmark Suite Models are scored on four factuality tests: Parametric, Search, Multimodal and Grounding v2. The FACTS Score averages accuracy over public and private sets of all four. Short factual questions Dec 2025 78.3% Fading
FACTS Parametric Models answer closed-book factoid questions from their own knowledge, without tools. A judge model grades each answer. Short factual questions Dec 2025 83.1% Fading
FACTS Search Models answer information-seeking questions with a web search tool (the Brave Search API) and must combine what they find correctly. Short factual questions Dec 2025 85.6% Not in cards
FACTS Parametric V2 Models answer closed-book factoid questions from their own knowledge, without tools. A judge model grades each answer. Short factual questions Oct 2026 83.8% Not in cards
FACTS Grounding v2 Models write long answers from a supplied document of up to 32k tokens. Judge models check that each answer stays faithful to the document. Faithful to sources Dec 2025 84.2% Not in cards
FACTS Multimodal Models answer questions about input images and must combine what they see with world knowledge. A judge checks coverage and contradictions. Short factual questions Dec 2025 60.2% Not in cards
FACTS Search V2 Models answer information-seeking questions with a search tool and must combine what they find correctly. Second version of FACTS Search on Kaggle. Short factual questions Jul 2026 89.9% Not in cards
Code2Video Benchmark 168 product-video briefs. Agent writes HTML motion-graphics code rendered by HyperFrames. A trained pairwise judge on five axes produces Elo. Video generation Sep 2026 1,585 Not in cards
Human Behavior Bench Models predict how specified groups of U.S. adults answer closed survey questions, as a distribution over options. Scored as 1 minus total variation distance. Real-world events Oct 2026 80.6 Not in cards
NeurIPS 2025 Experts Benchmark Models answer tasks submitted by participants at NeurIPS 2025 in San Diego. Kaggle reports an overall score and individual task outcomes. Language reasoning Dec 2025 73.3% Not in cards
ICML 2025 Experts Benchmark Models answer tasks submitted by participants at ICML 2025 in Vancouver. Kaggle reports overall performance and individual task outcomes. Language reasoning Jul 2025 81.1% Not in cards
Kaggle's Task Tuesday Benchmark Models attempt tasks selected by the Kaggle team from community submissions. The board reports an overall score and individual task results. Language reasoning Jan 2026 83.3 Not in cards
Automated Student Assessment Prize: Automated Essay Scoring Models score student essays written for eight prompts. The competition compares predicted scores with human grades using quadratic weighted kappa. Other professions Jan 2012 0.00 Not in cards
Drawing with LLMs Models generate SVG code from text prompts. Kaggle renders each image and combines visual question answering with an aesthetic score. Image generation and editing Feb 2025 Not in cards
Kaggle Standardized Agent Exams Agents answer 16 questions on multistep reasoning and handling manipulative prompts. Kaggle publishes an overall score. Deception and alignment Mar 2026 Not in cards
LMSYS Chatbot Arena Human Preference Predictions Models predict which answer a user preferred in an anonymous pair. Kaggle scores the submitted probabilities by log loss. Overall chat quality May 2024 Not in cards