Benchmark Almanac

MLCommons

We track 4 benchmarks that MLCommons made or helped make.

0 in use 3 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
AILuminate Safety 1.2 Models answer 3,900 hidden English prompts, 325 in each of 12 hazard categories, picked with item response theory. A calibrated evaluator gives a 0 to 100 score. Refusing harmful requests Sep 2026 96.0 Not in cards
MLCommons AILuminate Industry-standard hazard benchmark over 12 risk categories with graded ratings. Refusing harmful requests Dec 2024 96% Not in cards
MLCommons AI Safety Benchmark v0.5 Chat models answer 43,090 English prompts across seven hazards. LlamaGuard labels the responses, and the highest risk grade across hazards determines the overall grade. Refusing harmful requests Apr 2024 Not in cards
MLCommons AILuminate Jailbreak Models answer hazard prompts with and without taxonomy-based jailbreak attacks. The Resilience Gap is the drop in safety score under attack. Jailbreaks and prompt injection Dec 2025 Not in cards