Benchmark Almanac

Prime Intellect

We track 4 benchmarks that Prime Intellect made or helped make. Its models are on its lab page.

0 in use 1 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
NanoGPT Speedrun Frontier Agents change a GPT optimizer on fixed GPUs. Each candidate must reach the target validation loss across eight fixed seeds using fewer training steps. Machine learning engineering Aug 2026 2726.0 Not in cards
Mastermind (Prime Intellect) Models guess a hidden sequence and receive feedback after each turn. The default evaluation uses 50 games and rewards success, speed, and deductions. Puzzles and chess Oct 2025 1.62 Not in cards
LLM Writer Negative Style Models write responses to five prompts. Model judges flag stylistic habits such as promotional wording, editorializing, and formulaic conclusions. Taste and style Oct 2025 0.52 Not in cards
Click Calibrate Models click a circle or a named word in generated screenshots. The evaluation checks accuracy across image sizes and coordinate formats. Finding things on screen May 2026 Not in cards