Benchmark Almanac

UniPat AI

We track 12 benchmarks that UniPat AI made or helped make.

3 in use 9 with public code 12 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
BabyVision Model solves 388 perception-heavy visual puzzles (discrimination, tracking, spatial, patterns) that young children solve. Accuracy versus 94% human baseline. Spatial reasoning and counting Jan 2026 91.3% In use
SaaS-Bench 106 long-horizon GUI workflows across 23 locally deployed real SaaS apps. Scored from app databases with weighted checkpoints and full-resolution rate. Web browsing May 2026 31.1% In use
RoadmapBench Agents start from an older version of an open-source repository and implement a roadmap of features from its next real release. The set has 115 tasks from 17 repositories in 5 languages, scored by tests. Building whole projects May 2026 39.1% In use
Echo General AI Prediction Leaderboard Models forecast unresolved events drawn from prediction markets, synthesized news questions and expert-written questions. Forecasts made at aligned times are compared in pairwise battles, and an Elo fit ranks the models. Real-world events Mar 2026 1,018 Not in cards
EvoCode-Bench Coding agents keep one project working across 26 tasks of 5 to 15 rounds each, while requirements change, pile up and conflict. Cumulative hidden tests check every round in a persistent container. Building whole projects May 2026 54% Not in cards
EvoCode-Bench v2 Coding agents work through 26 multi-round tasks with 227 graded rounds on the official Harbor multi-step format. The score is the mean of binary per-round rewards. A case score gives partial credit. Building whole projects Jun 2026 59.1% Not in cards
Monthly-SWEBench Coding agents resolve 100 fresh GitHub tasks each month, half bug fixes and half features, from newly merged pull requests. Harbor verifiers score pass@1. Fixing real repositories Mar 2026 76% Not in cards
ScienceVision Agents inspect 3D scientific data such as proteins, crystals, CAD assemblies, buildings and CT volumes, then answer 250 expert questions. A rule-based scorer grades each answer field. Spatial reasoning and counting Oct 2026 0.53 Not in cards
PaperBenchX Agents reproduce published results from 12 research areas outside machine learning, using each field's own scientific software. Verifiers check that the evidence regenerates and supports the paper's claims. Research science Oct 2026 14% Not in cards
DeepTerminalBench Agents get one attempt at each of 50 deep engineering tasks in Docker, in the Terminal-Bench 2.0 format. A task passes only if every hidden test passes. Tasks were calibrated so Claude Opus 4.6 still fails some at pass@4. Terminal tasks May 2026 34% Not in cards
ExpertEval Models answer cases in medicine, finance and law. Practicing professionals write the cases and supply weighted grading rubrics. Other professions May 2026 66.9% Not in cards
Terminal-X A suite of three terminal datasets for coding agents. DeepTerminalBench tests single-shot depth, EvoCode-Bench tests multi-round iteration and RoadmapBench tests version upgrades on real repositories. Terminal tasks May 2026 42.5% Not in cards