Benchmark Almanac

Factory

We track 2 benchmarks that Factory made or helped make.

0 in use 1 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Factory Code Review Benchmark Models review real pull requests. Factory measures precision and recall against reference issues and reports mean F1. Fixing real repositories Apr 2026 60.5% Not in cards
Legacy-Bench Agents debug, add features to and migrate enterprise code in COBOL, Java 7, BASIC, C89, Fortran and Assembly. A task passes only when hidden tests all pass. Fixing real repositories Sep 2026 60% Not in cards