Cognition
We track 4 benchmarks that Cognition made or helped make. Its models are on its lab page.
3 in use 0 with public code 3 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| FrontierCode 1.1 Real-repo coding tasks graded for mergeability: correctness, regressions, lint, tests, scope and code quality. Failing any blocker scores zero. | Fixing real repositories | Jul 2026 | 53.5% | In use |
| FrontierCode 2.0 Agents resolve 132 maintainer-built tasks from deliberately ambiguous prompts. Rubric checks ask whether a maintainer would merge the change. | Fixing real repositories | Oct 2026 | 50.2% | In use |
| FrontierCode Real-repo coding tasks graded for mergeability: correctness, regressions, lint, tests, scope and code quality. Failing any blocker scores zero. | Fixing real repositories | Jun 2026 | 34.3% | In use |
| cognition-golden Cognition tests coding agents on tasks modeled on real Devin use cases, in authentic development environments with fully autonomous grading. A train split serves self-improvement and a test split measures capability. | Building whole projects | Sep 2024 | 51.8% | Not in cards |