Benchmark Almanac

Aider

We track 3 benchmarks that Aider made or helped make.

0 in use 3 with public code 3 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Aider Polyglot Models edit code to solve 225 hard Exercism exercises in C++, Go, Java, JavaScript, Python, and Rust. Scored by the share passing unit tests. Function-level problems Dec 2024 88% Fading
Aider-Edit Models edit Python source files to complete 133 Exercism exercises through Aider. Scored by the share of exercises whose unit tests all pass. Function-level problems Jul 2023 79.7% Dropped
Aider Refactoring Benchmark Models refactor large Python methods while preserving their implementation. The benchmark checks whether edits complete correctly. Fixing real repositories Dec 2023 92.1% Not in cards