Benchmark Almanac

Hamming AI

We track 2 benchmarks that Hamming AI made or helped make.

0 in use 0 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Bug in the Code Stack Models find a syntax error inserted into a long Python codebase. Hamming measures retrieval accuracy across context lengths and error positions. Finding facts in long inputs 82.7% Not in cards
Voice Agent LLM Benchmark Models handle simulated caller requests in text. Hamming grades task completion, policy compliance, and how naturally the models phrase their replies. Customer service Nov 2026 Not in cards