Benchmark Almanac

Speko

We track 4 benchmarks that Speko made or helped make.

0 in use 0 with public code 3 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Speko LLM Benchmark for Voice Agents v3 Models use simulated support tools to look up orders and issue refunds. The task score gives partial credit when a model stops early without causing harm. Customer service 94.7% Not in cards
Speko Speech-to-Speech Benchmark (six concierge scenarios) Voice agents book flights, move reservations, and revise budgets during scripted calls. Speko checks whether agents wait for confirmation and use the latest details. Customer service 0.91 Not in cards
Speko Speech-to-Speech Benchmark (10-turn call) Voice agents answer a weather request with a tool, handle an interruption, and recall an earlier detail. A language model grades task success. Customer service Jul 2026 1108% Not in cards
Speko Speech-to-Speech Benchmark (20-turn concierge call) Voice agents try to book a flight, handle a failed booking, and remember a changed date. Eight automated checks score their actions and replies. Customer service Jul 2026 1.00 Not in cards