Benchmark Almanac

Quesma

We track 5 benchmarks that Quesma made or helped make.

0 in use 5 with public code 5 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
BabaIsBench Agents play levels of the puzzle game Baba Is You through a text interface and win by rewriting the rules. Models are compared on pass rate, speed and cost. Puzzles and chess Jun 2026 100% Not in cards
BinaryAudit Agents use reverse engineering tools such as Ghidra and radare2 to find backdoors, timebombs and malicious code hidden in compiled binaries of real open-source software. Finding and exploiting real bugs Dec 2025 49.1% Not in cards
CompileBench Agents build real open-source projects through dependency problems, legacy toolchains and complex build systems, for example compiling 22-year-old code. Terminal tasks Aug 2025 100% Not in cards
OTelBench Coding agents add OpenTelemetry tracing, metrics and logging to codebases in several languages. Tests check the instrumentation. Fixing real repositories Jan 2026 29% Not in cards
trivial-prompt-bench Models answer a short request through a terminal agent. Quesma adds the API charge to the estimated cost of the developer's waiting time. Other Jul 2026 $0 Not in cards