Benchmark Almanac

Harbor

We track 4 benchmarks that Harbor made or helped make.

4 in use 4 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Terminal-Bench 4.0 Agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate. Terminal tasks Aug 2026 64.8% In use
Terminal-Bench 3.0 Agents complete 74 tasks across 7 domains in sandboxed terminal environments. Verifier scripts grade each trial as pass or fail. Terminal tasks Jul 2026 42.7% In use
Terminal-Bench 4.1 The Terminal-Bench 4.0 tasks with hardened verifiers: agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate. Terminal tasks Nov 2026 71.2% In use
Harbor-Index Agents solve about 80 hard tasks drawn from 29 benchmarks run through Harbor adapters, such as HLE, GSO and SWE-bench Pro. Each task keeps its source verifier. Terminal tasks Jun 2026 28.1% In use