Benchmark Almanac

Joel Niklaus

We track 4 benchmarks that Joel Niklaus made or helped make.

0 in use 1 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
LEXam-hard Agents with legal research tools answer the 518 LEXam open questions that strong open models score lowest on. Questions answerable from memory or unanswerable from their text are removed. Law Sep 2026 40.8% Not in cards
Harness Leaderboard Coding-agent harnesses such as Claude Code, Codex and OpenCode run the same open model at the same reasoning effort on 198 tasks from Terminal-Bench 2.1 and DeepSWE v1.1. Scored as the resolve rate averaged across the two benchmarks over three attempts, with any access to the answer counted as a fail. Terminal tasks Nov 2026 Not in cards
LEXTREME Models analyze legal text across languages. Tasks include predicting document labels, identifying legal arguments, and labeling words within documents. Law Jan 2023 Not in cards
Swiss-Judgment-Prediction Models predict whether the Swiss Federal Supreme Court accepts or dismisses a case. Law Oct 2021 Not in cards