Benchmark Almanac

Inc.

We track 2 benchmarks and 8 arenas that Inc. made or helped make.

0 in use 2 with public code 10 with a leaderboard here

Arena leaderboards

Top five in each of its 8 arenas with three or more rated models, the most rated first.

Benchmarks

Name Kind Released Best score Status
REAL Agents complete 112 tasks on replicas of 11 websites. State checks grade actions, while a model judge checks retrieved answers against references. Web browsing Apr 2025 41.1% Not in cards
Medical Chronology LLM Benchmark Models extract medical timelines from six synthetic document sets in three independent rounds. Checks compare extracted events with reference timelines and assess semantic fidelity, formatting, date order, and hallucinations. A weighted composite combines these checks with ROUGE and extraction F1. Medicine Apr 2026 88.9 Not in cards