Benchmark Almanac

Ohio State University

We track 8 benchmarks that Ohio State University made or helped make.

0 in use 8 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Online-Mind2Web Web agents do 300 tasks on 136 live websites. An LLM judge reads the agent's trajectory and decides whether the task succeeded. Web browsing Apr 2025 69% Fading
Mind2Web Models predict the next web action for over 2,000 crowdsourced tasks on 137 real websites in 31 domains. Scored by element accuracy and step success rate. Web browsing Jun 2023 55.8% Dropped
Realtime-Bench Agents act in 50 live game and web tasks where a late action fails: 30 web tasks (travel, shopping, services and security) and 20 games. The environment keeps moving while the model thinks. Three clocks separate speed from skill: Wall Clock runs on real time, Paused Clock stops while the model thinks, and Token Clock advances with each generated token. Scores are task success rates over three runs. Web browsing Oct 2026 42.7% Not in cards
ScienceAgentBench Agents write programs for 102 data-driven discovery tasks from 44 papers in four disciplines. Outputs are checked by execution and task-specific metrics. Research science Oct 2024 42.2% Not in cards
Mind2Web 2 Agents browse the live web for 130 long tasks and return answers with citations. Task-specific judge agents built from rubric trees grade correctness and attribution. Deep research Jun 2025 28% Not in cards
GrailQA Models answer questions by composing queries over Freebase. The evaluator measures semantic logical-form exact match and answer-set F1 across ordinary, compositional, and unseen-schema generalization settings. Language reasoning Nov 2020 Not in cards
TravelPlanner Agents plan multi-day trips from a user request, using tools to search nearly four million records of flights, hotels, restaurants and attractions. Each plan must meet the user's hard constraints, such as budget, and commonsense rules. The headline final pass rate counts a plan only when it meets every constraint. General assistants Feb 2024 Not in cards
WebGuard Models classify the risk of a web action using human labels for safe, low-risk, and high-risk actions. Web browsing Jul 2025 Not in cards