Benchmark Almanac

Megagon Labs

We track 3 benchmarks that Megagon Labs made or helped make.

0 in use 3 with public code 3 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
ARCS Models identify ambiguities in a database question and write SQL for the intended interpretation. The scorer executes the query against the database. Data science and SQL Oct 2026 57.9% Not in cards
HoloBench Models answer questions by combining facts across database rows written as sentences. A judge gives full, partial, or zero credit for each expected result. Reasoning over long documents Oct 2024 59.6% Not in cards
CypherBench Models translate questions about Wikidata into Cypher queries. The scorer executes each query and compares its result with the expected answer. Data science and SQL Dec 2024 61.6% Not in cards