New York UniversityWe track 21 benchmarks and 1 arena that New York University made or helped make. Its models are on its lab page.
Top five in each of its 1 arena with three or more rated models, the most rated first.
Models design robot bodies, controllers and strategy for a simulated sumo-style arena. Artifacts fight head-to-head and models get Elo ratings.
Gemini 3.8 Flash 1,223 Official leaderboard
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| GPQA Diamond 198 graduate-level 'Google-proof' multiple-choice questions in biology, chemistry and physics. | Science questions | Nov 2023 | 96.5% | In use |
| BBQ (Bias Benchmark for QA) Ambiguous and disambiguated questions probing social stereotypes across nine attributes. Reports accuracy and bias scores. | Bias, fairness and politics | Oct 2021 |  96.6% | System cards only |
| LiveBench Models answer monthly refreshed questions across math, coding, reasoning, language, instruction following, and data analysis. Scored automatically against objective ground truth. | Other | Jun 2024 | 84.9% | Dropped |
| CVBench Models answer 2,638 manually checked 2D and 3D vision questions built from ADE20K, COCO, and Omni3D. Scoring is accuracy. | Spatial reasoning and counting | Jun 2024 |  88% | Fading |
| CoT-Control Pairs GPQA, HLE, MMLU-Pro, BFCL and SWE-bench problems with instructions on what the chain of thought may contain. Measures compliance. | Deception and alignment | Mar 2026 | 83.7% | System cards only |
| GPQA Main Models answer the 448 question main set of GPQA, graduate level biology, chemistry and physics questions written by experts. Scored by accuracy. | Science questions | Nov 2023 | 80.3% | Dropped |
| VSI-Bench Models watch egocentric videos of 288 indoor scenes and answer over 5,000 questions on object counts, distances, sizes, room layout and route planning. | Spatial reasoning and counting | Dec 2024 |  69.5% | Fading |
| QuALITY Models answer multiple-choice questions about English passages that average about 5,000 tokens. Scoring is accuracy. | Reasoning over long documents | Dec 2021 | 90.5% | Dropped |
| MMVP Models answer 300 questions about 150 image pairs that CLIP embeds as near duplicates. A pair counts only when both answers are right. | Spatial reasoning and counting | Jan 2024 | 87% | Fading |
| SQuALITY Models write question-focused summaries of public-domain short stories. Outputs are scored against human-written reference summaries with ROUGE. | Reasoning over long documents | May 2022 |  24.3% | Dropped |
| MindCube Models answer 21,154 multiple choice questions about scenes seen from a few views. Questions test positions, orientations, and what-if movements. | Spatial reasoning and counting | Jun 2025 | 47.6% | Fading |
| CrowS-Pairs Models choose between paired sentences that differ only in a stereotyped social group. Measures preference for the stereotyped sentence. | Bias, fairness and politics | Sep 2020 | 51.3% | System cards only |
| GPQA Extended Models answer all 546 GPQA questions, graduate level biology, chemistry and physics questions written by experts. Scored by accuracy. | Science questions | Nov 2023 | 48.8% | System cards only |
| WSC Models decide whether a pronoun refers to a given noun phrase in sentences that need commonsense to resolve. Scored by accuracy. | Commonsense | May 2019 | 86.9% | Dropped |
| XNLI Models classify entailment, contradiction or neutral for sentence pairs translated from MultiNLI into 15 languages. | Reasoning in many languages | Sep 2018 |  45.3% | Dropped |
| SuperGLUE Models solve eight English language understanding tasks, including BoolQ, COPA, MultiRC, ReCoRD, RTE, WiC, and WSC. Scored by the average task score. | Language reasoning | May 2019 | 90.4% | Not in cards |
| NYU CTF Bench Agents solve 200 capture the flag challenges from seven years of CSAW competitions. Six categories cover crypto, forensics, pwn, reverse, web and misc. Each runs in Docker. | Capture-the-flag challenges | Jun 2024 | 22% | Not in cards |
| PaintBench Image editing models do precise single-answer edits in 20 operations such as geometric transforms, color changes and symbolic edits. Procedural generation gives deterministic checks. | General visual questions | May 2026 | 16.3% | Not in cards |
| VSI-Bench-Debiased Models answer the 2,362 VSI-Bench questions left after removing 2,768 that text-only models could solve from answer priors or world knowledge. | Spatial reasoning and counting | Nov 2025 | 41.5% | Not in cards |
| BLiMP Language models must assign higher probability to the grammatical sentence in each of 67,000 minimal pairs covering English syntax, morphology and semantics. | Commonsense | Dec 2019 | Not in cards | |
| VSI-SUPER Video models watch arbitrarily long streams and must recall where objects appeared (VSR) or keep counting objects across scenes (VSC). Video lengths run from 10 minutes to 4 hours. | Long videos | Nov 2025 | Not in cards |