Princeton University
We track 22 benchmarks and 1 arena that Princeton University made or helped make. Its models are on its lab page.
3 in use 19 with public code 19 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| SWE-bench Verified 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. | Fixing real repositories | Aug 2024 | 83.5% | In use |
| CharXiv Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. | Documents, charts and text in images | Jun 2024 | 93.5% | In use |
| COLLIE Models generate text that meets compositional constraints at word, sentence, paragraph, and passage level. Scored by the share passing automatic constraint checks. | Following precise instructions | Jul 2023 | 99% | Fading |
| SUNRGBD Models detect objects in about 10,000 indoor RGB-D images and predict their 3D bounding boxes. Scored by average precision. | Detection and captioning | Jun 2015 |  48.3 | Fading |
| CharXiv (DQ) Models answer descriptive questions about charts from arXiv papers, such as titles, axis labels and legend entries. It is the descriptive half of CharXiv. | Documents, charts and text in images | Jun 2024 | 95% | System cards only |
| HELMET Models handle long-context tasks in seven groups including recall, RAG, re-ranking, many-shot learning, long QA and summarization, at set input lengths. | Reasoning over long documents | Oct 2024 | 55.7% | Dropped |
| InterCode-CTF Agents solve 100 picoCTF capture-the-flag challenges in an interactive Bash and Python environment. | Capture-the-flag challenges | Jun 2023 | 98.7% | System cards only |
| SpatialSense VQA Models judge whether a stated spatial relation between two objects holds in an image. The relations were crowdsourced to defeat simple cues. | Spatial reasoning and counting | Aug 2019 | 60% | Dropped |
| HorizonMath Models attack 113 mostly unsolved problems in eight areas of mathematics and the mathematical sciences. A program checks each proposed closed form, construction or bound. | Research mathematics | Mar 2026 | 11.5% | In use |
| SORRY-Bench Models get unsafe requests across a fine-grained taxonomy of topics and linguistic variations. A judge model scores whether each answer refuses or complies. | Refusing harmful requests | Jun 2024 | 5.23% | Not in cards |
| CEO-Bench Agents steer a simulated AI startup for 500 days, gathering noisy information and adapting plans. The outcome of the simulated company is the score. | Multi-agent games | Jun 2026 | $22,148,357 | Not in cards |
| AlgoTune Agents rewrite code for 154 math, physics, and computer science functions to run faster and stay correct. Scoring is harmonic mean speedup over reference code. | Speed and optimization | Jul 2025 | 2.05× | Not in cards |
| CORE-Bench Agents reproduce the results of published studies from their code and data. 270 tasks from 90 papers in computer science, social science and medicine come in three difficulty levels. | Research science | Sep 2024 | 77.8% | Not in cards |
| SWE-bench Lite Models resolve 300 self-contained GitHub issues chosen from SWE-bench to cut evaluation cost. Hidden repository tests decide each patch. | Fixing real repositories | Mar 2024 | 60.3% | Not in cards |
| AutoLab Agents start from a correct but weak baseline and improve it within a fixed wall-clock budget by measuring, editing and measuring again. The 36 tasks span system optimization, puzzles, model development and CUDA kernels, such as scaling-law fitting, radix sort and FlashAttention. Each task scores the measured improvement, averaged over three runs. | Machine learning engineering | Jun 2026 | 0.68 | Not in cards |
| SWE-bench Models resolve 2,294 real GitHub issues from 12 Python repositories. A patch counts as resolved when the hidden repository tests pass. | Fixing real repositories | Oct 2023 | 52.6% | Not in cards |
| DiG-bench Agents play text-only games whose transformation rules and win conditions are hidden, so they must discover the rules by experimenting. Every game was beaten by at least one human on a first attempt. | Puzzles and chess | Aug 2026 | 71.4% | Not in cards |
| FutureX Agents predict upcoming events collected daily from 195 websites in 11 domains, within a one-week window. Answers are scored once real results are crawled, with harder tiers weighted more. | Real-world events | Aug 2025 | 27.9 | Not in cards |
| τ^τ-bench Coding agent builds a customer-service agent from business records, a client and an API. The built agent is scored against held-out simulated users. | Building whole projects | Sep 2026 | 23.9% | Not in cards |
| USACO Models solve 307 USA Computing Olympiad problems. Hidden unit tests check correctness and efficiency, scored as pass@1. | Competitive programming | Apr 2024 | 2.61% | Not in cards |
| AQuA Two LLM research loops (factor discovery, model development) reuse evidence from their own past experiments inside sealed sandboxes. Scored by IC and held-out Sharpe. | AI research tasks | Aug 2026 | Not in cards | |
| WebShop Agents search and click through a simulated shopping site with 1.18 million real products to buy an item that matches a text instruction. Reward reflects how well the purchase matches. | Web browsing | Jul 2022 | Not in cards |
Arenas
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| PokeAgent Challenge Agents compete in Pokemon battles under partial observability in a Battling Track and play the Pokemon RPG in a Speedrunning Track that needs long-horizon planning. | Video games | Mar 2026 | Not in cards |