Kept fresh
Benchmarks that add new items on a schedule or draw new ones every time. We count 369 benchmarks here, 241 of them arenas. A benchmark belongs here when refresh is new items on a schedule or live.
How fast it grows
76 were released since July, more than the 65 in the first half of 2026.
How much is still open
No model has passed 85% on 41 of the 52 benchmarks here that score in percent, 79% of them.
Every benchmark in this view
Newest first. Facets come from each benchmark’s own documents, and our methodology says how we tag them.
| Benchmark | Level | Released | Best score | Status |
|---|---|---|---|---|
| 3D GameDev Arena Agents build playable 3D browser games from specifications and reference images. Players compare anonymous game pairs. The authors also test functionality and playability. | Expert | Nov 2026 | 99.8% | Not in cards |
| AssemblyAI Speech Benchmarks Speech recognition systems transcribe clean, accented, multilingual and code-switched audio, identify entities and diarize speakers. Error rates compare production systems across public speech sets. | Expert | Nov 2026 | | Not in cards |
| Beyond Survival Models play social deduction games with human-aligned strategies and interaction patterns. | Everyday | Nov 2026 | 58.7% | Not in cards |
| Clocktower Bench Models play the social deduction game Blood on the Clocktower (the Trouble Brewing script) against each other, with a scripted Storyteller. One model takes every good seat and another every evil seat, so each game tests one model's detection of the Demon against another's deception. A Bradley–Terry fit turns 386 games into Elo ratings. | Everyday | Nov 2026 | 1,859 | Not in cards |
| Crazyrouter AI Trading Arena Models buy, sell, or hold three cryptocurrencies every five minutes using simulated accounts. The grader values portfolios at live quotes and deducts trading fees. | University | Nov 2026 | -7.45% | Not in cards |
| CREF Models submit forecasts before the target period begins and receive scores after the actual values become available. | University | Nov 2026 | Not in cards | |
| IdeaTaste Agents propose research ideas from literature frozen at 2025. Evaluation compares their citations and ideas with 2026 publications and measures diversity. | Research frontier | Nov 2026 | Not in cards | |
| KradleBench Models play live multi-agent Minecraft games and act by writing code. The overall score weights collaboration, economy and conflict at 30% each, and logic and creativity at 5% each. | Everyday | Nov 2026 | 78.3 | Not in cards |
| MindGames Agents compete in Colonel Blotto, Iterated Prisoner’s Dilemma, Codenames, and Secret Mafia. | Everyday | Nov 2026 | Not in cards | |
| Moneda GroundBench Models answer live questions about market figures. Moneda’s default board reports the share of correct answers against reference data, with incorrect answers reported separately. | Expert | Nov 2026 | Not in cards | |
| Ramen Bench Agents draw an interactive bowl of vegan ramen in one HTML file. Static checks verify the constraints and visitors rate the rendered pages. | University | Nov 2026 | 5 | Not in cards |
| RoboColosseum Physical robots execute fresh instructions in paired blind trials. Human evaluators choose the better run or a tie, and comparisons determine ratings for each embodiment. | Everyday | Nov 2026 | Not in cards | |
| RSLiveBench-UAV Models answer questions about vehicles in drone images. The benchmark checks perception and spatial reasoning across different views and adds newly captured scenes. | Everyday | Nov 2026 | 59.9 | Not in cards |
| Agents' Last Exam v1.1 Agents complete 122 public long-horizon professional workflow tasks across 55 sub-industries. Deterministic graders check outcomes against hidden references. | Expert | Oct 2026 | 47.5% | In use |
| Alignment Index (Arena) Arena ranks agent models on failure signals mined from real Agent Arena sessions, such as unauthorized actions, false attribution and deceptive completion. | Everyday | Oct 2026 | 87.9 | Not in cards |
| BallotBench Nine frontier agents forecast 863 contests in the 2026 US midterms each day until polls close. Proper scoring rules grade forecasts against certified results, beside Kalshi, Polymarket and expert ratings. | Expert | Oct 2026 | | Not in cards |
| CivBench Models play full games of Civilization V with the Vox Populi mod against each other on matched maps and seeds. Pairwise skill ratings rank long-term strategy. | Expert | Oct 2026 | 1,634 | Not in cards |
| Hanabi Agent Benchmark Models play cooperative Hanabi in four-player teams. The benchmark scores shared board progress while hints and lives constrain coordination. | Everyday | Oct 2026 | 20.5 | Not in cards |
| Human or AI? (Skills IL) Models write short Hebrew texts on the same assignments as human authors. Readers guess who wrote each text. | Everyday | Oct 2026 | | Not in cards |
| Kaggle ASCII Arena Models draw images using text characters. Voters compare the drawings. | Everyday | Oct 2026 | | Not in cards |
| Kaggle Explode Arena Models build interactive exploded views of objects in three.js. A submission must render a self contained page where the viewer can separate the parts. Votes determine the arena rating. | University | Oct 2026 | 1,712 | Not in cards |
| Kaggle Landmark Arena Models recreate named landmarks in three.js. Each submission must render as a self contained web page. Votes determine the arena rating. | University | Oct 2026 | 1,788 | Not in cards |
| Kaggle Pelican Arena Models draw a pelican riding a bicycle in SVG. The arena checks whether each drawing renders and asks people to compare the passing entries. | University | Oct 2026 | | Not in cards |
| Kaggle Sort Viz Arena Models use Manim to animate sorting algorithms and explain their time complexity. A submission must produce a valid video under the file size limit. Votes determine the arena rating. | University | Oct 2026 | 1,650 | Not in cards |
| Kaggle SVG Arena Models draw SVG images from 120 prompts across twelve categories. Submissions must render and satisfy the drawing rule for the prompt. Votes determine the arena rating. | University | Oct 2026 | 1,699 | Not in cards |
| LiveMACE Agents trade in live financial markets, and the benchmark scores the process capabilities behind their decisions, not only returns. | Expert | Oct 2026 | Not in cards | |
| LiveSearch Agents search for current financial figures. The evaluator runs a stored query against a professional data service to update the reference answer. | University | Oct 2026 | | Not in cards |
| micro1 Medical Index Models are scored on the equal-weight average of micro1 medical benchmarks, currently RealmPathology, PharmaBench and WellnessBench. | Expert | Oct 2026 | 71% | Not in cards |
| Minecraft Bed Wars (YC Labs) Teams of agents play Minecraft Bed Wars against one another. The researchers compare the number of matches each team wins. | Everyday | Oct 2026 | Not in cards | |
| Rotom Champ Bench Models draft six level 50 Pokémon, select three for each match, and choose their moves. The organizer rates them on wins under Champions singles rules. | Competition | Oct 2026 | | Not in cards |
| Rotom Gen9 Bench Models draft six level 100 Pokémon and play the whole team in Generation 9 OverUsed singles. Match results determine their ratings. | Competition | Oct 2026 | 2,318 | Not in cards |
| SettlerBench Season One Four models trade resources, build settlements, and negotiate in public or private messages. Final victory points determine pairwise results for the Bradley-Terry rating. | Everyday | Oct 2026 | 1,270 | Not in cards |
| SpireClimb StS2 A10 Agents play Slay the Spire 2 at Ascension 10 through screenshots and mouse and keyboard, 15 runs per character. Scoring is the win rate. | Everyday | Oct 2026 | 49.3% | Not in cards |
| TavernBench Agents build a Hearthstone Battlegrounds player through research, simulator updates, and live games. | Expert | Oct 2026 | 95.6 | Not in cards |
| Turn based Benchmarks Models play turn based games including tic tac toe and rock paper scissors. The tasks evaluate game decisions and adaptation to opponents. | Everyday | Oct 2026 | | Not in cards |
| ArtifactArena Models design robot bodies, controllers and strategy for a simulated sumo-style arena. Artifacts fight head-to-head and models get Elo ratings. | Expert | Sep 2026 | 1,539 | Not in cards |
| Bench on the Clocktower Ten models play the social deduction game Blood on the Clocktower together, one seat each, over 300 ten-player games of the Trouble Brewing script. Each model plays 210 games on the good team and 90 on the evil team. The score is the share of games its team won. | Everyday | Sep 2026 | 58% | Not in cards |
| Brood War Bench Coding agents play StarCraft: Brood War against each other in real time. Systems are ranked by win rate over matches, with cost per game. | Expert | Sep 2026 | 100% | Not in cards |
| CivBench (Civilization VI) Agents play Civilization VI and are evaluated on strategic progress in a long-running world. | Expert | Sep 2026 | | Not in cards |
| fAIntasy Football Models draft NFL players, set lineups, bid for free agents, and trade under the same rules. The standings show wins, fantasy points, and lineup efficiency. | Everyday | Sep 2026 | | Not in cards |
| Game Arena Go Language models play Go against each other through the Game Arena text harness. Ratings come from head-to-head games. | Expert | Sep 2026 | 1,877 | Not in cards |
| HappyWorld Embodied The HappyWorld arena publishes a separate embodied vote pool and Elo ranking. | Everyday | Sep 2026 | | Not in cards |
| HappyWorld Spatial Reconstruction The HappyWorld arena publishes a separate spatial reconstruction vote pool and Elo ranking. | Everyday | Sep 2026 | 1,252 | Not in cards |
| Large Language Model Football League Models draft, trade, and set weekly NFL lineups. Each team earns one result against its opponent and another against the league median. | Everyday | Sep 2026 | Not in cards | |
| LLM Fantasy Football Arena Guillotine Models draft NFL players and bid for free agents. Each week the lowest-scoring team leaves, and the remaining teams bid on its players. | Everyday | Sep 2026 | | Not in cards |
| Ludobench Backgammon Models play backgammon against opponents using dice rolls and legal moves. Ludobench publishes Glicko-2 ratings and match replays. | Everyday | Sep 2026 | | Not in cards |
| Ludobench Hokm Models play Hokm in partnerships. One player names trump, and partners try to take seven tricks before the other team. | Everyday | Sep 2026 | 100% | Not in cards |
| Multi-Agent Arena Codenames Two teams of anonymized models play the word game Codenames. The board rates the clue giver and the guessers on separate rating scales. | Everyday | Sep 2026 | 1,535 | Not in cards |
| Multi-Agent Arena Diplomacy Seven players, people or anonymized models, negotiate and fight over World War I Europe. Models are ranked by their mean share of the sum-of-squares score, where 14.3 is an even split. | Everyday | Sep 2026 | 37.5% | Not in cards |
| RSI Arena Ten frontier agents each post-train the same Nemotron base model on a shared GPU cluster for six days. People then rank the resulting models in blind side-by-side votes, and agents retrain on that feedback. | Research frontier | Sep 2026 | 1,095 | Not in cards |
| Xbench Measures firsthand X posts about frontier models and coding harnesses. An LLM labels each post's sentiment, aggregated per model per week. | Sep 2026 | 75.7 | Not in cards | |
| Agent-Only Fantasy Football League Models draft NFL players, set lineups, and negotiate trades using the same prompt and tools. The organizer compares points, lineup efficiency, and cost per point. | Everyday | Aug 2026 | 1371.4 | Not in cards |
| Game Arena Nine Men's Morris Language models play Nine Men's Morris. Forming a row of three, called a mill, lets a player remove one of the opponent's pieces. | Everyday | Aug 2026 | | Not in cards |
| Game Arena Reversi Language models play Reversi, also called Othello, flipping the opponent's disks that they flank in a line. | Everyday | Aug 2026 | | Not in cards |
| Game Arena Word Art Language models take turns drawing a word as ASCII art and guessing what the other model drew. | Everyday | Aug 2026 | | Not in cards |
| LLM Fantasy Football Arena Classic Models manage two NFL fantasy teams each. One league allows trades and messages. The other disables both. Each league scores its own draft and season. | Everyday | Aug 2026 | | Not in cards |
| MineAmongUs Vision-language agents play an embodied Among Us game with verbal and nonverbal deception. | Everyday | Aug 2026 | | Not in cards |
| Speech Agent Arena People hold live conversations with two hidden speech-to-speech agents on the same scenario, some with tool calls, then pick the one they preferred. Votes are fit to an Elo rating, and the arena also tracks task success. | Everyday | Aug 2026 | 1,070 | In use |
| Agentic Trader Agents search for market information and place simulated foreign exchange orders using live prices. Each agent manages its own positions, cash, and margin. | University | Jul 2026 | | Not in cards |
| AI Battle Arena Models play poker, Gomoku, and Connect Four through the same interface. The overall leaderboard averages each model’s finishing position across five head-to-head contests. | Everyday | Jul 2026 | 88.0 | Not in cards |
| AI Security Leaderboard Models face systematic jailbreak attacks across chemical, biological, radiological, nuclear, explosive and cyber domains. Scores count universal jailbreaks and estimate attacker cost. | Expert | Jul 2026 | $60 | Not in cards |
| Arena Search Factuality Arena combines user preferences with independent checks of verifiable claims in model responses. | Everyday | Jul 2026 | 1,218 | Not in cards |
| Artificial Turf War Models bid for draft position, pick NFL players, and submit weekly lineups. The league ranks teams by wins and breaks ties with fantasy points. | Everyday | Jul 2026 | 6.00 | Not in cards |
| Bot Analysis Arena Models write trading rules for simulated US stock accounts. Some agents revise their rules after trades. Software tests each revision against the current rules before adoption. | University | Jul 2026 | Not in cards | |
| Founder Bench Agents run four matched businesses per model through the acoco platform. The evaluation tracks real advertising, customer attention, and website traffic after seven days. Its headline customer-impressions metric adds ad impressions to organic site visits. | Expert | Jul 2026 | 127902.0 | Not in cards |
| Game Arena Bargaining Two language models negotiate how to split a pool of items, each with private item values. Each side tries to reach a good deal without giving away its values. | Everyday | Jul 2026 | | Not in cards |
| Game Arena Coin Game Two language models move on a grid and pick up coins. Each should collect the coins that score for it and avoid the rest. | Everyday | Jul 2026 | 514 | Not in cards |
| Game Arena Lines of Action Language models play Lines of Action, where a piece moves as far as the number of pieces on its line. The goal is to join all of one's pieces into one group. | Everyday | Jul 2026 | | Not in cards |
| Game Arena Ultimate Tic-Tac-Toe Language models play nine linked tic-tac-toe boards. Each move decides which small board the opponent must play next. | Everyday | Jul 2026 | | Not in cards |
| LMArcade Agents build browser games from the same prompt. Players try two games and vote before learning which models and coding tools made them. | University | Jul 2026 | 1,798 | Not in cards |
| Multi-Agent Arena Risk Four commanders, people or anonymized models, trade sealed messages and then attack for territory on a 42-territory map. Models are ranked by their mean share of territory, where 25 is an even split. | Everyday | Jul 2026 | 38.8% | Not in cards |
| Multi-Agent Arena Social Poker No-limit hold’em tournaments with table talk, where people and anonymized models share a table. Models are ranked by a rating fit on who finished ahead at each table. | Everyday | Jul 2026 | 1,553 | Not in cards |
| Olam Labs Multi-Agent Arena People and anonymized models share a table in social strategy games: Diplomacy, Social Poker, Codenames and Risk. One rating combines every game, fit on who finished ahead of whom at each table. | Everyday | Jul 2026 | 1,544 | Not in cards |
| ParliamentBench Models play Secret Hitler while deceiving, persuading, and reasoning under hidden roles. | Everyday | Jul 2026 | 81% | Not in cards |
| PatientAgentBench Patient-facing health agents talk with a simulated patient and use a sandbox of healthcare tools for scheduling, prescriptions, telehealth and escalation. An LLM jury scores over a hundred clinician-grounded criteria. | Expert | Jul 2026 | 4.25 | Not in cards |
| Real World VoiceEQ Voice systems recognize, produce and respond to acoustic information across speech recognition, speech to speech, speech understanding and text to speech tasks. | Expert | Jul 2026 | | Not in cards |
| Agent Arena Arena ranks models as the orchestrator in real Agent Mode sessions on arena.ai. Scores use behavioral signals such as file downloads, disapproval events, retries and steerability. | Everyday | Jun 2026 | 28.6% | In use |
| Agents' Last Exam 1,500+ expert-sourced tasks from real projects across 55 occupations in three difficulty tiers. Objective verifiers grade deliverables. 5-hour cap. | Expert | Jun 2026 | 38.2% | In use |
| BTF-3 Agents forecast already-resolved binary and numeric questions while researching a frozen snapshot of the web. Scoring pools Brier and ranked probability scores. | Expert | Jun 2026 | 0.12 | Not in cards |
| Game Arena Checkers Language models play American checkers against each other, with forced jumps and crowned kings. Ratings come from head-to-head games. | Everyday | Jun 2026 | | Not in cards |
| Game Arena Clobber Language models play Clobber, where every move captures an adjacent enemy stone. The player who makes the last legal move wins. | Everyday | Jun 2026 | | Not in cards |
| Game Arena Dark Hex Language models play Hex without seeing the opponent's stones, so they must probe cells to learn the hidden board while building a path. | Everyday | Jun 2026 | | Not in cards |
| Game Arena Dots and Boxes Language models take turns drawing lines between dots on a grid. Closing the fourth side of a box claims it. | Everyday | Jun 2026 | 1,393 | Not in cards |
| Guesswork (The Aggregate) Models predict benchmark scores before publication. The maker ranks forecasts by mean absolute error after scaling each error by the variation in that benchmark’s scores. | University | Jun 2026 | 0.56 | Not in cards |
| LLM SoccerArena World Cup 2026 Models forecast World Cup match scores and outcome probabilities before kickoff. The organizers compare forecasts with match results under different prompts and web access rules. | Everyday | Jun 2026 | 168.0 | Not in cards |
| OmniGameArena Models play games while response latency advances the game clock. | Everyday | Jun 2026 | 0.47 | Not in cards |
| Poker Arena Models play no-limit Texas Hold’em across repeated sessions. The benchmark scores tournament chips and nine strategic dimensions, including memory and bluffing. | Everyday | Jun 2026 | 15730.0 | Not in cards |
| Questflow Traders Arena Agents and people trade on Hyperliquid and Polymarket in a public competition. The leaderboard reports weekly portfolio returns alongside gains and trading volume. | University | Jun 2026 | Not in cards | |
| RTSGameBench Agents play the open real-time strategy game Beyond All Reason. | Expert | Jun 2026 | Not in cards | |
| 3DCodeArena Models generate 3D assets through code. Human preference votes determine the published Bradley-Terry Elo rating. | Expert | May 2026 | 1,230 | Not in cards |
| Agent Island Models play multi-agent games that require cooperation, conflict, and persuasion. | Everyday | May 2026 | 0.88 | Not in cards |
| BoardGameBench (Homer Quan) Models play seven board game variants against a search engine. The scorer rewards quick wins and gives partial credit for longer losses. | Everyday | May 2026 | | Not in cards |
| Cattle Trade Agents play a long economic game with auctions, hidden offers, bargaining, bluffing, and resource limits. | Everyday | May 2026 | Not in cards | |
| euchre-bench Coding agents build a euchre bot and test it against four opponents. Each opponent contributes up to 100 wins to the total. | University | May 2026 | 140.0 | Not in cards |
| Game Arena Word Association Teams of language models give one-word clues and guess linked words, in a game like Codenames, against an opposing team. | Everyday | May 2026 | | Not in cards |
| NewsBench (Forum AI) Models answer questions about current events. Judges calibrated against expert ratings score factual accuracy, neutrality, and source quality. | Everyday | May 2026 | 88.3% | Not in cards |
| Prompt injection live bug bounty (Anthropic with Gray Swan) Expert red teamers submit prompt injection attacks against hidden frontier models in tool use, coding and computer use scenarios. Score is attack success rate. | Expert | May 2026 | 0.08% | System cards only |
| PTCG-Bench Agents play the Pokémon Trading Card Game by choosing legal actions from a partially visible board. Match results determine their Glicko-2 ratings. | School | May 2026 | | Not in cards |
| SIGHT (Search and Intent Gauge for Hospitality and Tourism) Agents recommend restaurants, hotels and attractions from user preferences and search results. Evaluators check constraints, evidence and privacy. | Everyday | May 2026 | Not in cards | |
| ArXivLean Models prove or disprove theorems from recent arXiv papers, stated in Lean 4. A strict comparator checks each submitted proof against the fixed reference statement. | Research frontier | Apr 2026 | 79.3% | System cards only |
| Clash of AIs Season 1 Models publish trade entries, targets, and stops before prices move. Clash of AIs simulates fills against market prices and ranks accounts by total equity. | University | Apr 2026 | $283,903 | Not in cards |
| Claw-Eval-Live Agents complete 105 workflow tasks in 17 families drawn from live ClawHub workflow signals, scored on both the workflow and the final workspace state. | University | Apr 2026 | 66.7% | Not in cards |
| Cooperate to Compete (C2C) Multi-player conquest game with private non-binding negotiation and secret asymmetric objectives. LM agents and humans are ranked by Plackett-Luce player strength from their game results. | Everyday | Apr 2026 | | Not in cards |
| Game Arena Five in a Row Language models play a gomoku-style game and try to line up five of their tokens before the opponent does. | Everyday | Apr 2026 | | Not in cards |
| GomokuBench Models place stones on a 19 by 19 board against a search engine. The score rewards wins, quicker wins, and longer losses. | Everyday | Apr 2026 | | Not in cards |
| Image-to-WebDev Arena Users upload a screenshot, wireframe or mockup, two anonymous models build the website in Code Arena, and users vote for the better result. Votes become ratings with bootstrapped confidence intervals. | Everyday | Apr 2026 | 1,756 | Not in cards |
| LiveMathematicianBench Models answer multiple choice questions about theorems from arXiv papers published after their training cutoff. Wrong options follow plausible but invalid proof sketches. | Research frontier | Apr 2026 | | System cards only |
| Sparse Fourier Recovery Models reconstruct a sparse one-dimensional signal from incomplete Fourier measurements. A reference solver provides a comparison for reconstruction accuracy. | Expert | Apr 2026 | 0.36 | Not in cards |
| BrokenArXiv Models are asked to prove statements from recent arXiv papers that were changed to be plausible but false. Full credit goes to models that say the statement is false. | Research frontier | Mar 2026 | | In use |
| Clocktower Radio Models play the social deduction game Blood on the Clocktower (the Trouble Brewing script) in mirrored pairs of games. In each match one model plays good and the other evil, and then they swap. A coded Storyteller runs the game, and a Bradley–Terry fit over all matches gives ratings where 1500 is average. | Everyday | Mar 2026 | 1,708 | Not in cards |
| Document Arena People upload their own PDFs with a question, compare answers from two anonymous models, and vote for the better one. Votes become Bradley-Terry ratings. | Everyday | Mar 2026 | 1,500 | Not in cards |
| Echo General AI Prediction Leaderboard Models forecast unresolved events drawn from prediction markets, synthesized news questions and expert-written questions. Forecasts made at aligned times are compared in pairwise battles, and an Elo fit ranks the models. | Expert | Mar 2026 | | Not in cards |
| Gray Swan IPI Arena benchmark Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. | Mar 2026 | | In use | |
| GTO Wizard Benchmark Models play heads-up no-limit Texas Hold’em against GTO Wizard AI. AIVAT reduces variance while the benchmark reports poker winnings. | Expert | Mar 2026 | -9.23 | Not in cards |
| InteractiveBench Texas Hold’em Models play no-limit Texas Hold’em from partial observations of cards, chip stacks, and recent actions. Repeated matches measure chip gains, with invalid actions receiving one retry before an automatic fold. | Competition | Mar 2026 | | Not in cards |
| InteractiveBench Trust Game Models choose whether to cooperate or defect in repeated prisoner’s dilemma matches. A round-robin tournament scores average realized payoff per round, with a random stopping rule. | Everyday | Mar 2026 | | Not in cards |
| Monthly-SWEBench Coding agents resolve 100 fresh GitHub tasks each month, half bug fixes and half features, from newly merged pull requests. Harbor verifiers score pass@1. | Expert | Mar 2026 | 76% | Not in cards |
| PokeAgent Challenge Agents compete in Pokemon battles under partial observability in a Battling Track and play the Pokemon RPG in a Speedrunning Track that needs long-horizon planning. | Expert | Mar 2026 | Not in cards | |
| Slop Arena Site collecting writing-preference data to tune slop detection to a user's taste, paired with the slop-guard tool. | Mar 2026 | Not in cards | ||
| ArXivMath Final-answer research-level math problems extracted monthly from recent arXiv papers and filtered to be self-contained. | Research frontier | Feb 2026 | | In use |
| Code Review Bench (Martian) AI code review tools review pull requests from five large open-source projects. An LLM judge matches their comments to human-verified golden comments. | Expert | Feb 2026 | Not in cards | |
| FutureEval (Metaculus) Models forecast open Metaculus questions with a fixed prompt and are scored as the questions resolve. Seasonal bot tournaments and Pro Forecaster baselines run alongside. | Expert | Feb 2026 | Not in cards | |
| Game Arena Chess Openings Language models play chess against each other from varied two-move openings sampled from 20 popular human openings. The board is given as text. | Expert | Feb 2026 | | Not in cards |
| Game Arena Four in a Row Language models play a Connect Four style game and try to line up four of their tokens before the opponent does. | Everyday | Feb 2026 | | Not in cards |
| Game Arena Poker Language models play heads-up no-limit Texas Hold'em against each other. Models are ranked by mean big blinds won per 100 hands. | Expert | Feb 2026 | 30.3 | Not in cards |
| Game Arena Werewolf Teams of language models play the social deduction game Werewolf in natural language. Villagers must find the hidden werewolves, who try to deceive them. | Expert | Feb 2026 | 0 | Not in cards |
| ProxyWar Models write bots for nine games. The harness tests each bot and runs tournaments to measure wins, errors, and decision time. | University | Feb 2026 | | Not in cards |
| Wiki Live Challenge Deep research agents write articles on topics of 100 recent Wikipedia Good Articles, which serve as expert references. Scored on 39 writing criteria and factual verifiability. | Expert | Feb 2026 | 53.6 | Not in cards |
| Agent Poker Bench Models play 10-seat no-limit Texas Hold'em tournaments against each other. Chip counts and elimination order feed TrueSkill ratings. | Expert | Jan 2026 | 1,132 | Not in cards |
| Buyout Game Eight language models play a money-driven elimination game with private transfers and a buyout endgame. | Everyday | Jan 2026 | 1975.4 | Not in cards |
| KradleVerse Agent harnesses compete in a live Minecraft arena. | Everyday | Jan 2026 | Not in cards | |
| LMArena Image-to-Video Arena Video generation models animate the same input image with the same prompt, anonymously and side by side. Votes give Elo-style ratings. | Everyday | Jan 2026 | | Not in cards |
| LMArena Text-to-Video Arena Video generation models make clips from the same text prompt, anonymously and side by side. Votes from logged-in users give Elo-style ratings. | Everyday | Jan 2026 | | Not in cards |
| mage-bench Models play Magic: The Gathering in a benchmark that compares strategic play across games. | Expert | Jan 2026 | 1617.0 | Not in cards |
| MegaGem Three models bid in sealed auctions with private cards. The benchmark rotates starting hands and fits an Elo-scaled rating to the order in which players finish. | Everyday | Jan 2026 | Not in cards | |
| MineBench Models turn a prompt into a Minecraft-style voxel build as raw 3D coordinates or tool calls. Users compare builds blind, and votes become rankings. | Everyday | Jan 2026 | 2,169 | Not in cards |
| Prediction Arena Agents research events and trade contracts on Kalshi and Polymarket. The leaderboard ranks accounts by cash plus the market value of open positions. | University | Jan 2026 | $138,945 | Not in cards |
| Spyfall study Models play Spyfall while inferring hidden locations and managing deceptive dialogue. | Everyday | Jan 2026 | | Not in cards |
| Strategy Bench Models compete in six social deduction and strategy game environments. | Everyday | Jan 2026 | Not in cards | |
| TetrisBench Models play Tetris head-to-head in the original benchmark. The Harbor strategy track grades an agent-written JavaScript placement function on 20 fixed piece sequences. | Everyday | Jan 2026 | 66.7% | Not in cards |
| TradeRank Arena Models choose positions in cryptocurrencies and US stocks using shared market data and simulated capital. The leaderboard ranks portfolios by return. | University | Jan 2026 | 0% | Not in cards |
| Encyclo-K Models judge questions composed at test time from knowledge statements extracted from authoritative textbooks. Random composition is meant to resist contamination and test several facts at once. | University | Dec 2025 | 62.1% | Fading |
| Pieverse Agent Prediction Arena Agents trade event contracts on Polymarket with $1,000 each in real money. Pieverse tracks account value as market prices change. | University | Dec 2025 | $981 | Not in cards |
| WOLF Models play Werewolf to measure deception production and deception detection as separate abilities. | Everyday | Dec 2025 | Not in cards | |
| Arena Expert Experts compare model answers to questions drawn from their professional work. Arena ranks models from their votes. | Expert | Nov 2025 | | Not in cards |
| BigLaw Bench: Arena Legal experts compare two answers to a legal task. The benchmark reports pairwise win percentages. | Expert | Nov 2025 | Not in cards | |
| Code Arena Models act as coding agents that build web apps from user prompts, and users vote between two results. Votes yield Elo-style ratings. | Nov 2025 | 1,826 | In use | |
| CodeClash Models build and improve their own codebases over many rounds, then compete in arenas such as Halite, poker and BattleSnake. Results give Elo ratings. | Expert | Nov 2025 | 1,385 | Not in cards |
| OCR Arena Users upload a document, compare two anonymous OCR or vision-language models side by side, and vote. Votes become Elo ratings. | Everyday | Nov 2025 | | Not in cards |
| Vending-Bench Arena Several agents each run a vending machine at the same location in the Vending-Bench 2 environment. They can email, pay and trade with each other, but each is scored on its own money balance. | Expert | Nov 2025 | | Not in cards |
| Alpha Arena Frontier models trade live markets with real money over a season, and the leaderboard ranks them by returns. | Expert | Oct 2025 | Not in cards | |
| BigCodeArena Users ask for code, both models' programs run in a sandbox, and users try the results before voting for the better answer. Votes give Elo ratings. | Everyday | Oct 2025 | | Not in cards |
| CATArena Code agents write programs that play board and card games in repeated tournaments. Between rounds they refine their code through self-reflection and by studying rivals' code. Scores are average ranks that separate first-round skill from improvement. | Expert | Oct 2025 | 1,799 | Not in cards |
| CATArena Gomoku Code agents compete in Gomoku. The public competition board reports Elo Rating for each agent configuration. | Expert | Oct 2025 | 1,799 | Not in cards |
| CATArena Pommerman Code agents compete in Pommerman. The public competition board reports Elo Rating for each agent configuration. | Expert | Oct 2025 | Not in cards | |
| CATArena Texas Holdem Code agents compete in Texas Holdem. The public competition board reports Elo Rating for each agent configuration. | Expert | Oct 2025 | 1,463 | Not in cards |
| Epoch Capabilities Index An index that fits model capability and benchmark difficulty from scores on over 50 benchmarks in Epoch AI's hub, using item response theory. Scores read like Elo ratings. | Research frontier | Oct 2025 | 174.0 | System cards only |
| HLE-Rolling A dynamic fork of Humanity's Last Exam where the makers remove, fix and re-add questions over time and publish an update log. | Research frontier | Oct 2025 | Not in cards | |
| Magentic Marketplace Agents search for sellers, exchange proposals and make purchases. The simulation measures how much value customers receive after paying for their purchases. | Everyday | Oct 2025 | Not in cards | |
| Prophet Arena Models and agents forecast open Kalshi prediction-market events and are scored once the events resolve. The live board ranks a difficulty-adjusted Arena score and also reports calibration error and market return. | Expert | Oct 2025 | 86.2 | In use |
| Real-time Audio-Visual Interaction Evaluation Trained conversationalists hold three minute live video calls with a model. Real users rate each session for naturalness and fluency on a 0 to 3 scale. | Everyday | Oct 2025 | | System cards only |
| HUMAINE Prolific participants, stratified across 22 demographic groups in the US and UK, hold multi-turn conversations with two anonymous models on topics they choose. They compare the models overall and on dimensions such as trust, ethics and safety. A hierarchical Bradley-Terry-Davidson model turns the votes into ratings, adjusted to census data. | Everyday | Sep 2025 | | Not in cards |
| Project Euler (MathArena) Models solve recent Project Euler problems that need both mathematical insight and code. MathArena adds problems as Project Euler releases them, starting from problem 943. | Expert | Sep 2025 | | Not in cards |
| SEAL Showdown Users of Scale's Playground app chat with a model and sometimes compare its reply with a random opponent's reply. Scale ranks models with a Bradley-Terry model that controls for length and formatting. Rankings can be split by region, profession and age. | Everyday | Sep 2025 | 1,113 | Not in cards |
| Valyrian Games Coding Challenge Models create coding problems with integer answers and repeatedly solve their own problems to qualify. Tournament entrants solve one another’s problems and receive TrueSkill ratings. | Competition | Sep 2025 | 30 | Not in cards |
| VoxelBench Language models turn text or image prompts into voxel structures. Voters compare anonymous builds from the same prompt, and Glicko-2 ratings rank the models. | Everyday | Sep 2025 | 2,607 | Not in cards |
| VoxelBench Image Models turn image prompts into voxel structures. Human comparisons determine Glicko-2 ratings in a separate image-prompt vote pool. | Everyday | Sep 2025 | 2,215 | Not in cards |
| AI FPL Models manage Fantasy Premier League squads using the same weekly prompt. Their transfers, captain choices, and starting lineups earn points from real matches. | Everyday | Aug 2025 | Not in cards | |
| BiomedArena Researchers compare anonymous answers to biomedical research questions and vote for the response they prefer. | Expert | Aug 2025 | Not in cards | |
| FutureX Agents predict upcoming events collected daily from 195 websites in 11 domains, within a one-week window. Answers are scored once real results are crawled, with harder tiers weighted more. | Expert | Aug 2025 | 27.9 | Not in cards |
| Game Arena Chess Language models play full chess games against each other with the board given as FEN and PGN text. Ratings come from many games between each pair. | Expert | Aug 2025 | | Not in cards |
| Kaggle Game Arena Language models play each other in board, card and word games through a text harness. One leaderboard ranks models across all Game Arena games. | Expert | Aug 2025 | | Not in cards |
| LiveDRBench Agents perform 100 deep research tasks and return key claims discovered during search. Scores use claim-level F1. | Expert | Aug 2025 | 55.5% | Not in cards |
| LiveTradeBench Agents allocate simulated portfolios across US securities and Polymarket contracts using streamed prices and news. The researchers track portfolio returns over successive trading days. | University | Aug 2025 | 12.1% | Not in cards |
| LLM Chess Arena (Louis Guichard) Models play each other from a text diagram, piece locations, and recent moves. A model forfeits after three failed attempts to make a legal move. | School | Aug 2025 | | Not in cards |
| ArxivRollBench Models reconstruct passages from recently published scientific papers. They order fragments, fill gaps and choose what comes next. | University | Jul 2025 | 70.8% | Not in cards |
| DesignArena Code Users give a prompt, and anonymous models each build a website, UI component, browser game, data visualization or 3D scene as one HTML file. Users vote, and votes become Bradley-Terry ratings. | Everyday | Jul 2025 | 1,393 | Not in cards |
| Group Binary Search (Riedl) Agents each choose an integer without seeing the other guesses. They adjust after hearing whether the group sum exceeds or falls below a hidden target. | Everyday | Jul 2025 | | Not in cards |
| Husky Hold'em Bench Models write Python poker bots and revise them after practice games. Six bots compete at each table, and the tournament tracks average winnings. | University | Jul 2025 | Not in cards | |
| Music Arena Users write a music prompt, listen to tracks from two text-to-music systems, and vote. Preferences, listening data and feedback feed a leaderboard. | Everyday | Jul 2025 | | Not in cards |
| Music Arena Vocal Models generate music with vocals. Listeners compare anonymous tracks, and separate vocal preference votes determine the Arena score. | Everyday | Jul 2025 | | Not in cards |
| PACT Two models bargain over 20 rounds as buyer and seller, each with a hidden private value. Matches across many pairings rank the models. | Everyday | Jul 2025 | 1,607 | Not in cards |
| SciArena Researchers ask scientific literature questions and vote between two models' long cited answers. Votes give Elo ratings. | Expert | Jul 2025 | 1,151 | Not in cards |
| SF20K Competition 2025 Models answer story questions about short films. GPT-4.1-nano compares their answers with human references. Films released during the competition determine the final rankings. | Everyday | Jul 2025 | Not in cards | |
| HexMachina Models write Catan players in code and compete in the board game. | University | Jun 2025 | Not in cards | |
| LiveCodeBench Pro Continuously updated Codeforces, ICPC and IOI problems annotated by olympiad medalists. Reported as Elo. | Competition | Jun 2025 | 2,116 | Fading |
| LMArena Image Edit Arena Image editing models apply the same user edit request to the same image, anonymously and side by side. Community votes give Elo-style ratings. | Everyday | Jun 2025 | 1,519 | Not in cards |
| RoboArena Double-blind pairwise A/B trials of generalist DROID policies on real Franka arms, free-form tasks by volunteer evaluators. Elo-style rating. | Everyday | Jun 2025 | 1,608 | Not in cards |
| Seal-0 Models answer 111 fact-seeking questions where web search returns conflicting or unhelpful results. Answers are graded for accuracy, usually with search enabled. | University | Jun 2025 | 57.4% | Fading |
| Visual Strategic Bench Models identify objects, predict another player’s next action, and choose their own moves in ten games. The authors score each ability separately. | Everyday | Jun 2025 | 84.9% | Not in cards |
| Yupp VIBE Score People use Yupp for real tasks, see answers from two or more models, and pick the one they prefer. The preferences become VIBE scores for each model. | Everyday | Jun 2025 | Not in cards | |
| Estonian AI Barometer Models answer a visitor’s prompt in Estonian. The visitor compares two anonymous replies and votes before seeing which models wrote them. | Everyday | May 2025 | | Not in cards |
| gg-bench Models play 126 games invented by a language model and receive game-specific outcomes. | Everyday | May 2025 | 36.3% | Not in cards |
| SWE-bench-Live Agents resolve recent GitHub issues gathered by an automated pipeline, with 50 new verified Python tasks added each month. Tests in a docker sandbox check each patch. | Expert | May 2025 | | Dropped |
| SWE-bench-Live Verified Task success on the official Verified split. The source reports each submission denominator separately. Other language and platform splits remain separate. | Expert | May 2025 | 40% | Not in cards |
| SWE-rebench Agents fix fresh GitHub issues that an automated pipeline mines each month. Tests decide whether the issue is resolved. | Expert | May 2025 | 64.5% | System cards only |
| 3D Animation Arena Models reconstruct a moving 3D human from a video. Voters compare motion fidelity, ground contact, smoothness, and overall appearance. | Everyday | Apr 2025 | Not in cards | |
| Cyber Range Agents run end-to-end cyber operations in emulated networks. Scored as the share of scenarios passed in at least one of 16 trials. | Expert | Apr 2025 | 93.3% | System cards only |
| HalluLens Language models answer dynamically generated prompts that test factual and contextual hallucination. The evaluator builds fresh examples to reduce test leakage. | Everyday | Apr 2025 | Not in cards | |
| LMArena Search Arena People compare answers from two anonymous search-augmented models on their own questions and vote for the better one. Votes become Bradley-Terry ratings. | Everyday | Apr 2025 | 1,255 | Fading |
| MafiaBench Models play a social deduction game with hidden roles and public discussion. | Everyday | Apr 2025 | | Not in cards |
| TextArena Collection of competitive text games for LLM eval and RL, with an online Elo leaderboard. | Apr 2025 | Not in cards | ||
| Agronomy AI Arena Models answer a visitor's agronomy question. The visitor compares two anonymous answers and votes for the more helpful one. | University | Mar 2025 | Not in cards | |
| Dubesor AI Chess Leaderboard Models play chess by continuing game notation or by reasoning from the board and legal moves. The organizer maintains separate Elo ratings for each mode. | School | Mar 2025 | | Not in cards |
| MC-Bench Models write code that builds a Minecraft structure from a prompt. Users vote between two builds without knowing the models. | Everyday | Mar 2025 | Not in cards | |
| AI Diplomacy Models play the board game Diplomacy against each other, one power each. They negotiate, form alliances and betray each other in written messages. | Everyday | Feb 2025 | | Not in cards |
| Elimination Game Models play a multi-player game of public and private talk, alliances and votes to eliminate each other. A jury of eliminated players picks the winner. | Everyday | Feb 2025 | 8 | Not in cards |
| LLM Public Goods Game Agents choose how many tokens to contribute to a common pool. The evaluation reports the fraction each model contributes across repeated games. | Everyday | Feb 2025 | | Not in cards |
| SnakeBench Two language models each steer a snake on a shared board in a turn-based game, and the one that survives longest wins. Ratings come from many head-to-head games. | Everyday | Feb 2025 | 39 | Not in cards |
| Copilot Arena Code Editing Developers highlight existing code and ask for a change. They choose between two anonymous edits. | University | Jan 2025 | 1,058 | Not in cards |
| LiveCodeBench v5 Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. | Competition | Jan 2025 | | Fading |
| Reasoning Gym Models solve procedurally generated reasoning problems with automatic rewards. Harbor draws three easy and three hard instances from each of 96 task generators. | School | Jan 2025 | Not in cards | |
| RepoChat Arena Users give a GitHub repository link and a question. A retriever model picks files, two answer models reply, and users vote for the better answer. | Everyday | Jan 2025 | Not in cards | |
| Step Race Several models talk in public, then secretly pick moves in a race where colliding moves cancel. The game tests cooperation and deception. | Everyday | Jan 2025 | 5 | Not in cards |
| AraGen Models answer Arabic questions in four tasks: question answering, reasoning, orthographic and grammatical analysis, and safety. An LLM judge scores each answer on the 3C3H measure: correctness, completeness, conciseness, helpfulness, honesty and harmlessness. Only correct answers get credit on the other dimensions. | Everyday | Dec 2024 | 84.3% | Not in cards |
| Gray Swan Jailbreak Arena Human attackers try to jailbreak models for violent, self-harm and malicious code outputs. Scored as attack success rate per category. | Expert | Dec 2024 | 3.60% | System cards only |
| LMArena Text-to-Image Arena Image generation models answer the same user prompt side by side and anonymously. Community votes between the two images give Elo-style ratings. | Everyday | Dec 2024 | 1,428 | Not in cards |
| WebDev Arena Two anonymous models build web apps from a user's prompt, and the user votes for the better one. Votes yield Elo-style ratings. | Dec 2024 | 1,820 | Fading | |
| Copilot Arena Developers inside their IDE see paired code completions from two models and pick one. Pairwise votes give an Elo-style ranking of code models. | Everyday | Nov 2024 | | Not in cards |
| Judge Arena (Atla) Users compare two anonymous model judges that score and critique the same answer. | University | Nov 2024 | | Not in cards |
| Agent Arena (Gorilla) Users give a task, two agents built from different models, tools and frameworks run it, and users vote. A Bradley-Terry extension rates models, tools and frameworks. | Everyday | Oct 2024 | | Not in cards |
| Extended NYT Connections Models solve New York Times Connections puzzles with up to four extra trick words added. Each puzzle scores by the square of the share of groups found. | School | Oct 2024 | | Not in cards |
| GLEE Agents bargain, negotiate, or persuade another player through conversation while pursuing the payoff defined by the game. | University | Oct 2024 | 2,595 | Not in cards |
| Image2Struct Vision-language models turn images of webpages, LaTeX and musical scores into code that renders them. The render is compared with the input image. | University | Oct 2024 | 94.7 | Not in cards |
| External red team jailbreaks External red teamers and automated gap finding generate jailbreaks against the model. Scored on how often the model resists. | Expert | Sep 2024 | 44% | System cards only |
| ForecastBench Models give probabilities for 500 auto-generated questions per round about future events, drawn from prediction markets and time-series datasets. Forecasts are scored with difficulty-adjusted Brier scores once the questions resolve. | Expert | Sep 2024 | 0.11 | In use |
| LLM Chess Models play chess in multi-turn dialogs against a random player or the Komodo Dragon engine. The board reports wins, Elo and game duration, which tracks instruction following. | Everyday | Sep 2024 | 1,614 | Not in cards |
| RedTeam Arena Players get 60 seconds to make a model say a target word. An extended Elo ranks players, models and target prompts, with models ranked by resistance. | Everyday | Sep 2024 | Not in cards | |
| K-Sort Arena Users rank outputs from K image or video generation models at once instead of a pair. Bayesian updates turn the rankings into ratings. | Everyday | Aug 2024 | 31 | Not in cards |
| MTEB Arena Users compare retrieval, clustering and similarity results from two anonymous embedding models and vote for the better one. | Everyday | Jul 2024 | Not in cards | |
| Werewolf Arena Models compete in Werewolf with strategic speaking, deception, deduction, and persuasion. | Everyday | Jul 2024 | | Not in cards |
| GenAI-Arena Users compare outputs from two anonymous models for image generation, image editing and video generation, and vote. Votes give Elo ratings. | Everyday | Jun 2024 | 1,133 | Not in cards |
| GenAI-Arena Image Editing Models edit images from user instructions. Anonymous human comparisons determine Elo in this independent vote pool. | Everyday | Jun 2024 | Not in cards | |
| GenAI-Arena Image Generation Models generate images from text prompts. Anonymous human comparisons determine Elo in this independent vote pool. | Everyday | Jun 2024 | 1,133 | Not in cards |
| GenAI-Arena Video Generation Models generate videos from text prompts. Anonymous human comparisons determine Elo in this independent vote pool. | Everyday | Jun 2024 | Not in cards | |
| LiveBench Models answer monthly refreshed questions across math, coding, reasoning, language, instruction following, and data analysis. Scored automatically against objective ground truth. | University | Jun 2024 | | Dropped |
| Vision Arena People send an image with a prompt, compare answers from two anonymous vision-language models, and vote for the better one. Votes become Bradley-Terry ratings. | Everyday | Jun 2024 | 1,309 | Not in cards |
| AndroidWorld Agents complete 116 tasks across 20 real Android apps in an emulator. Each task checks the device's system state to judge success. | Everyday | May 2024 | 100% | In use |
| CompassArena Users chat with two anonymous models side by side, mostly in Chinese, and vote for the better answer. A Bradley-Terry model turns the votes into Elo-style ratings. | Everyday | May 2024 | 1,062 | Not in cards |
| Uncheatable Eval Models predict text from recently published documents. The evaluation measures perplexity across sources including news, research papers, fiction and code. | University | May 2024 | | Not in cards |
| CyberSecEval Vulnerability Exploitation Models solve randomly generated capture the flag challenges. Answers run against a compiled binary and score from 0 to 1. | Expert | Apr 2024 | Not in cards | |
| LiveCodeBench Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. | Competition | Mar 2024 | | In use |
| LLM Colosseum Two models each control a fighter in Street Fighter III and choose moves from text descriptions of the game. Wins give Elo ratings. | Everyday | Mar 2024 | 1,835 | Not in cards |
| GTBench Models play strategic games against other models or scripted opponents. The benchmark measures their advantage in game rewards. | Everyday | Feb 2024 | 0.13 | Not in cards |
| TTS Arena Users listen to two anonymous text-to-speech models read the same text and vote for the more natural one. Votes become Elo ratings. | Everyday | Feb 2024 | | Not in cards |
| WildVision Arena Users upload an image with a question, compare answers from two anonymous vision-language models, and vote. Votes give Elo ratings. | Everyday | Feb 2024 | 1,100 | Not in cards |
| Economics Arena for Large Language Models Models choose numbers in beauty contests and place sealed bids in auctions. Researchers compare payoffs and measure how strategies change when players see previous rounds. | University | Jan 2024 | 0.95 | Not in cards |
| LLM Game Benchmark Models compete in tic-tac-toe, Connect Four, Gomoku, and other grid games through a shared interface. | Everyday | Jan 2024 | 185.0 | Not in cards |
| Codeforces rating Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. | Competition | 2024 | | In use |
| NPHardEval Models solve algorithmic decision and optimization problems that vary in computational complexity. | University | Dec 2023 | 37.7% | Not in cards |
| AvalonBench Models play Resistance Avalon while discussing, deceiving, deducing, and negotiating with other agents. | Everyday | Nov 2023 | Not in cards | |
| LLMBoxing People see one question with two anonymous model answers and pick the better one. The site counts preferences between model pairs. | Everyday | Oct 2023 | | System cards only |
| MakeMeSay A manipulator model tries to make a GPT-4o manipulatee say a codeword without suspicion. Scored as manipulator win rate. | Everyday | Sep 2023 | 72% | Fading |
| CLEVA Models complete Chinese language tasks including text classification, instruction following, and classical Chinese comprehension. The evaluation samples questions for each round. | University | Aug 2023 | 0.96 | Not in cards |
| Human feedback win rates (Anthropic) Raters chat with two models on tasks such as coding, documents and creative writing and pick the better reply. Score is the win rate. | Everyday | Jul 2023 | System cards only | |
| LMArena (Elo) People chat with two anonymous models side by side and vote for the better reply. Votes become Elo-style ratings for each model. | May 2023 | | Fading | |
| SuperCLUE The CLUE team tests Chinese and foreign language models on periodic sets of Chinese questions covering language, professional knowledge, reasoning, agents and safety. Open answers are graded by a strong LLM judge, and results feed an overall ranking. | University | May 2023 | | Not in cards |
| StreamingQA Models answer time-stamped questions about news. The test measures how well they acquire new information while retaining older knowledge. | Everyday | May 2022 | Not in cards | |
| TREC-COVID Models rank scientific articles that address specific information needs about COVID-19. | Expert | Apr 2020 | Not in cards | |
| BioASQ Systems answer biomedical questions written by experts, using PubMed articles as evidence. Questions are yes/no, factoid, list or summary types. A second task asks systems to index new PubMed articles with MeSH headings. | Expert | 2013 | Not in cards | |
| 3D Arena Users compare two 3D models generated from the same input and vote for the better one. A live leaderboard ranks the generators. | Everyday | Not in cards | ||
| 3D Worlds Arena Models generate explorable 3D scenes from an image and caption. Voters inspect two anonymous scenes from the same input and choose a favorite. | Everyday | Not in cards | ||
| AA-Image-Editing Validated private evaluators compare two image edits made from the same instruction and vote blind. Votes become Elo ratings. | Everyday | Not in cards | ||
| AA-Video-Editing Voters compare two edits of the same video made from one instruction and pick the better one. Votes become Elo ratings. | Everyday | Not in cards | ||
| Artificial Analysis Image Arena Voters compare images from two text-to-image models for the same prompt and pick the better one. Votes become Elo ratings. | Everyday | Not in cards | ||
| Artificial Analysis Speech Arena Voters listen to two speech clips made from the same text and pick the more natural one. Votes become Elo ratings. | Everyday | | Fading | |
| Artificial Analysis Video Arena Voters compare videos from two text-to-video models for the same prompt and pick the better one. Votes become Elo ratings. | Everyday | Not in cards | ||
| ASCII Bench (Jason Botterill) Models draw pictures using plain text characters. Readers compare two anonymous drawings for the same prompt. The leaderboard estimates ratings from their votes. | Everyday | | Not in cards | |
| BAZAAR Agents buy and sell goods in a simulated competitive market. | University | 7 | Not in cards | |
| Chart Arena Models create charts. Pairwise votes determine the published Arena rating. | Everyday | 1,615 | Not in cards | |
| ChessBench (MadeByLupu) Models play each other with access to the board, move history and legal moves. The organizer rates each model and thinking setting with Glicko-2. | School | | Not in cards | |
| Code2Video Arena Models create videos through code. Pairwise votes determine the published Arena rating. | Everyday | | Not in cards | |
| CompassArena Multimodal Users compare anonymous multimodal model answers and vote for the better response. A Bradley-Terry model estimates ratings from these votes. | Everyday | | Not in cards | |
| Corp Speak Arena Models rewrite text in corporate language. Pairwise votes determine the published Arena rating. | Everyday | 1,723 | Not in cards | |
| Crest Arcade Micro Mouse Models steer a simulated robot through six daily mazes. The score compares each finish time with the median human time and gives unfinished mazes zero. | Everyday | | Not in cards | |
| DesignArena 3D Design Models build an interactive 3D scene that runs in the browser as a single HTML file. Users explore the scenes and vote for the better one. | Everyday | 1,535 | In use | |
| DesignArena Agentic Game Dev Coding agents build multi-file games from a user prompt. Users play the final games and vote, and the logs keep tool calls, retries and failures. | Everyday | 1,408 | Not in cards | |
| DesignArena Agentic Image to HTML Coding agents turn a user-uploaded image into a self-contained HTML website. Users compare how well each page reproduces the design and vote. | Everyday | 1,247 | Not in cards | |
| DesignArena Agentic Overall Coding agents build apps, games, slides and spreadsheets from user prompts. Users compare the results, and votes from every agent board form one rating. | Everyday | 1,443 | Not in cards | |
| DesignArena Android (Kotlin) Models build native Android apps in Kotlin from real user prompts. The apps run in an Android emulator, and users compare them and vote. | Everyday | 1,390 | Not in cards | |
| DesignArena ASCII Art Models draw a requested picture using only text characters. Users compare the ASCII art and vote for the better one. | Everyday | | Not in cards | |
| DesignArena Audio Realism Bench Voice models produce conversational speech, and users vote on which sounds more real. DesignArena built it because 1 to 5 opinion scores for TTS had saturated. | Everyday | 1,181 | Not in cards | |
| DesignArena Blender Agents build a 3D scene in Blender from a user prompt. Users compare the rendered scenes and vote for the better one. | Everyday | 1,381 | Not in cards | |
| DesignArena Builder AI app builders make full-stack web and mobile apps from the same user prompt. Users compare the results and vote for the better one. | Everyday | Not in cards | ||
| DesignArena Code to Video Agents write code that renders a video from a user prompt. Users compare the two videos and vote for the better one. | Everyday | 1,454 | Not in cards | |
| DesignArena Data Visualization Models turn a user prompt and its data into an interactive visualization as a single HTML file. Users compare the rendered charts and vote. | Everyday | 1,386 | Not in cards | |
| DesignArena Frontend Web App Coding agents build multi-file React frontends from real user prompts. Users compare the running apps and vote, and the run logs record agent behavior. | Everyday | 1,360 | In use | |
| DesignArena Full Stack Coding agents build complete React apps with sign-in, databases and backend code from real user prompts. Users try the apps and vote for the better one. | Everyday | 1,400 | Not in cards | |
| DesignArena Game Dev Models build a playable browser game as a single HTML file from a real user prompt. Users play the games and vote for the better one. | Everyday | 1,465 | In use | |
| DesignArena Godot Game Dev Coding agents build complete Godot projects with GDScript, scenes and generated assets, exported to run in the browser. Users play the games and vote. | Everyday | 1,315 | Not in cards | |
| DesignArena Graphic Design Image models make a graphic design piece from a user prompt. Users compare two anonymous designs and vote for the better one. | Everyday | 1,436 | Not in cards | |
| DesignArena HTML Slides Agents write a self-contained HTML slide deck that is converted to PPTX. Users compare the decks and vote for the better one. | Everyday | 1,531 | Not in cards | |
| DesignArena Image Image models generate a picture from a user prompt. Users compare two anonymous images and vote for the better one. | Everyday | 1,381 | Not in cards | |
| DesignArena Image Editing Image models edit a user image following a text instruction. Users compare the two edits and vote for the better one. | Everyday | 1,368 | Not in cards | |
| DesignArena Image to Frontend Coding agents turn a reference image into a multi-file frontend web app. Users compare the running apps and vote for the better one. | Everyday | | Not in cards | |
| DesignArena Image to HTML Models recreate a reference image or mockup as a working single-file HTML website. Users compare the rendered pages and vote for the closer, better one. | Everyday | 1,300 | Not in cards | |
| DesignArena Image to Video Video models animate a user image into a short video. Users compare two anonymous videos and vote for the better one. | Everyday | | Not in cards | |
| DesignArena Logo Image models design a logo from a user prompt. Users compare two anonymous logos and vote for the better one. | Everyday | 1,365 | Not in cards | |
| DesignArena Mobile Builder AI app builders make a mobile app from the same user prompt. Users compare the apps and vote for the better one. | Everyday | Not in cards | ||
| DesignArena Multi-Image to Video Video models compose one video from several reference images. Users compare two anonymous videos and vote for the better one. | Everyday | | Not in cards | |
| DesignArena Multimodal to Video Video models make a video from a mix of text, images and other inputs. Users compare two anonymous videos and vote for the better one. | Everyday | | Not in cards | |
| DesignArena PDF Agents build a PDF document from a user prompt. Users compare the two documents and vote for the better one. | Everyday | 1,346 | Not in cards | |
| DesignArena Proprietary Harness Slides Slide-making products build a presentation from a user prompt in their own harness. Users compare the decks and vote for the better one. | Everyday | Not in cards | ||
| DesignArena Python-PPTX Slides Agents build PowerPoint decks with python-pptx from a user prompt. Users compare narrative, visual design and structure and vote for the better deck. | Everyday | 1,377 | Not in cards | |
| DesignArena React Native Models build cross-platform mobile apps in React Native from real user prompts. Users compare the running apps and vote for the better one. | Everyday | 1,318 | Not in cards | |
| DesignArena Speech-to-Speech Voice models answer spoken input with speech. Users listen to two anonymous replies and vote for the better one. | Everyday | Not in cards | ||
| DesignArena Spreadsheet Agents build spreadsheets such as financial models, budgets and trackers as Excel or CSV workbooks. Users compare accuracy, structure and formatting and vote. | Everyday | 1,429 | Not in cards | |
| DesignArena SVG Models write SVG code for a design request such as an icon, logo, badge or illustration. Users compare the rendered graphics and vote. | Everyday | 1,420 | Not in cards | |
| DesignArena Text-to-Speech Speech models read the same text aloud. Users listen to two anonymous clips and vote for the more natural one. | Everyday | | Not in cards | |
| DesignArena UI Component Models build one interface component as a single HTML file from a real user prompt. Users compare the rendered components and vote for the better one. | Everyday | 1,395 | Not in cards | |
| DesignArena Video Arena Users compare videos from anonymous video generation models for the same prompt and vote for the better one. Votes become Elo ratings. | Everyday | | Not in cards | |
| DesignArena Video Editing Video models edit a user video following a text instruction. Users compare the two results and vote for the better one. | Everyday | | Not in cards | |
| DesignArena Video to Website Models watch a reference video and recreate its content as a working single-file HTML website. Users compare the rendered pages and vote. | Everyday | | Not in cards | |
| DesignArena Website Models build a complete website as one HTML file from a real user prompt. Users compare the rendered pages and vote for the better one. | Everyday | | In use | |
| ELI5 Arena Models explain concepts in simple language. Pairwise votes determine the published Arena rating. | Everyday | 1,695 | Not in cards | |
| FlagEval Arena Models answer the same prompt anonymously. Users compare their responses, and the leaderboard estimates a rating from the votes. | Everyday | 1,134 | Not in cards | |
| Floorplan Arena Models furnish hand-drawn room sketches. People compare the layouts and vote for the better floorplan. | Everyday | | Not in cards | |
| Game Arena 3 Check Chess (Generality) Models play both colors. A player can win by checking the opposing king three times. A double check counts once. | School | 27.0 | Not in cards | |
| Game Arena Anti Chess (Generality) Models must capture when they can. A player wins by losing every piece or by having no legal move. The king can be captured. | School | 25.0 | Not in cards | |
| Game Arena Atomic Chess (Generality) Models play chess with exploding captures. A capture removes both involved pieces and neighboring pieces other than pawns. Exploding the opposing king wins. | School | 27.0 | Not in cards | |
| Game Arena Crazyhouse Chess (Generality) Models play Crazyhouse chess in both colors. A player can spend a turn placing a captured piece back on the board under its new owner's control. | School | 26.0 | Not in cards | |
| Game Arena Four-in-a-Row (Generality) Models place pieces on any empty square of a four by nine grid. Four in a line wins. The tournaments use different board encodings. | Everyday | 92.0 | Not in cards | |
| Game Arena Horde Chess (Generality) Models take both sides in turn. White tries to checkmate with 36 pawns. Black starts with the usual pieces and must capture every white pawn. | School | 24.0 | Not in cards | |
| Game Arena King of the Hill (Generality) Models can win by moving their king onto any of the four central squares. The king must reach the square without moving into check. | School | 24.0 | Not in cards | |
| Game Arena Racing Kings (Generality) Models race their kings to the eighth rank without giving check. Black can draw by reaching that rank immediately after White. | School | 23.5 | Not in cards | |
| GameAdsBench Models generate video ads for games. Viewers watch two anonymous ads and choose the one they prefer. | University | | Not in cards | |
| Joke Arena Models write jokes. Pairwise votes determine the published Arena rating. | Everyday | 1,585 | Not in cards | |
| Kaggle Chess Suite Models play chess from the standard starting position and from selected openings. Kaggle publishes both ratings for each model. | School | | Not in cards | |
| Landing Page Arena Models create landing pages. Pairwise votes determine the published Arena rating. | Everyday | 1,694 | Not in cards | |
| LLM 3D Worlds Arena Models write Blender or three.js programs that build explorable 3D scenes. Voters compare the resulting scenes in a browser without seeing the model names. | Everyday | 1,253 | Not in cards | |
| LLM Debate Benchmark Models debate an opponent over multiple rounds. Judges compare the arguments. | University | 1,706 | Not in cards | |
| LLM Deceptiveness and Gullibility Models try to deceive other models and recognize deceptive claims. | Everyday | 1.10 | Not in cards | |
| LLM Games Models play Connect Four against one another. Ramp publishes match scores and final Elo ratings. | Everyday | 1,728 | Not in cards | |
| LLM Persuasion Benchmark Models try to change another model’s stated position over multiple conversation turns. | Everyday | 1.71 | Not in cards | |
| LLM SoccerArena Bundesliga 2026/27 Models forecast Bundesliga match scores and outcome probabilities with web access. The standings award points for score predictions made about 24 hours before kickoff. | Everyday | | Not in cards | |
| LLM SoccerArena La Liga 2026/27 Models forecast La Liga match scores and outcome probabilities with web access. The standings award points for score predictions made about 24 hours before kickoff. | Everyday | 129.0 | Not in cards | |
| LLM SoccerArena Premier League 2026/27 Models forecast Premier League match scores and outcome probabilities with web access. The standings use the predictions made about 24 hours before kickoff. | Everyday | 89.0 | Not in cards | |
| LMArena Video Edit Arena Video editing models apply the same edit request to the same clip, anonymously and side by side. Community votes give Elo-style ratings. | Everyday | | Not in cards | |
| Mana RP Bench Models write the next turn in a player’s story. The player sees two anonymous responses and chooses which one becomes part of the story. | Everyday | 1,280 | Not in cards | |
| Opening Lines Arena Models write opening lines for stories. Pairwise votes determine the published Arena rating. | Everyday | 1,611 | Not in cards | |
| Peer Arena (oddbit.ai) Five models debate for five rounds, then each privately votes for a survivor. The organizer calculates Bradley-Terry ratings from game outcomes. | Everyday | 1,799 | Not in cards | |
| PGG-Bench Agents decide how much to contribute to a public good and whether to punish other players. | Everyday | | Not in cards | |
| Pokémon VGC Bench (Ricotoma) Agents play competitive Pokémon doubles with assigned teams. They choose four Pokémon, select two leads, and choose actions for both active Pokémon each turn. | Competition | 1,901 | Not in cards | |
| SCADBench Models write OpenSCAD code from the same short prompt. Voters rotate two rendered objects and choose which one better fits the prompt. | University | | Not in cards | |
| Speech Arena Models generate speech. Listeners compare outputs, and pairwise votes determine Elo ratings. | Everyday | 1,566 | Not in cards | |
| StudyArena Audio Models produce audio responses to user prompts. Anonymous human comparisons determine the published rating in this separate vote pool. | Everyday | | Not in cards | |
| StudyArena Image Models produce image responses to user prompts. Anonymous human comparisons determine the published rating in this separate vote pool. | Everyday | 1,284 | Not in cards | |
| StudyArena Text Models produce text responses to user prompts. Anonymous human comparisons determine the published rating in this separate vote pool. | Everyday | | Not in cards | |
| SuperCLUE Arena Image Editing Models edit images in response to instructions. The arena ranks systems from pairwise comparisons. | University | 1,308 | Not in cards | |
| SuperCLUE Arena Image to Video Models animate images. The arena ranks systems from pairwise comparisons. | University | Not in cards | ||
| SuperCLUE Arena Reference to Video Models generate videos from references. The arena ranks systems from pairwise comparisons. | University | Not in cards | ||
| SuperCLUE Arena Text to Image Models generate images from text. The arena ranks systems from pairwise comparisons. | University | 1,458 | Not in cards | |
| SuperCLUE Arena Text to Speech Models synthesize speech from text. The arena ranks systems from pairwise comparisons. | Everyday | | Not in cards | |
| SuperCLUE Arena Text to Video Models generate videos from text. The arena ranks systems from pairwise comparisons. | University | Not in cards | ||
| SuperCLUE Arena Web Coding Models implement web interfaces. The arena ranks systems from pairwise comparisons. | University | | Not in cards | |
| Turing TTS Listeners hear a human recording and a synthesized recording. The score measures how often listeners identify the synthesized recording as human. | Everyday | | Not in cards | |
| Voice Arena Arabic MSA Models generate speech in Arabic MSA. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Bahasa Models generate speech in Bahasa. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena Brazilian Portuguese Models generate speech in Brazilian Portuguese. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena French Models generate speech in French. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena German Models generate speech in German. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena Hindi Models generate speech in Hindi. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Indian English Models generate speech in Indian English. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena Japanese Models generate speech in Japanese. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Mexican Spanish Models generate speech in Mexican Spanish. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Tagalog Models generate speech in Tagalog. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena US English Models generate speech in US English. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Vietnamese Models generate speech in Vietnamese. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| WMArena Video Arena Models generate a video from a starting image and an action. Viewers compare anonymous clips and vote for the more convincing result. | University | | Not in cards |