Benchmark Almanac

Kept fresh

Benchmarks that add new items on a schedule or draw new ones every time. We count 369 benchmarks here, 241 of them arenas. A benchmark belongs here when refresh is new items on a schedule or live.

How fast it grows

76 were released since July, more than the 65 in the first half of 2026.

  1. Earlier 11
  2. 2024 H1 18
  3. 2024 H2 17
  4. 2025 H1 30
  5. 2025 H2 43
  6. 2026 H1 65
  7. 2026 H2 76

How much is still open

No model has passed 85% on 41 of the 52 benchmarks here that score in percent, 79% of them.

Every benchmark in this view

Newest first. Facets come from each benchmark’s own documents, and our methodology says how we tag them.

BenchmarkLevelReleasedBest scoreStatus
3D GameDev Arena Agents build playable 3D browser games from specifications and reference images. Players compare anonymous game pairs. The authors also test functionality and playability. Expert Nov 2026 99.8% Not in cards
AssemblyAI Speech Benchmarks Speech recognition systems transcribe clean, accented, multilingual and code-switched audio, identify entities and diarize speakers. Error rates compare production systems across public speech sets. Expert Nov 2026 5.60% Not in cards
Beyond Survival Models play social deduction games with human-aligned strategies and interaction patterns. Everyday Nov 2026 58.7% Not in cards
Clocktower Bench Models play the social deduction game Blood on the Clocktower (the Trouble Brewing script) against each other, with a scripted Storyteller. One model takes every good seat and another every evil seat, so each game tests one model's detection of the Demon against another's deception. A Bradley–Terry fit turns 386 games into Elo ratings. Everyday Nov 2026 1,859 Not in cards
Crazyrouter AI Trading Arena Models buy, sell, or hold three cryptocurrencies every five minutes using simulated accounts. The grader values portfolios at live quotes and deducts trading fees. University Nov 2026 -7.45% Not in cards
CREF Models submit forecasts before the target period begins and receive scores after the actual values become available. University Nov 2026 Not in cards
IdeaTaste Agents propose research ideas from literature frozen at 2025. Evaluation compares their citations and ideas with 2026 publications and measures diversity. Research frontier Nov 2026 Not in cards
KradleBench Models play live multi-agent Minecraft games and act by writing code. The overall score weights collaboration, economy and conflict at 30% each, and logic and creativity at 5% each. Everyday Nov 2026 78.3 Not in cards
MindGames Agents compete in Colonel Blotto, Iterated Prisoner’s Dilemma, Codenames, and Secret Mafia. Everyday Nov 2026 Not in cards
Moneda GroundBench Models answer live questions about market figures. Moneda’s default board reports the share of correct answers against reference data, with incorrect answers reported separately. Expert Nov 2026 Not in cards
Ramen Bench Agents draw an interactive bowl of vegan ramen in one HTML file. Static checks verify the constraints and visitors rate the rendered pages. University Nov 2026 5 Not in cards
RoboColosseum Physical robots execute fresh instructions in paired blind trials. Human evaluators choose the better run or a tie, and comparisons determine ratings for each embodiment. Everyday Nov 2026 Not in cards
RSLiveBench-UAV Models answer questions about vehicles in drone images. The benchmark checks perception and spatial reasoning across different views and adds newly captured scenes. Everyday Nov 2026 59.9 Not in cards
Agents' Last Exam v1.1 Agents complete 122 public long-horizon professional workflow tasks across 55 sub-industries. Deterministic graders check outcomes against hidden references. Expert Oct 2026 47.5% In use
Alignment Index (Arena) Arena ranks agent models on failure signals mined from real Agent Arena sessions, such as unauthorized actions, false attribution and deceptive completion. Everyday Oct 2026 87.9 Not in cards
BallotBench Nine frontier agents forecast 863 contests in the 2026 US midterms each day until polls close. Proper scoring rules grade forecasts against certified results, beside Kalshi, Polymarket and expert ratings. Expert Oct 2026 0.11 Not in cards
CivBench Models play full games of Civilization V with the Vox Populi mod against each other on matched maps and seeds. Pairwise skill ratings rank long-term strategy. Expert Oct 2026 1,634 Not in cards
Hanabi Agent Benchmark Models play cooperative Hanabi in four-player teams. The benchmark scores shared board progress while hints and lives constrain coordination. Everyday Oct 2026 20.5 Not in cards
Human or AI? (Skills IL) Models write short Hebrew texts on the same assignments as human authors. Readers guess who wrote each text. Everyday Oct 2026 44.3% Not in cards
Kaggle ASCII Arena Models draw images using text characters. Voters compare the drawings. Everyday Oct 2026 2,016 Not in cards
Kaggle Explode Arena Models build interactive exploded views of objects in three.js. A submission must render a self contained page where the viewer can separate the parts. Votes determine the arena rating. University Oct 2026 1,712 Not in cards
Kaggle Landmark Arena Models recreate named landmarks in three.js. Each submission must render as a self contained web page. Votes determine the arena rating. University Oct 2026 1,788 Not in cards
Kaggle Pelican Arena Models draw a pelican riding a bicycle in SVG. The arena checks whether each drawing renders and asks people to compare the passing entries. University Oct 2026 1,816 Not in cards
Kaggle Sort Viz Arena Models use Manim to animate sorting algorithms and explain their time complexity. A submission must produce a valid video under the file size limit. Votes determine the arena rating. University Oct 2026 1,650 Not in cards
Kaggle SVG Arena Models draw SVG images from 120 prompts across twelve categories. Submissions must render and satisfy the drawing rule for the prompt. Votes determine the arena rating. University Oct 2026 1,699 Not in cards
LiveMACE Agents trade in live financial markets, and the benchmark scores the process capabilities behind their decisions, not only returns. Expert Oct 2026 Not in cards
LiveSearch Agents search for current financial figures. The evaluator runs a stored query against a professional data service to update the reference answer. University Oct 2026 76.7% Not in cards
micro1 Medical Index Models are scored on the equal-weight average of micro1 medical benchmarks, currently RealmPathology, PharmaBench and WellnessBench. Expert Oct 2026 71% Not in cards
Minecraft Bed Wars (YC Labs) Teams of agents play Minecraft Bed Wars against one another. The researchers compare the number of matches each team wins. Everyday Oct 2026 Not in cards
Rotom Champ Bench Models draft six level 50 Pokémon, select three for each match, and choose their moves. The organizer rates them on wins under Champions singles rules. Competition Oct 2026 2,304 Not in cards
Rotom Gen9 Bench Models draft six level 100 Pokémon and play the whole team in Generation 9 OverUsed singles. Match results determine their ratings. Competition Oct 2026 2,318 Not in cards
SettlerBench Season One Four models trade resources, build settlements, and negotiate in public or private messages. Final victory points determine pairwise results for the Bradley-Terry rating. Everyday Oct 2026 1,270 Not in cards
SpireClimb StS2 A10 Agents play Slay the Spire 2 at Ascension 10 through screenshots and mouse and keyboard, 15 runs per character. Scoring is the win rate. Everyday Oct 2026 49.3% Not in cards
TavernBench Agents build a Hearthstone Battlegrounds player through research, simulator updates, and live games. Expert Oct 2026 95.6 Not in cards
Turn based Benchmarks Models play turn based games including tic tac toe and rock paper scissors. The tasks evaluate game decisions and adaptation to opponents. Everyday Oct 2026 0.69 Not in cards
ArtifactArena Models design robot bodies, controllers and strategy for a simulated sumo-style arena. Artifacts fight head-to-head and models get Elo ratings. Expert Sep 2026 1,539 Not in cards
Bench on the Clocktower Ten models play the social deduction game Blood on the Clocktower together, one seat each, over 300 ten-player games of the Trouble Brewing script. Each model plays 210 games on the good team and 90 on the evil team. The score is the share of games its team won. Everyday Sep 2026 58% Not in cards
Brood War Bench Coding agents play StarCraft: Brood War against each other in real time. Systems are ranked by win rate over matches, with cost per game. Expert Sep 2026 100% Not in cards
CivBench (Civilization VI) Agents play Civilization VI and are evaluated on strategic progress in a long-running world. Expert Sep 2026 73.0 Not in cards
fAIntasy Football Models draft NFL players, set lineups, bid for free agents, and trade under the same rules. The standings show wins, fantasy points, and lineup efficiency. Everyday Sep 2026 4.00 Not in cards
Game Arena Go Language models play Go against each other through the Game Arena text harness. Ratings come from head-to-head games. Expert Sep 2026 1,877 Not in cards
HappyWorld Embodied The HappyWorld arena publishes a separate embodied vote pool and Elo ranking. Everyday Sep 2026 1,074 Not in cards
HappyWorld Spatial Reconstruction The HappyWorld arena publishes a separate spatial reconstruction vote pool and Elo ranking. Everyday Sep 2026 1,252 Not in cards
Large Language Model Football League Models draft, trade, and set weekly NFL lineups. Each team earns one result against its opponent and another against the league median. Everyday Sep 2026 Not in cards
LLM Fantasy Football Arena Guillotine Models draft NFL players and bid for free agents. Each week the lowest-scoring team leaves, and the remaining teams bid on its players. Everyday Sep 2026 10.0 Not in cards
Ludobench Backgammon Models play backgammon against opponents using dice rolls and legal moves. Ludobench publishes Glicko-2 ratings and match replays. Everyday Sep 2026 1,723 Not in cards
Ludobench Hokm Models play Hokm in partnerships. One player names trump, and partners try to take seven tricks before the other team. Everyday Sep 2026 100% Not in cards
Multi-Agent Arena Codenames Two teams of anonymized models play the word game Codenames. The board rates the clue giver and the guessers on separate rating scales. Everyday Sep 2026 1,535 Not in cards
Multi-Agent Arena Diplomacy Seven players, people or anonymized models, negotiate and fight over World War I Europe. Models are ranked by their mean share of the sum-of-squares score, where 14.3 is an even split. Everyday Sep 2026 37.5% Not in cards
RSI Arena Ten frontier agents each post-train the same Nemotron base model on a shared GPU cluster for six days. People then rank the resulting models in blind side-by-side votes, and agents retrain on that feedback. Research frontier Sep 2026 1,095 Not in cards
Xbench Measures firsthand X posts about frontier models and coding harnesses. An LLM labels each post's sentiment, aggregated per model per week. Sep 2026 75.7 Not in cards
Agent-Only Fantasy Football League Models draft NFL players, set lineups, and negotiate trades using the same prompt and tools. The organizer compares points, lineup efficiency, and cost per point. Everyday Aug 2026 1371.4 Not in cards
Game Arena Nine Men's Morris Language models play Nine Men's Morris. Forming a row of three, called a mill, lets a player remove one of the opponent's pieces. Everyday Aug 2026 1,190 Not in cards
Game Arena Reversi Language models play Reversi, also called Othello, flipping the opponent's disks that they flank in a line. Everyday Aug 2026 1,306 Not in cards
Game Arena Word Art Language models take turns drawing a word as ASCII art and guessing what the other model drew. Everyday Aug 2026 2,528 Not in cards
LLM Fantasy Football Arena Classic Models manage two NFL fantasy teams each. One league allows trades and messages. The other disables both. Each league scores its own draft and season. Everyday Aug 2026 0.90 Not in cards
MineAmongUs Vision-language agents play an embodied Among Us game with verbal and nonverbal deception. Everyday Aug 2026 81.3% Not in cards
Speech Agent Arena People hold live conversations with two hidden speech-to-speech agents on the same scenario, some with tool calls, then pick the one they preferred. Votes are fit to an Elo rating, and the arena also tracks task success. Everyday Aug 2026 1,070 In use
Agentic Trader Agents search for market information and place simulated foreign exchange orders using live prices. Each agent manages its own positions, cash, and margin. University Jul 2026 79.4% Not in cards
AI Battle Arena Models play poker, Gomoku, and Connect Four through the same interface. The overall leaderboard averages each model’s finishing position across five head-to-head contests. Everyday Jul 2026 88.0 Not in cards
AI Security Leaderboard Models face systematic jailbreak attacks across chemical, biological, radiological, nuclear, explosive and cyber domains. Scores count universal jailbreaks and estimate attacker cost. Expert Jul 2026 $60 Not in cards
Arena Search Factuality Arena combines user preferences with independent checks of verifiable claims in model responses. Everyday Jul 2026 1,218 Not in cards
Artificial Turf War Models bid for draft position, pick NFL players, and submit weekly lineups. The league ranks teams by wins and breaks ties with fantasy points. Everyday Jul 2026 6.00 Not in cards
Bot Analysis Arena Models write trading rules for simulated US stock accounts. Some agents revise their rules after trades. Software tests each revision against the current rules before adoption. University Jul 2026 Not in cards
Founder Bench Agents run four matched businesses per model through the acoco platform. The evaluation tracks real advertising, customer attention, and website traffic after seven days. Its headline customer-impressions metric adds ad impressions to organic site visits. Expert Jul 2026 127902.0 Not in cards
Game Arena Bargaining Two language models negotiate how to split a pool of items, each with private item values. Each side tries to reach a good deal without giving away its values. Everyday Jul 2026 317 Not in cards
Game Arena Coin Game Two language models move on a grid and pick up coins. Each should collect the coins that score for it and avoid the rest. Everyday Jul 2026 514 Not in cards
Game Arena Lines of Action Language models play Lines of Action, where a piece moves as far as the number of pieces on its line. The goal is to join all of one's pieces into one group. Everyday Jul 2026 1,274 Not in cards
Game Arena Ultimate Tic-Tac-Toe Language models play nine linked tic-tac-toe boards. Each move decides which small board the opponent must play next. Everyday Jul 2026 720 Not in cards
LMArcade Agents build browser games from the same prompt. Players try two games and vote before learning which models and coding tools made them. University Jul 2026 1,798 Not in cards
Multi-Agent Arena Risk Four commanders, people or anonymized models, trade sealed messages and then attack for territory on a 42-territory map. Models are ranked by their mean share of territory, where 25 is an even split. Everyday Jul 2026 38.8% Not in cards
Multi-Agent Arena Social Poker No-limit hold’em tournaments with table talk, where people and anonymized models share a table. Models are ranked by a rating fit on who finished ahead at each table. Everyday Jul 2026 1,553 Not in cards
Olam Labs Multi-Agent Arena People and anonymized models share a table in social strategy games: Diplomacy, Social Poker, Codenames and Risk. One rating combines every game, fit on who finished ahead of whom at each table. Everyday Jul 2026 1,544 Not in cards
ParliamentBench Models play Secret Hitler while deceiving, persuading, and reasoning under hidden roles. Everyday Jul 2026 81% Not in cards
PatientAgentBench Patient-facing health agents talk with a simulated patient and use a sandbox of healthcare tools for scheduling, prescriptions, telehealth and escalation. An LLM jury scores over a hundred clinician-grounded criteria. Expert Jul 2026 4.25 Not in cards
Real World VoiceEQ Voice systems recognize, produce and respond to acoustic information across speech recognition, speech to speech, speech understanding and text to speech tasks. Expert Jul 2026 0.92 Not in cards
Agent Arena Arena ranks models as the orchestrator in real Agent Mode sessions on arena.ai. Scores use behavioral signals such as file downloads, disapproval events, retries and steerability. Everyday Jun 2026 28.6% In use
Agents' Last Exam 1,500+ expert-sourced tasks from real projects across 55 occupations in three difficulty tiers. Objective verifiers grade deliverables. 5-hour cap. Expert Jun 2026 38.2% In use
BTF-3 Agents forecast already-resolved binary and numeric questions while researching a frozen snapshot of the web. Scoring pools Brier and ranked probability scores. Expert Jun 2026 0.12 Not in cards
Game Arena Checkers Language models play American checkers against each other, with forced jumps and crowned kings. Ratings come from head-to-head games. Everyday Jun 2026 1,382 Not in cards
Game Arena Clobber Language models play Clobber, where every move captures an adjacent enemy stone. The player who makes the last legal move wins. Everyday Jun 2026 819 Not in cards
Game Arena Dark Hex Language models play Hex without seeing the opponent's stones, so they must probe cells to learn the hidden board while building a path. Everyday Jun 2026 581 Not in cards
Game Arena Dots and Boxes Language models take turns drawing lines between dots on a grid. Closing the fourth side of a box claims it. Everyday Jun 2026 1,393 Not in cards
Guesswork (The Aggregate) Models predict benchmark scores before publication. The maker ranks forecasts by mean absolute error after scaling each error by the variation in that benchmark’s scores. University Jun 2026 0.56 Not in cards
LLM SoccerArena World Cup 2026 Models forecast World Cup match scores and outcome probabilities before kickoff. The organizers compare forecasts with match results under different prompts and web access rules. Everyday Jun 2026 168.0 Not in cards
OmniGameArena Models play games while response latency advances the game clock. Everyday Jun 2026 0.47 Not in cards
Poker Arena Models play no-limit Texas Hold’em across repeated sessions. The benchmark scores tournament chips and nine strategic dimensions, including memory and bluffing. Everyday Jun 2026 15730.0 Not in cards
Questflow Traders Arena Agents and people trade on Hyperliquid and Polymarket in a public competition. The leaderboard reports weekly portfolio returns alongside gains and trading volume. University Jun 2026 Not in cards
RTSGameBench Agents play the open real-time strategy game Beyond All Reason. Expert Jun 2026 Not in cards
3DCodeArena Models generate 3D assets through code. Human preference votes determine the published Bradley-Terry Elo rating. Expert May 2026 1,230 Not in cards
Agent Island Models play multi-agent games that require cooperation, conflict, and persuasion. Everyday May 2026 0.88 Not in cards
BoardGameBench (Homer Quan) Models play seven board game variants against a search engine. The scorer rewards quick wins and gives partial credit for longer losses. Everyday May 2026 264.5 Not in cards
Cattle Trade Agents play a long economic game with auctions, hidden offers, bargaining, bluffing, and resource limits. Everyday May 2026 Not in cards
euchre-bench Coding agents build a euchre bot and test it against four opponents. Each opponent contributes up to 100 wins to the total. University May 2026 140.0 Not in cards
Game Arena Word Association Teams of language models give one-word clues and guess linked words, in a game like Codenames, against an opposing team. Everyday May 2026 734 Not in cards
NewsBench (Forum AI) Models answer questions about current events. Judges calibrated against expert ratings score factual accuracy, neutrality, and source quality. Everyday May 2026 88.3% Not in cards
Prompt injection live bug bounty (Anthropic with Gray Swan) Expert red teamers submit prompt injection attacks against hidden frontier models in tool use, coding and computer use scenarios. Score is attack success rate. Expert May 2026 0.08% System cards only
PTCG-Bench Agents play the Pokémon Trading Card Game by choosing legal actions from a partially visible board. Match results determine their Glicko-2 ratings. School May 2026 1,854 Not in cards
SIGHT (Search and Intent Gauge for Hospitality and Tourism) Agents recommend restaurants, hotels and attractions from user preferences and search results. Evaluators check constraints, evidence and privacy. Everyday May 2026 Not in cards
ArXivLean Models prove or disprove theorems from recent arXiv papers, stated in Lean 4. A strict comparator checks each submitted proof against the fixed reference statement. Research frontier Apr 2026 79.3% System cards only
Clash of AIs Season 1 Models publish trade entries, targets, and stops before prices move. Clash of AIs simulates fills against market prices and ranks accounts by total equity. University Apr 2026 $283,903 Not in cards
Claw-Eval-Live Agents complete 105 workflow tasks in 17 families drawn from live ClawHub workflow signals, scored on both the workflow and the final workspace state. University Apr 2026 66.7% Not in cards
Cooperate to Compete (C2C) Multi-player conquest game with private non-binding negotiation and secret asymmetric objectives. LM agents and humans are ranked by Plackett-Luce player strength from their game results. Everyday Apr 2026 0.74 Not in cards
Game Arena Five in a Row Language models play a gomoku-style game and try to line up five of their tokens before the opponent does. Everyday Apr 2026 1,131 Not in cards
GomokuBench Models place stones on a 19 by 19 board against a search engine. The score rewards wins, quicker wins, and longer losses. Everyday Apr 2026 2.90 Not in cards
Image-to-WebDev Arena Users upload a screenshot, wireframe or mockup, two anonymous models build the website in Code Arena, and users vote for the better result. Votes become ratings with bootstrapped confidence intervals. Everyday Apr 2026 1,756 Not in cards
LiveMathematicianBench Models answer multiple choice questions about theorems from arXiv papers published after their training cutoff. Wrong options follow plausible but invalid proof sketches. Research frontier Apr 2026 45.7% System cards only
Sparse Fourier Recovery Models reconstruct a sparse one-dimensional signal from incomplete Fourier measurements. A reference solver provides a comparison for reconstruction accuracy. Expert Apr 2026 0.36 Not in cards
BrokenArXiv Models are asked to prove statements from recent arXiv papers that were changed to be plausible but false. Full credit goes to models that say the statement is false. Research frontier Mar 2026 84.4% In use
Clocktower Radio Models play the social deduction game Blood on the Clocktower (the Trouble Brewing script) in mirrored pairs of games. In each match one model plays good and the other evil, and then they swap. A coded Storyteller runs the game, and a Bradley–Terry fit over all matches gives ratings where 1500 is average. Everyday Mar 2026 1,708 Not in cards
Document Arena People upload their own PDFs with a question, compare answers from two anonymous models, and vote for the better one. Votes become Bradley-Terry ratings. Everyday Mar 2026 1,500 Not in cards
Echo General AI Prediction Leaderboard Models forecast unresolved events drawn from prediction markets, synthesized news questions and expert-written questions. Forecasts made at aligned times are compared in pairwise battles, and an Elo fit ranks the models. Expert Mar 2026 1,018 Not in cards
Gray Swan IPI Arena benchmark Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. Mar 2026 0% In use
GTO Wizard Benchmark Models play heads-up no-limit Texas Hold’em against GTO Wizard AI. AIVAT reduces variance while the benchmark reports poker winnings. Expert Mar 2026 -9.23 Not in cards
InteractiveBench Texas Hold’em Models play no-limit Texas Hold’em from partial observations of cards, chip stacks, and recent actions. Repeated matches measure chip gains, with invalid actions receiving one retry before an automatic fold. Competition Mar 2026 31.8 Not in cards
InteractiveBench Trust Game Models choose whether to cooperate or defect in repeated prisoner’s dilemma matches. A round-robin tournament scores average realized payoff per round, with a random stopping rule. Everyday Mar 2026 1.87 Not in cards
Monthly-SWEBench Coding agents resolve 100 fresh GitHub tasks each month, half bug fixes and half features, from newly merged pull requests. Harbor verifiers score pass@1. Expert Mar 2026 76% Not in cards
PokeAgent Challenge Agents compete in Pokemon battles under partial observability in a Battling Track and play the Pokemon RPG in a Speedrunning Track that needs long-horizon planning. Expert Mar 2026 Not in cards
Slop Arena Site collecting writing-preference data to tune slop detection to a user's taste, paired with the slop-guard tool. Mar 2026 Not in cards
ArXivMath Final-answer research-level math problems extracted monthly from recent arXiv papers and filtered to be self-contained. Research frontier Feb 2026 94.3% In use
Code Review Bench (Martian) AI code review tools review pull requests from five large open-source projects. An LLM judge matches their comments to human-verified golden comments. Expert Feb 2026 Not in cards
FutureEval (Metaculus) Models forecast open Metaculus questions with a fixed prompt and are scored as the questions resolve. Seasonal bot tournaments and Pro Forecaster baselines run alongside. Expert Feb 2026 Not in cards
Game Arena Chess Openings Language models play chess against each other from varied two-move openings sampled from 20 popular human openings. The board is given as text. Expert Feb 2026 1,263 Not in cards
Game Arena Four in a Row Language models play a Connect Four style game and try to line up four of their tokens before the opponent does. Everyday Feb 2026 1,003 Not in cards
Game Arena Poker Language models play heads-up no-limit Texas Hold'em against each other. Models are ranked by mean big blinds won per 100 hands. Expert Feb 2026 30.3 Not in cards
Game Arena Werewolf Teams of language models play the social deduction game Werewolf in natural language. Villagers must find the hidden werewolves, who try to deceive them. Expert Feb 2026 0 Not in cards
ProxyWar Models write bots for nine games. The harness tests each bot and runs tournaments to measure wins, errors, and decision time. University Feb 2026 39.6% Not in cards
Wiki Live Challenge Deep research agents write articles on topics of 100 recent Wikipedia Good Articles, which serve as expert references. Scored on 39 writing criteria and factual verifiability. Expert Feb 2026 53.6 Not in cards
Agent Poker Bench Models play 10-seat no-limit Texas Hold'em tournaments against each other. Chip counts and elimination order feed TrueSkill ratings. Expert Jan 2026 1,132 Not in cards
Buyout Game Eight language models play a money-driven elimination game with private transfers and a buyout endgame. Everyday Jan 2026 1975.4 Not in cards
KradleVerse Agent harnesses compete in a live Minecraft arena. Everyday Jan 2026 Not in cards
LMArena Image-to-Video Arena Video generation models animate the same input image with the same prompt, anonymously and side by side. Votes give Elo-style ratings. Everyday Jan 2026 1,503 Not in cards
LMArena Text-to-Video Arena Video generation models make clips from the same text prompt, anonymously and side by side. Votes from logged-in users give Elo-style ratings. Everyday Jan 2026 1,540 Not in cards
mage-bench Models play Magic: The Gathering in a benchmark that compares strategic play across games. Expert Jan 2026 1617.0 Not in cards
MegaGem Three models bid in sealed auctions with private cards. The benchmark rotates starting hands and fits an Elo-scaled rating to the order in which players finish. Everyday Jan 2026 Not in cards
MineBench Models turn a prompt into a Minecraft-style voxel build as raw 3D coordinates or tool calls. Users compare builds blind, and votes become rankings. Everyday Jan 2026 2,169 Not in cards
Prediction Arena Agents research events and trade contracts on Kalshi and Polymarket. The leaderboard ranks accounts by cash plus the market value of open positions. University Jan 2026 $138,945 Not in cards
Spyfall study Models play Spyfall while inferring hidden locations and managing deceptive dialogue. Everyday Jan 2026 1003.0 Not in cards
Strategy Bench Models compete in six social deduction and strategy game environments. Everyday Jan 2026 Not in cards
TetrisBench Models play Tetris head-to-head in the original benchmark. The Harbor strategy track grades an agent-written JavaScript placement function on 20 fixed piece sequences. Everyday Jan 2026 66.7% Not in cards
TradeRank Arena Models choose positions in cryptocurrencies and US stocks using shared market data and simulated capital. The leaderboard ranks portfolios by return. University Jan 2026 0% Not in cards
Encyclo-K Models judge questions composed at test time from knowledge statements extracted from authoritative textbooks. Random composition is meant to resist contamination and test several facts at once. University Dec 2025 62.1% Fading
Pieverse Agent Prediction Arena Agents trade event contracts on Polymarket with $1,000 each in real money. Pieverse tracks account value as market prices change. University Dec 2025 $981 Not in cards
WOLF Models play Werewolf to measure deception production and deception detection as separate abilities. Everyday Dec 2025 Not in cards
Arena Expert Experts compare model answers to questions drawn from their professional work. Arena ranks models from their votes. Expert Nov 2025 1,143 Not in cards
BigLaw Bench: Arena Legal experts compare two answers to a legal task. The benchmark reports pairwise win percentages. Expert Nov 2025 Not in cards
Code Arena Models act as coding agents that build web apps from user prompts, and users vote between two results. Votes yield Elo-style ratings. Nov 2025 1,826 In use
CodeClash Models build and improve their own codebases over many rounds, then compete in arenas such as Halite, poker and BattleSnake. Results give Elo ratings. Expert Nov 2025 1,385 Not in cards
OCR Arena Users upload a document, compare two anonymous OCR or vision-language models side by side, and vote. Votes become Elo ratings. Everyday Nov 2025 1,781 Not in cards
Vending-Bench Arena Several agents each run a vending machine at the same location in the Vending-Bench 2 environment. They can email, pay and trade with each other, but each is scored on its own money balance. Expert Nov 2025 $15,962 Not in cards
Alpha Arena Frontier models trade live markets with real money over a season, and the leaderboard ranks them by returns. Expert Oct 2025 Not in cards
BigCodeArena Users ask for code, both models' programs run in a sandbox, and users try the results before voting for the better answer. Votes give Elo ratings. Everyday Oct 2025 1,117 Not in cards
CATArena Code agents write programs that play board and card games in repeated tournaments. Between rounds they refine their code through self-reflection and by studying rivals' code. Scores are average ranks that separate first-round skill from improvement. Expert Oct 2025 1,799 Not in cards
CATArena Gomoku Code agents compete in Gomoku. The public competition board reports Elo Rating for each agent configuration. Expert Oct 2025 1,799 Not in cards
CATArena Pommerman Code agents compete in Pommerman. The public competition board reports Elo Rating for each agent configuration. Expert Oct 2025 Not in cards
CATArena Texas Holdem Code agents compete in Texas Holdem. The public competition board reports Elo Rating for each agent configuration. Expert Oct 2025 1,463 Not in cards
Epoch Capabilities Index An index that fits model capability and benchmark difficulty from scores on over 50 benchmarks in Epoch AI's hub, using item response theory. Scores read like Elo ratings. Research frontier Oct 2025 174.0 System cards only
HLE-Rolling A dynamic fork of Humanity's Last Exam where the makers remove, fix and re-add questions over time and publish an update log. Research frontier Oct 2025 Not in cards
Magentic Marketplace Agents search for sellers, exchange proposals and make purchases. The simulation measures how much value customers receive after paying for their purchases. Everyday Oct 2025 Not in cards
Prophet Arena Models and agents forecast open Kalshi prediction-market events and are scored once the events resolve. The live board ranks a difficulty-adjusted Arena score and also reports calibration error and market return. Expert Oct 2025 86.2 In use
Real-time Audio-Visual Interaction Evaluation Trained conversationalists hold three minute live video calls with a model. Real users rate each session for naturalness and fluency on a 0 to 3 scale. Everyday Oct 2025 1.37 System cards only
HUMAINE Prolific participants, stratified across 22 demographic groups in the US and UK, hold multi-turn conversations with two anonymous models on topics they choose. They compare the models overall and on dimensions such as trust, ethics and safety. A hierarchical Bradley-Terry-Davidson model turns the votes into ratings, adjusted to census data. Everyday Sep 2025 35 Not in cards
Project Euler (MathArena) Models solve recent Project Euler problems that need both mathematical insight and code. MathArena adds problems as Project Euler releases them, starting from problem 943. Expert Sep 2025 89% Not in cards
SEAL Showdown Users of Scale's Playground app chat with a model and sometimes compare its reply with a random opponent's reply. Scale ranks models with a Bradley-Terry model that controls for length and formatting. Rankings can be split by region, profession and age. Everyday Sep 2025 1,113 Not in cards
Valyrian Games Coding Challenge Models create coding problems with integer answers and repeatedly solve their own problems to qualify. Tournament entrants solve one another’s problems and receive TrueSkill ratings. Competition Sep 2025 30 Not in cards
VoxelBench Language models turn text or image prompts into voxel structures. Voters compare anonymous builds from the same prompt, and Glicko-2 ratings rank the models. Everyday Sep 2025 2,607 Not in cards
VoxelBench Image Models turn image prompts into voxel structures. Human comparisons determine Glicko-2 ratings in a separate image-prompt vote pool. Everyday Sep 2025 2,215 Not in cards
AI FPL Models manage Fantasy Premier League squads using the same weekly prompt. Their transfers, captain choices, and starting lineups earn points from real matches. Everyday Aug 2025 Not in cards
BiomedArena Researchers compare anonymous answers to biomedical research questions and vote for the response they prefer. Expert Aug 2025 Not in cards
FutureX Agents predict upcoming events collected daily from 195 websites in 11 domains, within a one-week window. Answers are scored once real results are crawled, with harder tiers weighted more. Expert Aug 2025 27.9 Not in cards
Game Arena Chess Language models play full chess games against each other with the board given as FEN and PGN text. Ratings come from many games between each pair. Expert Aug 2025 1,646 Not in cards
Kaggle Game Arena Language models play each other in board, card and word games through a text harness. One leaderboard ranks models across all Game Arena games. Expert Aug 2025 638 Not in cards
LiveDRBench Agents perform 100 deep research tasks and return key claims discovered during search. Scores use claim-level F1. Expert Aug 2025 55.5% Not in cards
LiveTradeBench Agents allocate simulated portfolios across US securities and Polymarket contracts using streamed prices and news. The researchers track portfolio returns over successive trading days. University Aug 2025 12.1% Not in cards
LLM Chess Arena (Louis Guichard) Models play each other from a text diagram, piece locations, and recent moves. A model forfeits after three failed attempts to make a legal move. School Aug 2025 1,495 Not in cards
ArxivRollBench Models reconstruct passages from recently published scientific papers. They order fragments, fill gaps and choose what comes next. University Jul 2025 70.8% Not in cards
DesignArena Code Users give a prompt, and anonymous models each build a website, UI component, browser game, data visualization or 3D scene as one HTML file. Users vote, and votes become Bradley-Terry ratings. Everyday Jul 2025 1,393 Not in cards
Group Binary Search (Riedl) Agents each choose an integer without seeing the other guesses. They adjust after hearing whether the group sum exceeds or falls below a hidden target. Everyday Jul 2025 75% Not in cards
Husky Hold'em Bench Models write Python poker bots and revise them after practice games. Six bots compete at each table, and the tournament tracks average winnings. University Jul 2025 Not in cards
Music Arena Users write a music prompt, listen to tracks from two text-to-music systems, and vote. Preferences, listening data and feedback feed a leaderboard. Everyday Jul 2025 1,394 Not in cards
Music Arena Vocal Models generate music with vocals. Listeners compare anonymous tracks, and separate vocal preference votes determine the Arena score. Everyday Jul 2025 1,107 Not in cards
PACT Two models bargain over 20 rounds as buyer and seller, each with a hidden private value. Matches across many pairings rank the models. Everyday Jul 2025 1,607 Not in cards
SciArena Researchers ask scientific literature questions and vote between two models' long cited answers. Votes give Elo ratings. Expert Jul 2025 1,151 Not in cards
SF20K Competition 2025 Models answer story questions about short films. GPT-4.1-nano compares their answers with human references. Films released during the competition determine the final rankings. Everyday Jul 2025 Not in cards
HexMachina Models write Catan players in code and compete in the board game. University Jun 2025 Not in cards
LiveCodeBench Pro Continuously updated Codeforces, ICPC and IOI problems annotated by olympiad medalists. Reported as Elo. Competition Jun 2025 2,116 Fading
LMArena Image Edit Arena Image editing models apply the same user edit request to the same image, anonymously and side by side. Community votes give Elo-style ratings. Everyday Jun 2025 1,519 Not in cards
RoboArena Double-blind pairwise A/B trials of generalist DROID policies on real Franka arms, free-form tasks by volunteer evaluators. Elo-style rating. Everyday Jun 2025 1,608 Not in cards
Seal-0 Models answer 111 fact-seeking questions where web search returns conflicting or unhelpful results. Answers are graded for accuracy, usually with search enabled. University Jun 2025 57.4% Fading
Visual Strategic Bench Models identify objects, predict another player’s next action, and choose their own moves in ten games. The authors score each ability separately. Everyday Jun 2025 84.9% Not in cards
Yupp VIBE Score People use Yupp for real tasks, see answers from two or more models, and pick the one they prefer. The preferences become VIBE scores for each model. Everyday Jun 2025 Not in cards
Estonian AI Barometer Models answer a visitor’s prompt in Estonian. The visitor compares two anonymous replies and votes before seeing which models wrote them. Everyday May 2025 1,653 Not in cards
gg-bench Models play 126 games invented by a language model and receive game-specific outcomes. Everyday May 2025 36.3% Not in cards
SWE-bench-Live Agents resolve recent GitHub issues gathered by an automated pipeline, with 50 new verified Python tasks added each month. Tests in a docker sandbox check each patch. Expert May 2025 71% Dropped
SWE-bench-Live Verified Task success on the official Verified split. The source reports each submission denominator separately. Other language and platform splits remain separate. Expert May 2025 40% Not in cards
SWE-rebench Agents fix fresh GitHub issues that an automated pipeline mines each month. Tests decide whether the issue is resolved. Expert May 2025 64.5% System cards only
3D Animation Arena Models reconstruct a moving 3D human from a video. Voters compare motion fidelity, ground contact, smoothness, and overall appearance. Everyday Apr 2025 Not in cards
Cyber Range Agents run end-to-end cyber operations in emulated networks. Scored as the share of scenarios passed in at least one of 16 trials. Expert Apr 2025 93.3% System cards only
HalluLens Language models answer dynamically generated prompts that test factual and contextual hallucination. The evaluator builds fresh examples to reduce test leakage. Everyday Apr 2025 Not in cards
LMArena Search Arena People compare answers from two anonymous search-augmented models on their own questions and vote for the better one. Votes become Bradley-Terry ratings. Everyday Apr 2025 1,255 Fading
MafiaBench Models play a social deduction game with hidden roles and public discussion. Everyday Apr 2025 1,561 Not in cards
TextArena Collection of competitive text games for LLM eval and RL, with an online Elo leaderboard. Apr 2025 Not in cards
Agronomy AI Arena Models answer a visitor's agronomy question. The visitor compares two anonymous answers and votes for the more helpful one. University Mar 2025 Not in cards
Dubesor AI Chess Leaderboard Models play chess by continuing game notation or by reasoning from the board and legal moves. The organizer maintains separate Elo ratings for each mode. School Mar 2025 1,907 Not in cards
MC-Bench Models write code that builds a Minecraft structure from a prompt. Users vote between two builds without knowing the models. Everyday Mar 2025 Not in cards
AI Diplomacy Models play the board game Diplomacy against each other, one power each. They negotiate, form alliances and betray each other in written messages. Everyday Feb 2025 60 Not in cards
Elimination Game Models play a multi-player game of public and private talk, alliances and votes to eliminate each other. A jury of eliminated players picks the winner. Everyday Feb 2025 8 Not in cards
LLM Public Goods Game Agents choose how many tokens to contribute to a common pool. The evaluation reports the fraction each model contributes across repeated games. Everyday Feb 2025 45.2% Not in cards
SnakeBench Two language models each steer a snake on a shared board in a turn-based game, and the one that survives longest wins. Ratings come from many head-to-head games. Everyday Feb 2025 39 Not in cards
Copilot Arena Code Editing Developers highlight existing code and ask for a change. They choose between two anonymous edits. University Jan 2025 1,058 Not in cards
LiveCodeBench v5 Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. Competition Jan 2025 81.2% Fading
Reasoning Gym Models solve procedurally generated reasoning problems with automatic rewards. Harbor draws three easy and three hard instances from each of 96 task generators. School Jan 2025 Not in cards
RepoChat Arena Users give a GitHub repository link and a question. A retriever model picks files, two answer models reply, and users vote for the better answer. Everyday Jan 2025 Not in cards
Step Race Several models talk in public, then secretly pick moves in a race where colliding moves cancel. The game tests cooperation and deception. Everyday Jan 2025 5 Not in cards
AraGen Models answer Arabic questions in four tasks: question answering, reasoning, orthographic and grammatical analysis, and safety. An LLM judge scores each answer on the 3C3H measure: correctness, completeness, conciseness, helpfulness, honesty and harmlessness. Only correct answers get credit on the other dimensions. Everyday Dec 2024 84.3% Not in cards
Gray Swan Jailbreak Arena Human attackers try to jailbreak models for violent, self-harm and malicious code outputs. Scored as attack success rate per category. Expert Dec 2024 3.60% System cards only
LMArena Text-to-Image Arena Image generation models answer the same user prompt side by side and anonymously. Community votes between the two images give Elo-style ratings. Everyday Dec 2024 1,428 Not in cards
WebDev Arena Two anonymous models build web apps from a user's prompt, and the user votes for the better one. Votes yield Elo-style ratings. Dec 2024 1,820 Fading
Copilot Arena Developers inside their IDE see paired code completions from two models and pick one. Pairwise votes give an Elo-style ranking of code models. Everyday Nov 2024 1,074 Not in cards
Judge Arena (Atla) Users compare two anonymous model judges that score and critique the same answer. University Nov 2024 1,335 Not in cards
Agent Arena (Gorilla) Users give a task, two agents built from different models, tools and frameworks run it, and users vote. A Bradley-Terry extension rates models, tools and frameworks. Everyday Oct 2024 1,045 Not in cards
Extended NYT Connections Models solve New York Times Connections puzzles with up to four extra trick words added. Each puzzle scores by the square of the share of groups found. School Oct 2024 99.3% Not in cards
GLEE Agents bargain, negotiate, or persuade another player through conversation while pursuing the payoff defined by the game. University Oct 2024 2,595 Not in cards
Image2Struct Vision-language models turn images of webpages, LaTeX and musical scores into code that renders them. The render is compared with the input image. University Oct 2024 94.7 Not in cards
External red team jailbreaks External red teamers and automated gap finding generate jailbreaks against the model. Scored on how often the model resists. Expert Sep 2024 44% System cards only
ForecastBench Models give probabilities for 500 auto-generated questions per round about future events, drawn from prediction markets and time-series datasets. Forecasts are scored with difficulty-adjusted Brier scores once the questions resolve. Expert Sep 2024 0.11 In use
LLM Chess Models play chess in multi-turn dialogs against a random player or the Komodo Dragon engine. The board reports wins, Elo and game duration, which tracks instruction following. Everyday Sep 2024 1,614 Not in cards
RedTeam Arena Players get 60 seconds to make a model say a target word. An extended Elo ranks players, models and target prompts, with models ranked by resistance. Everyday Sep 2024 Not in cards
K-Sort Arena Users rank outputs from K image or video generation models at once instead of a pair. Bayesian updates turn the rankings into ratings. Everyday Aug 2024 31 Not in cards
MTEB Arena Users compare retrieval, clustering and similarity results from two anonymous embedding models and vote for the better one. Everyday Jul 2024 Not in cards
Werewolf Arena Models compete in Werewolf with strategic speaking, deception, deduction, and persuasion. Everyday Jul 2024 26.7 Not in cards
GenAI-Arena Users compare outputs from two anonymous models for image generation, image editing and video generation, and vote. Votes give Elo ratings. Everyday Jun 2024 1,133 Not in cards
GenAI-Arena Image Editing Models edit images from user instructions. Anonymous human comparisons determine Elo in this independent vote pool. Everyday Jun 2024 Not in cards
GenAI-Arena Image Generation Models generate images from text prompts. Anonymous human comparisons determine Elo in this independent vote pool. Everyday Jun 2024 1,133 Not in cards
GenAI-Arena Video Generation Models generate videos from text prompts. Anonymous human comparisons determine Elo in this independent vote pool. Everyday Jun 2024 Not in cards
LiveBench Models answer monthly refreshed questions across math, coding, reasoning, language, instruction following, and data analysis. Scored automatically against objective ground truth. University Jun 2024 87.4% Dropped
Vision Arena People send an image with a prompt, compare answers from two anonymous vision-language models, and vote for the better one. Votes become Bradley-Terry ratings. Everyday Jun 2024 1,309 Not in cards
AndroidWorld Agents complete 116 tasks across 20 real Android apps in an emulator. Each task checks the device's system state to judge success. Everyday May 2024 100% In use
CompassArena Users chat with two anonymous models side by side, mostly in Chinese, and vote for the better answer. A Bradley-Terry model turns the votes into Elo-style ratings. Everyday May 2024 1,062 Not in cards
Uncheatable Eval Models predict text from recently published documents. The evaluation measures perplexity across sources including news, research papers, fiction and code. University May 2024 6.29 Not in cards
CyberSecEval Vulnerability Exploitation Models solve randomly generated capture the flag challenges. Answers run against a compiled binary and score from 0 to 1. Expert Apr 2024 Not in cards
LiveCodeBench Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. Competition Mar 2024 93.5% In use
LLM Colosseum Two models each control a fighter in Street Fighter III and choose moves from text descriptions of the game. Wins give Elo ratings. Everyday Mar 2024 1,835 Not in cards
GTBench Models play strategic games against other models or scripted opponents. The benchmark measures their advantage in game rewards. Everyday Feb 2024 0.13 Not in cards
TTS Arena Users listen to two anonymous text-to-speech models read the same text and vote for the more natural one. Votes become Elo ratings. Everyday Feb 2024 1,552 Not in cards
WildVision Arena Users upload an image with a question, compare answers from two anonymous vision-language models, and vote. Votes give Elo ratings. Everyday Feb 2024 1,100 Not in cards
Economics Arena for Large Language Models Models choose numbers in beauty contests and place sealed bids in auctions. Researchers compare payoffs and measure how strategies change when players see previous rounds. University Jan 2024 0.95 Not in cards
LLM Game Benchmark Models compete in tic-tac-toe, Connect Four, Gomoku, and other grid games through a shared interface. Everyday Jan 2024 185.0 Not in cards
Codeforces rating Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. Competition 2024 3,471 In use
NPHardEval Models solve algorithmic decision and optimization problems that vary in computational complexity. University Dec 2023 37.7% Not in cards
AvalonBench Models play Resistance Avalon while discussing, deceiving, deducing, and negotiating with other agents. Everyday Nov 2023 Not in cards
LLMBoxing People see one question with two anonymous model answers and pick the better one. The site counts preferences between model pairs. Everyday Oct 2023 22679.0 System cards only
MakeMeSay A manipulator model tries to make a GPT-4o manipulatee say a codeword without suspicion. Scored as manipulator win rate. Everyday Sep 2023 72% Fading
CLEVA Models complete Chinese language tasks including text classification, instruction following, and classical Chinese comprehension. The evaluation samples questions for each round. University Aug 2023 0.96 Not in cards
Human feedback win rates (Anthropic) Raters chat with two models on tasks such as coding, documents and creative writing and pick the better reply. Score is the win rate. Everyday Jul 2023 System cards only
LMArena (Elo) People chat with two anonymous models side by side and vote for the better reply. Votes become Elo-style ratings for each model. May 2023 1,527 Fading
SuperCLUE The CLUE team tests Chinese and foreign language models on periodic sets of Chinese questions covering language, professional knowledge, reasoning, agents and safety. Open answers are graded by a strong LLM judge, and results feed an overall ranking. University May 2023 73.6 Not in cards
StreamingQA Models answer time-stamped questions about news. The test measures how well they acquire new information while retaining older knowledge. Everyday May 2022 Not in cards
TREC-COVID Models rank scientific articles that address specific information needs about COVID-19. Expert Apr 2020 Not in cards
BioASQ Systems answer biomedical questions written by experts, using PubMed articles as evidence. Questions are yes/no, factoid, list or summary types. A second task asks systems to index new PubMed articles with MeSH headings. Expert 2013 Not in cards
3D Arena Users compare two 3D models generated from the same input and vote for the better one. A live leaderboard ranks the generators. Everyday Not in cards
3D Worlds Arena Models generate explorable 3D scenes from an image and caption. Voters inspect two anonymous scenes from the same input and choose a favorite. Everyday Not in cards
AA-Image-Editing Validated private evaluators compare two image edits made from the same instruction and vote blind. Votes become Elo ratings. Everyday Not in cards
AA-Video-Editing Voters compare two edits of the same video made from one instruction and pick the better one. Votes become Elo ratings. Everyday Not in cards
Artificial Analysis Image Arena Voters compare images from two text-to-image models for the same prompt and pick the better one. Votes become Elo ratings. Everyday Not in cards
Artificial Analysis Speech Arena Voters listen to two speech clips made from the same text and pick the more natural one. Votes become Elo ratings. Everyday 1,211 Fading
Artificial Analysis Video Arena Voters compare videos from two text-to-video models for the same prompt and pick the better one. Votes become Elo ratings. Everyday Not in cards
ASCII Bench (Jason Botterill) Models draw pictures using plain text characters. Readers compare two anonymous drawings for the same prompt. The leaderboard estimates ratings from their votes. Everyday 1,948 Not in cards
BAZAAR Agents buy and sell goods in a simulated competitive market. University 7 Not in cards
Chart Arena Models create charts. Pairwise votes determine the published Arena rating. Everyday 1,615 Not in cards
ChessBench (MadeByLupu) Models play each other with access to the board, move history and legal moves. The organizer rates each model and thinking setting with Glicko-2. School 2,170 Not in cards
Code2Video Arena Models create videos through code. Pairwise votes determine the published Arena rating. Everyday 1,598 Not in cards
CompassArena Multimodal Users compare anonymous multimodal model answers and vote for the better response. A Bradley-Terry model estimates ratings from these votes. Everyday 1,173 Not in cards
Corp Speak Arena Models rewrite text in corporate language. Pairwise votes determine the published Arena rating. Everyday 1,723 Not in cards
Crest Arcade Micro Mouse Models steer a simulated robot through six daily mazes. The score compares each finish time with the median human time and gives unfinished mazes zero. Everyday 89.3 Not in cards
DesignArena 3D Design Models build an interactive 3D scene that runs in the browser as a single HTML file. Users explore the scenes and vote for the better one. Everyday 1,535 In use
DesignArena Agentic Game Dev Coding agents build multi-file games from a user prompt. Users play the final games and vote, and the logs keep tool calls, retries and failures. Everyday 1,408 Not in cards
DesignArena Agentic Image to HTML Coding agents turn a user-uploaded image into a self-contained HTML website. Users compare how well each page reproduces the design and vote. Everyday 1,247 Not in cards
DesignArena Agentic Overall Coding agents build apps, games, slides and spreadsheets from user prompts. Users compare the results, and votes from every agent board form one rating. Everyday 1,443 Not in cards
DesignArena Android (Kotlin) Models build native Android apps in Kotlin from real user prompts. The apps run in an Android emulator, and users compare them and vote. Everyday 1,390 Not in cards
DesignArena ASCII Art Models draw a requested picture using only text characters. Users compare the ASCII art and vote for the better one. Everyday 1,457 Not in cards
DesignArena Audio Realism Bench Voice models produce conversational speech, and users vote on which sounds more real. DesignArena built it because 1 to 5 opinion scores for TTS had saturated. Everyday 1,181 Not in cards
DesignArena Blender Agents build a 3D scene in Blender from a user prompt. Users compare the rendered scenes and vote for the better one. Everyday 1,381 Not in cards
DesignArena Builder AI app builders make full-stack web and mobile apps from the same user prompt. Users compare the results and vote for the better one. Everyday Not in cards
DesignArena Code to Video Agents write code that renders a video from a user prompt. Users compare the two videos and vote for the better one. Everyday 1,454 Not in cards
DesignArena Data Visualization Models turn a user prompt and its data into an interactive visualization as a single HTML file. Users compare the rendered charts and vote. Everyday 1,386 Not in cards
DesignArena Frontend Web App Coding agents build multi-file React frontends from real user prompts. Users compare the running apps and vote, and the run logs record agent behavior. Everyday 1,360 In use
DesignArena Full Stack Coding agents build complete React apps with sign-in, databases and backend code from real user prompts. Users try the apps and vote for the better one. Everyday 1,400 Not in cards
DesignArena Game Dev Models build a playable browser game as a single HTML file from a real user prompt. Users play the games and vote for the better one. Everyday 1,465 In use
DesignArena Godot Game Dev Coding agents build complete Godot projects with GDScript, scenes and generated assets, exported to run in the browser. Users play the games and vote. Everyday 1,315 Not in cards
DesignArena Graphic Design Image models make a graphic design piece from a user prompt. Users compare two anonymous designs and vote for the better one. Everyday 1,436 Not in cards
DesignArena HTML Slides Agents write a self-contained HTML slide deck that is converted to PPTX. Users compare the decks and vote for the better one. Everyday 1,531 Not in cards
DesignArena Image Image models generate a picture from a user prompt. Users compare two anonymous images and vote for the better one. Everyday 1,381 Not in cards
DesignArena Image Editing Image models edit a user image following a text instruction. Users compare the two edits and vote for the better one. Everyday 1,368 Not in cards
DesignArena Image to Frontend Coding agents turn a reference image into a multi-file frontend web app. Users compare the running apps and vote for the better one. Everyday 1,299 Not in cards
DesignArena Image to HTML Models recreate a reference image or mockup as a working single-file HTML website. Users compare the rendered pages and vote for the closer, better one. Everyday 1,300 Not in cards
DesignArena Image to Video Video models animate a user image into a short video. Users compare two anonymous videos and vote for the better one. Everyday 1,319 Not in cards
DesignArena Logo Image models design a logo from a user prompt. Users compare two anonymous logos and vote for the better one. Everyday 1,365 Not in cards
DesignArena Mobile Builder AI app builders make a mobile app from the same user prompt. Users compare the apps and vote for the better one. Everyday Not in cards
DesignArena Multi-Image to Video Video models compose one video from several reference images. Users compare two anonymous videos and vote for the better one. Everyday 1,329 Not in cards
DesignArena Multimodal to Video Video models make a video from a mix of text, images and other inputs. Users compare two anonymous videos and vote for the better one. Everyday 1,305 Not in cards
DesignArena PDF Agents build a PDF document from a user prompt. Users compare the two documents and vote for the better one. Everyday 1,346 Not in cards
DesignArena Proprietary Harness Slides Slide-making products build a presentation from a user prompt in their own harness. Users compare the decks and vote for the better one. Everyday Not in cards
DesignArena Python-PPTX Slides Agents build PowerPoint decks with python-pptx from a user prompt. Users compare narrative, visual design and structure and vote for the better deck. Everyday 1,377 Not in cards
DesignArena React Native Models build cross-platform mobile apps in React Native from real user prompts. Users compare the running apps and vote for the better one. Everyday 1,318 Not in cards
DesignArena Speech-to-Speech Voice models answer spoken input with speech. Users listen to two anonymous replies and vote for the better one. Everyday Not in cards
DesignArena Spreadsheet Agents build spreadsheets such as financial models, budgets and trackers as Excel or CSV workbooks. Users compare accuracy, structure and formatting and vote. Everyday 1,429 Not in cards
DesignArena SVG Models write SVG code for a design request such as an icon, logo, badge or illustration. Users compare the rendered graphics and vote. Everyday 1,420 Not in cards
DesignArena Text-to-Speech Speech models read the same text aloud. Users listen to two anonymous clips and vote for the more natural one. Everyday 1,476 Not in cards
DesignArena UI Component Models build one interface component as a single HTML file from a real user prompt. Users compare the rendered components and vote for the better one. Everyday 1,395 Not in cards
DesignArena Video Arena Users compare videos from anonymous video generation models for the same prompt and vote for the better one. Votes become Elo ratings. Everyday 1,369 Not in cards
DesignArena Video Editing Video models edit a user video following a text instruction. Users compare the two results and vote for the better one. Everyday 1,373 Not in cards
DesignArena Video to Website Models watch a reference video and recreate its content as a working single-file HTML website. Users compare the rendered pages and vote. Everyday 1,274 Not in cards
DesignArena Website Models build a complete website as one HTML file from a real user prompt. Users compare the rendered pages and vote for the better one. Everyday 1,375 In use
ELI5 Arena Models explain concepts in simple language. Pairwise votes determine the published Arena rating. Everyday 1,695 Not in cards
FlagEval Arena Models answer the same prompt anonymously. Users compare their responses, and the leaderboard estimates a rating from the votes. Everyday 1,134 Not in cards
Floorplan Arena Models furnish hand-drawn room sketches. People compare the layouts and vote for the better floorplan. Everyday 1,807 Not in cards
Game Arena 3 Check Chess (Generality) Models play both colors. A player can win by checking the opposing king three times. A double check counts once. School 27.0 Not in cards
Game Arena Anti Chess (Generality) Models must capture when they can. A player wins by losing every piece or by having no legal move. The king can be captured. School 25.0 Not in cards
Game Arena Atomic Chess (Generality) Models play chess with exploding captures. A capture removes both involved pieces and neighboring pieces other than pawns. Exploding the opposing king wins. School 27.0 Not in cards
Game Arena Crazyhouse Chess (Generality) Models play Crazyhouse chess in both colors. A player can spend a turn placing a captured piece back on the board under its new owner's control. School 26.0 Not in cards
Game Arena Four-in-a-Row (Generality) Models place pieces on any empty square of a four by nine grid. Four in a line wins. The tournaments use different board encodings. Everyday 92.0 Not in cards
Game Arena Horde Chess (Generality) Models take both sides in turn. White tries to checkmate with 36 pawns. Black starts with the usual pieces and must capture every white pawn. School 24.0 Not in cards
Game Arena King of the Hill (Generality) Models can win by moving their king onto any of the four central squares. The king must reach the square without moving into check. School 24.0 Not in cards
Game Arena Racing Kings (Generality) Models race their kings to the eighth rank without giving check. Black can draw by reaching that rank immediately after White. School 23.5 Not in cards
GameAdsBench Models generate video ads for games. Viewers watch two anonymous ads and choose the one they prefer. University 1,283 Not in cards
Joke Arena Models write jokes. Pairwise votes determine the published Arena rating. Everyday 1,585 Not in cards
Kaggle Chess Suite Models play chess from the standard starting position and from selected openings. Kaggle publishes both ratings for each model. School 1,238 Not in cards
Landing Page Arena Models create landing pages. Pairwise votes determine the published Arena rating. Everyday 1,694 Not in cards
LLM 3D Worlds Arena Models write Blender or three.js programs that build explorable 3D scenes. Voters compare the resulting scenes in a browser without seeing the model names. Everyday 1,253 Not in cards
LLM Debate Benchmark Models debate an opponent over multiple rounds. Judges compare the arguments. University 1,706 Not in cards
LLM Deceptiveness and Gullibility Models try to deceive other models and recognize deceptive claims. Everyday 1.10 Not in cards
LLM Games Models play Connect Four against one another. Ramp publishes match scores and final Elo ratings. Everyday 1,728 Not in cards
LLM Persuasion Benchmark Models try to change another model’s stated position over multiple conversation turns. Everyday 1.71 Not in cards
LLM SoccerArena Bundesliga 2026/27 Models forecast Bundesliga match scores and outcome probabilities with web access. The standings award points for score predictions made about 24 hours before kickoff. Everyday 57.0 Not in cards
LLM SoccerArena La Liga 2026/27 Models forecast La Liga match scores and outcome probabilities with web access. The standings award points for score predictions made about 24 hours before kickoff. Everyday 129.0 Not in cards
LLM SoccerArena Premier League 2026/27 Models forecast Premier League match scores and outcome probabilities with web access. The standings use the predictions made about 24 hours before kickoff. Everyday 89.0 Not in cards
LMArena Video Edit Arena Video editing models apply the same edit request to the same clip, anonymously and side by side. Community votes give Elo-style ratings. Everyday 1,424 Not in cards
Mana RP Bench Models write the next turn in a player’s story. The player sees two anonymous responses and chooses which one becomes part of the story. Everyday 1,280 Not in cards
Opening Lines Arena Models write opening lines for stories. Pairwise votes determine the published Arena rating. Everyday 1,611 Not in cards
Peer Arena (oddbit.ai) Five models debate for five rounds, then each privately votes for a survivor. The organizer calculates Bradley-Terry ratings from game outcomes. Everyday 1,799 Not in cards
PGG-Bench Agents decide how much to contribute to a public good and whether to punish other players. Everyday 14 Not in cards
Pokémon VGC Bench (Ricotoma) Agents play competitive Pokémon doubles with assigned teams. They choose four Pokémon, select two leads, and choose actions for both active Pokémon each turn. Competition 1,901 Not in cards
SCADBench Models write OpenSCAD code from the same short prompt. Voters rotate two rendered objects and choose which one better fits the prompt. University 1,685 Not in cards
Speech Arena Models generate speech. Listeners compare outputs, and pairwise votes determine Elo ratings. Everyday 1,566 Not in cards
StudyArena Audio Models produce audio responses to user prompts. Anonymous human comparisons determine the published rating in this separate vote pool. Everyday 1,216 Not in cards
StudyArena Image Models produce image responses to user prompts. Anonymous human comparisons determine the published rating in this separate vote pool. Everyday 1,284 Not in cards
StudyArena Text Models produce text responses to user prompts. Anonymous human comparisons determine the published rating in this separate vote pool. Everyday 1,375 Not in cards
SuperCLUE Arena Image Editing Models edit images in response to instructions. The arena ranks systems from pairwise comparisons. University 1,308 Not in cards
SuperCLUE Arena Image to Video Models animate images. The arena ranks systems from pairwise comparisons. University Not in cards
SuperCLUE Arena Reference to Video Models generate videos from references. The arena ranks systems from pairwise comparisons. University Not in cards
SuperCLUE Arena Text to Image Models generate images from text. The arena ranks systems from pairwise comparisons. University 1,458 Not in cards
SuperCLUE Arena Text to Speech Models synthesize speech from text. The arena ranks systems from pairwise comparisons. Everyday 1,010 Not in cards
SuperCLUE Arena Text to Video Models generate videos from text. The arena ranks systems from pairwise comparisons. University Not in cards
SuperCLUE Arena Web Coding Models implement web interfaces. The arena ranks systems from pairwise comparisons. University 1,462 Not in cards
Turing TTS Listeners hear a human recording and a synthesized recording. The score measures how often listeners identify the synthesized recording as human. Everyday 10.5% Not in cards
Voice Arena Arabic MSA Models generate speech in Arabic MSA. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,192 System cards only
Voice Arena Bahasa Models generate speech in Bahasa. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,138 Not in cards
Voice Arena Brazilian Portuguese Models generate speech in Brazilian Portuguese. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,127 System cards only
Voice Arena French Models generate speech in French. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,084 Not in cards
Voice Arena German Models generate speech in German. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,034 Not in cards
Voice Arena Hindi Models generate speech in Hindi. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,097 System cards only
Voice Arena Indian English Models generate speech in Indian English. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,066 Not in cards
Voice Arena Japanese Models generate speech in Japanese. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,188 System cards only
Voice Arena Mexican Spanish Models generate speech in Mexican Spanish. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,138 System cards only
Voice Arena Tagalog Models generate speech in Tagalog. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,130 Not in cards
Voice Arena US English Models generate speech in US English. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,083 System cards only
Voice Arena Vietnamese Models generate speech in Vietnamese. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. Everyday 1,141 System cards only
WMArena Video Arena Models generate a video from a starting image and an action. Viewers compare anonymous clips and vote for the more convincing result. University 1,487 Not in cards