Science and reasoning
We track 97 benchmarks in science and reasoning. They run from everyday level to research frontier level.
OpenAI held the top science and reasoning score 39% of the contested time
That's about 18 of the 46 months since January 2023, averaged over these benchmarks. We only count time while other labs are still testing, and stop once a model passes 85%. Faded stretches no longer count.
OpenAI DeepSeek xAI Google DeepMind Anthropic NVIDIA Tencent Qwen Baidu Meituan Ant Group ByteDance Seed Mistral AI Meta Moonshot AI
Everyday Most adults, without training
School Students, from grade school to high school
University Undergraduates and entry-level professionals
Competition The strongest students, in olympiad-style contests
Expert Specialists with years of training, often over hours of work
Research frontier Researchers, on open problems or tasks that take days
Sources: Official leaderboards, Epoch AI runs (CC BY 4.0) and labs’ model cards
Share of contested time at number one
OpenAI 39% Anthropic 26% 
Google DeepMind 19% xAI 6% 
Ant Group 3% 
Meta 2% 
DeepSeek 1% 
Tencent 1% 
ByteDance Seed 1% 
Mistral AI 1% 
NVIDIA 1%
Google DeepMind 19% xAI 6% Ant Group 3% Meta 2% DeepSeek 1% Tencent 1% ByteDance Seed 1% Mistral AI 1% NVIDIA 1% Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Closed out: 7 benchmarks passed 85%
OpenAI 4 Anthropic 1 Google DeepMind 1 
Qwen 1
Google DeepMind 1 Qwen 1 Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
- Sep 2026 ARC-AGI-3 GPT-6 Astra
- Sep 2026 SimpleBench Claude Fable 5.1
- Apr 2026 ARC-AGI-2 GPT-5.5
- Dec 2025 ARC-AGI GPT-5.2 Pro
- Jun 2025 GPQA Diamond Gemini 2.5 Pro Jun 2025
- Jun 2025 DTBench o3-pro
- May 2025 ZebraLogic Qwen3-30B-A3B
Each benchmark counts from the one source its own page leads with: its official leaderboard when that is current, else Epoch AI’s own runs, else labs’ own numbers. We date independent runs by the model’s release and lab reports by when they came out.
OpenAI DeepSeek xAI Google DeepMind Anthropic NVIDIA Mistral AI Meta Moonshot AI
Everyday Most adults, without training
School Students, from grade school to high school
University Undergraduates and entry-level professionals
Competition The strongest students, in olympiad-style contests
Expert Specialists with years of training, often over hours of work
Research frontier Researchers, on open problems or tasks that take days
Sources: Official leaderboards and Epoch AI runs (CC BY 4.0)
Share of contested time at number one, independent runs only
OpenAI 39% Anthropic 25% Google DeepMind 24% xAI 6% Meta 2% DeepSeek 1% Mistral AI 1% NVIDIA 1%
Google DeepMind 24% xAI 6% Meta 2% DeepSeek 1% Mistral AI 1% NVIDIA 1% Sources: Official leaderboards and Epoch AI (CC BY 4.0)
Closed out: 7 benchmarks passed 85%
OpenAI 5 Anthropic 1 Google DeepMind 1
Google DeepMind 1 Sources: Official leaderboards and Epoch AI (CC BY 4.0)
- Sep 2026 ARC-AGI-3 GPT-6 Astra
- Sep 2026 SimpleBench Claude Fable 5.1
- Apr 2026 ARC-AGI-2 GPT-5.5
- Dec 2025 ARC-AGI GPT-5.2 Pro
- Jun 2025 GPQA Diamond Gemini 2.5 Pro Jun 2025
- Jun 2025 DTBench o3-pro
- Feb 2025 ZebraLogic o3-mini
Every independent run: each benchmark’s own leaderboard, Epoch AI’s runs and boards Epoch AI copied, dated by the model’s release. We leave out benchmarks that only labs report.
OpenAI DeepSeek ByteDance Seed xAI Google DeepMind Anthropic NVIDIA Meituan Qwen Tencent Mistral AI Meta
Everyday Most adults, without training
School Students, from grade school to high school
University Undergraduates and entry-level professionals
Competition The strongest students, in olympiad-style contests
Expert Specialists with years of training, often over hours of work
Research frontier Researchers, on open problems or tasks that take days
Sources: Official leaderboards, Epoch AI runs (CC BY 4.0) and labs’ model cards
Share of contested time at number one, claimed scores included
OpenAI 39% Anthropic 25% Google DeepMind 22% xAI 6% ByteDance Seed 3% Meta 1% DeepSeek 1% Qwen 1% Mistral AI 1%
Google DeepMind 22% xAI 6% ByteDance Seed 3% Meta 1% DeepSeek 1% Qwen 1% Mistral AI 1% Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
Closed out: 9 benchmarks passed 85%
OpenAI 5 Anthropic 2 Qwen 1 Google DeepMind 1
Qwen 1 Google DeepMind 1 Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards
- Sep 2026 ARC-AGI-3 GPT-6 Astra
- Sep 2026 SimpleBench Claude Fable 5.1
- Aug 2026 HiPhO Qwen 3.8 Max
- Jul 2026 BioMysteryBench Claude Opus 5
- Apr 2026 ARC-AGI-2 GPT-5.5
- Dec 2025 ARC-AGI GPT-5.2 Pro
- Jun 2025 GPQA Diamond Gemini 2.5 Pro Jun 2025
- Jun 2025 DTBench o3-pro
- Feb 2025 ZebraLogic o3-mini
Every score, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.
From everyday to research frontier level
Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.
Everyday
Most adults, without training
University
Undergraduates and entry-level professionals
Competition
The strongest students, in olympiad-style contests
Expert
Specialists with years of training, often over hours of work
- GPQA Diamond
- DTBench
- GPQA Main
- BioMysteryBench
- Terminal-Bench-Science
- SciArena
- MysteryMechanism
- xbench-ScienceQA
- LatchBio Capabilities v1.0
- scBench (LatchBio SingleCellBench)
- ChemBench
- Surface Evolver Bench
- Verifying the Verifiers (Project APE CRED benchmark)
- FrontierPhysics
- PHYSICS
- SpatialBench (LatchBio)
- PHYSICS-Verified
- CORE-Bench
- SGI-Bench
- FrontierScience
- FrontierScience-Research
- AstaBench
- SciPredict
- FrontierChallenge
- LABBench2
- SciUniverse
- BixBench
- Organic Chemistry (Anthropic)
- Matbench
- SciReasoner evaluation suite
- Biomedical image analysis (Anthropic)
- BioPipelineBench (Anthropic)
- De novo protein binder design (Anthropic)
- GPQA Extended
- Medicinal Chemistry (Anthropic)
- Morphology-to-molecule matching (Anthropic)
- Phylogenetics (Anthropic)
- ScholarCatalyst
- MaCBench
- Organic Chemistry V2 (Anthropic)
- Structural Biology (Anthropic)
- SuperMatBench
- BioPipelineBench Verified (Anthropic)
- Protein Design (Anthropic)
- Anthropic internal life-sciences evals
- ScienceAgentBench
- SciTS
Research frontier
Researchers, on open problems or tasks that take days
By kind
Science questions
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| GPQA Diamond 198 graduate-level 'Google-proof' multiple-choice questions in biology, chemistry and physics. | Expert | Checked automatically | 96.5% | In use |
| GPQA Main Models answer the 448 question main set of GPQA, graduate level biology, chemistry and physics questions written by experts. Scored by accuracy. | Expert | Checked automatically | 80.3% | Dropped |
| SciArena Researchers ask scientific literature questions and vote between two models' long cited answers. Votes give Elo ratings. | Expert | Ranked by votes | 1,151 | Not in cards |
| xbench-ScienceQA Models answer postgraduate-level science questions written by experts, in Chinese, as part of the xbench AGI Tracking series. | Expert | Checked automatically | 76.4% | Not in cards |
| ChemBench Models answer more than 2,700 chemistry questions on knowledge and reasoning. Results are compared with a group of human chemists. | Expert | Checked automatically | 70.9% | Dropped |
| PHYSICS Models solve 1,297 PhD qualifying exam physics problems across six subfields, 298 of them with images. Answers are checked by symbolic equivalence. | Expert | Checked automatically | 59.9% | Dropped |
| PHYSICS-Verified Corrected PHYSICS test problems with answer-level accuracy. The complete verified test set. | Expert | Checked automatically | 91.1% | Not in cards |
| PhyX Models solve 3,000 image-based physics problems across six domains such as mechanics and optics. Versions include multiple choice and open-ended answers. | University | Graded by a model | 45.8% | Fading |
| TheoremQA Models solve 800 university-level questions in math, physics, EE and CS, and finance that each need a specific theorem. Scored by answer accuracy. | University | Checked automatically | 44.4% | Dropped |
| A^3-Bench Models solve 2,198 science reasoning problems annotated with the anchors and attractors (prior knowledge) they need. An AAUI metric measures how much of that knowledge the model activates. | University | Checked automatically | 65.1% | Not in cards |
| ATLAS (OpenCompass) Models answer about 800 original PhD-level problems across mathematics, physics, chemistry, biology, computer science, earth science and materials science. Answers are open-form and checked by an LLM judge. | Research frontier | Graded by a model | 42.9% | Not in cards |
| PhysicsFinals Models answer 61 undergraduate physics final exam problems written by physics professors. A professor grades the answers. | University | Graded by people | 63.9% | Dropped |
| GPQA Extended Models answer all 546 GPQA questions, graduate level biology, chemistry and physics questions written by experts. Scored by accuracy. | Expert | Checked automatically | 48.8% | System cards only |
| GRE Physics Models answer questions from official GRE Physics practice tests. Scored as accuracy. | University | Checked automatically | 80% | System cards only |
| SciBench Models solve open-ended college-level problems in mathematics, chemistry and physics. Most answers are numbers, which a script checks against the key. | University | Checked automatically | 8.45% | Not in cards |
| SciQ Models answer crowdsourced multiple-choice science exam questions in physics, chemistry and biology, each with a supporting passage. | School | Checked automatically | Not in cards |
Science olympiads
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| HiPhO Models solve theory problems from 2024 and 2025 physics olympiad exams, some with diagrams. Official marking schemes grade answers and steps, mapped to medal levels. | Competition | Graded by a model | 78.3% | In use |
| PHYBench Models solve 500 original physics problems from high school to Physics Olympiad level. Answers are expressions checked against the key. | Competition | Checked automatically | 77.4% | Fading |
| OlympicArena Models solve 11,163 bilingual problems from 62 international olympiad competitions in seven fields, some with images. Answers are checked against keys. | Competition | Checked automatically | 40.5% | Dropped |
| FrontierScience-Olympiad Models solve olympiad-style physics, chemistry and biology problems at the level of IPhO, IChO and IBO, written by medalists and coaches. Graded on short final answers. | Competition | Checked automatically | 77.1% | System cards only |
| IPhO 2025 Models solve the theory problems of the 2025 International Physics Olympiad. Answers are graded against the official marking scheme. | Competition | Graded by people | 79.3% | Fading |
| IChO 2025 Models solve the written theoretical problems of the 57th International Chemistry Olympiad, held in Dubai in July 2025. Answers are graded against the official marking scheme. | Competition | Graded by people | Fading | |
| USABO Semifinal Exam 2020 Models take the 2020 USA Biology Olympiad semifinal exam. Scored out of 150 points. | Competition | Checked automatically | 87.0 | System cards only |
| USNCO Local Section Exam 2022 Models take the 2022 US National Chemistry Olympiad local section exam. Scored out of 60 questions. | Competition | Checked automatically | 36.0 | System cards only |
Research science
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Terminal-Bench-Science 70 scientific research-workflow tasks across five science areas, run in Harbor containers. Hidden tests give the resolution rate. | Expert | Checked automatically | 68.1% | In use |
| CritPt 71 frontier physics research challenges with checkable answers. Anthropic uses a corrected internal version. | Research frontier | Checked automatically | 83.1% | In use |
| Surface Evolver Bench Agents write Surface Evolver datafiles that model liquid surfaces under physical constraints across 16 tasks. Deterministic checks give partial credit for matching hidden reference quantities. | Expert | Checked automatically | 95% | Not in cards |
| ResearchClawBench Agents get the literature and raw data behind a published paper, with the paper hidden, and try to rediscover its results. Expert rubrics with weighted criteria grade the outputs. | Research frontier | Graded by a model | 38.6 | Not in cards |
| Verifying the Verifiers (Project APE CRED benchmark) Verifier LLMs must find planted errors (prose-table, paper-code, execution, data) in AI-generated economics papers. Credit needs a verbatim quote of the edit. | Expert | Checked automatically | 99.5% | Not in cards |
| FrontierPhysics Agents do end-to-end physics research on 56 tasks that 42 physicists from 26 institutions built from their own work, across nine domains of theoretical, computational and experimental physics. Each task asks the agent to review the literature, plan, run the research and write a manuscript. Deterministic verifiers and expert rubrics grade whether the research is correct and publishable, over 3 trials. | Expert | Checked automatically | 27.4% | Not in cards |
| BixBench3 Agents rebuild the full computational analysis of 20 published biology papers from raw data on a cloud VM. Output tables are graded against the papers' own artifacts. | Research frontier | Checked automatically | 52.3 | Not in cards |
| NatureBench Coding agents work on 90 tasks distilled from Nature-family papers, each in its own container, and try to match or beat the published results. | Research frontier | Checked automatically | 26.7% | Not in cards |
| CORE-Bench Agents reproduce the results of published studies from their code and data. 270 tasks from 90 papers in computer science, social science and medicine come in three difficulty levels. | Expert | Checked automatically | 77.8% | Not in cards |
| SGI-Bench Models work through more than 1,000 expert samples in four scientist tasks: deep research, idea generation, dry and wet experiments, and experimental reasoning. | Expert | Graded by a model | 33.8% | In use |
| FrontierScience Models solve physics, chemistry, and biology problems in an Olympiad track graded by short answer and a Research track graded by 10-point rubrics. | Expert | Graded by a model |  87% | In use |
| CMT-Benchmark Models solve 50 research-level condensed matter theory problems written and checked by a panel of expert physicists. Answers are graded automatically. | Research frontier | Checked automatically | 30% | Fading |
| FrontierScience-Research Models work on open-ended PhD-level research subtasks in physics, chemistry and biology, written by working scientists. A grader scores each answer against a 10-point rubric. | Expert | Graded by a model | 38% | Fading |
| AstaBench Agents work through more than 2,400 scientific research problems in 11 benchmarks across four areas: literature understanding, code and execution, data analysis, and end-to-end discovery. Many problems come from real requests to Ai2's deployed Asta agents. Agents use a fixed set of production-grade search tools, and the leaderboard plots score against cost per problem. | Expert | Graded by a model | 58% | Not in cards |
| CritPt v2 Models solve unpublished research-level physics challenges written by working physicists, with answers graded automatically. The v2 release revised the public test challenges. | Research frontier | Checked automatically | 83.1% | In use |
| SciPredict Models predict the outcomes of 405 real physics, biology and chemistry experiments published after March 31, 2025, with and without expert background knowledge. | Expert | Checked automatically | 25.3% | Not in cards |
| ASI-Bench AI systems try to explore unknown problems, create new knowledge and turn new ideas into verifiable results, rather than apply learned knowledge. | Research frontier | Checked automatically | 55.8 | Not in cards |
| FrontierChallenge Agents complete end-to-end scientific workflows in chemistry, materials, molecular dynamics and life science from fixed inputs, producing a set of required deliverables. Task-specific graders check each deliverable. | Expert | Checked automatically | 20.6% | Not in cards |
| ExplorationBench Agents explore verifiable alien worlds with hidden laws, framing hypotheses and running experiments. The worlds are built so that recalled pretraining knowledge does not help. | Research frontier | Checked automatically | 89% | Not in cards |
| PaperBenchX Agents reproduce published results from 12 research areas outside machine learning, using each field's own scientific software. Verifiers check that the evidence regenerates and supports the paper's claims. | Research frontier | Checked automatically | 14% | Not in cards |
| Into the Unknown Agents get a science paper and an unpublished follow-up direction from a DOE national lab scientist, then write the follow-up paper. Rubrics built from the scientist's conditions grade it, plus an integrity check. | Research frontier | Graded by a model | 0% | Not in cards |
| Science's Last Exam Agents conduct scientific research through tools, code, desktop software, laboratory recordings, and withheld references across 113 tasks in nine disciplines. The official composite equally averages six assessment scores on a zero-to-100 scale. Checks combine reference comparisons, executable tests, and literature audits. | Research frontier | Checked automatically | 44.7 | Not in cards |
| Apodex Discovery Solver systems of model, harness and tools run long, verifiable investigations on high-value problems scouted from 561 industries in 16 sectors. | Research frontier | Checked automatically | 2.80 | Not in cards |
| EurekaBench Agents run long experiments and propose mechanisms that explain observations in six sciences. Mechanisms are scored on known constraints, predictive accuracy and the 306 expert-listed insights they should support. | Research frontier | Graded by a model | 29.8 | Not in cards |
| PhysClosureBench Agents run 50 expert-verified physics research tasks in strong-field and attosecond physics, deciding how to generate and validate evidence. A 20-criterion score and a closure level from RC0 to RC4 grade them. | Research frontier | Graded by a model | 6.29 | Not in cards |
| Organic Chemistry (Anthropic) Models predict structures from spectra, design synthetic routes, predict reaction products and convert between IUPAC names, SMILES and structure images. | Expert | Checked automatically | 86.2% | System cards only |
| Matbench Models predict properties of inorganic materials in 13 supervised tasks built from DFT and experimental data. Scored with error or classification metrics per task. | Expert | Checked automatically | 75.0 | Fading |
| SciReasoner evaluation suite Models perform scientific sequence and text tasks such as format translation, extraction, property prediction and molecule design, drawn from up to 103 tasks. | Expert | Checked automatically | 63.3% | Fading |
| Biomedical image analysis (Anthropic) Models run quantitative analyses of CT, MRI, histopathology and other images from public competitions. Scores are scaled to the best competition results. | Expert | Checked automatically | 72.2% | System cards only |
| De novo protein binder design (Anthropic) Agents get a target protein structure and 24 hours to design 30 miniprotein binders. The evaluation covers 15 targets. | Expert | Checked automatically | 85.1% | System cards only |
| Medicinal Chemistry (Anthropic) Models see several real drug-discovery compounds and pick which had the highest or lowest measured ADME property. The set has 504 questions. | Expert | Checked automatically | 65.3% | System cards only |
| Morphology-to-molecule matching (Anthropic) Models match four sets of multi-channel liver cell images to the four shuffled molecule and dose treatments that produced them. | Expert | Checked automatically | 34% | System cards only |
| Phylogenetics (Anthropic) Models analyze phylogenetic data and tree diagrams, testing quantitative reasoning about tree structure and visual interpretation. | Expert | Checked automatically | 79.6% | System cards only |
| ScholarCatalyst Given a research question, retrieve the 'catalyst papers' that 184 lead authors labeled for 207 projects, searching only prior literature. Recall@20. | Expert | Checked automatically | 53% | Not in cards |
| Organic Chemistry V2 (Anthropic) Models predict structures from spectra, design multi-step syntheses, predict reaction products and read chemical structure images. V2 adds harder expert-recommended problems. | Expert | Checked automatically | 69.2% | System cards only |
| Structural Biology (Anthropic) Models answer questions about a biomolecule's function from structural data and basic tools, in multiple-choice and open-ended variants. | Expert | Checked automatically | 79% | System cards only |
| SuperMatBench Models answer materials chemistry questions in a QA part and solve materials tasks with tools in an Agent part. | Expert | Checked automatically | 51.2% | In use |
| Protein Design (Anthropic) Models generate novel protein sequences under design constraints and rank protein design libraries for experimental testing. | Expert | Checked automatically | 42.5% | System cards only |
| STEM Professor Bench Agents solve unpublished research problems written by professional scientists. Most problems concern theoretical physics, with some partial credit. | Research frontier | Graded by a model | 61.8% | System cards only |
| Anthropic internal life-sciences evals Internal suite: morphology-to-molecule matching, medicinal chemistry, protein design, binder design, biomedical image analysis, organic chemistry. | Expert | Checked automatically | Not in cards | |
| ArXivPhys Models answer research-level physics questions derived from recent arXiv papers, each with a reference final answer. | Research frontier | Checked automatically | Not in cards | |
| ProteinGym Deep mutational scanning benchmarks. The model ranks mutations by predicted effect on protein function, scored by Spearman rank correlation with lab measurements. Anthropic reports a hard subset, as the correlation times 100. | Research frontier | Checked automatically | System cards only | |
| ScienceAgentBench Agents write programs for 102 data-driven discovery tasks from 44 papers in four disciplines. Outputs are checked by execution and task-specific metrics. | Expert | Checked automatically | Not in cards | |
| SciTS Models understand and generate scientific time series in 43 tasks across 12 domains. Signals range from one to ten million points. Each task uses its own metric. | Expert | Checked automatically | System cards only | |
| TheoPhysEval Agents solve 50 unpublished research-level theoretical physics problems on their own, with literature search and scripts. Each final answer is checked against an expert answer, and the score is the pass rate. | Research frontier | Checked automatically | Not in cards |
Lab and data work
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| BioMysteryBench Bioinformatics problems on anonymized real datasets that require real analysis with tools. Rubric-graded, with human-solvable and human-difficult splits. | Expert | Checked automatically | 83.7% | In use |
| LatchBio Capabilities v1.0 LatchBio cross-benchmark suite of agentic biological data analysis (transcriptomics, epigenomics, variants, pharmacology) with deterministic graders. | Expert | Checked automatically | 58% | System cards only |
| scBench (LatchBio SingleCellBench) 195 verifiable single-cell RNA-seq analysis problems. Each pairs a data snapshot with a task and a deterministic grader. | Expert | Checked automatically | 63.9% | System cards only |
| SpatialBench (LatchBio) Agentic spatial-transcriptomics data analysis problems with deterministic graders. Anthropic reports a 115-problem Verified variant. | Expert | Checked automatically | 57.6% | System cards only |
| LABBench2 11 biology research sub-tasks (literature, figures, tables, patents, trials and more) with tool access. Macro-averaged accuracy. | Expert | Checked automatically | 60.9% | In use |
| SciUniverse 92 tasks in 17 chemistry, biology and materials families run in C5R's lab or its digital twin. Pass@1 averaged equally across families. | Expert | Checked automatically | 45.3% | Not in cards |
| BixBench Computational-biology analysis tasks (differential expression, assembly QC, cell typing) answered by agents with tools. | Expert | Checked automatically | 93.8% | Fading |
| BioPipelineBench (Anthropic) Agents execute bioinformatics workflows such as long-read sequence analysis, metagenome assembly and chromatin profiling with a bash tool. | Expert | Checked automatically | 87.7% | System cards only |
| MaCBench Vision language models answer chemistry and materials questions from images. Tasks cover reading data, understanding experiments and interpreting results. | Expert | Checked automatically | 72.6% | Not in cards |
| BioPipelineBench Verified (Anthropic) Agents execute bioinformatics workflows with a bash tool. Only problems that passed a validation check by external reviewers are kept. | Expert | Checked automatically | 83.6% | System cards only |
| ScienceWorld Agents carry out elementary-school science tasks, such as measuring conductivity or growing plants, by acting in an interactive text environment. Each task scores progress toward its goal. | School | Checked automatically | Not in cards |
Abstract puzzles and logic
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| DTBench Models answer 407 multiple-choice questions on decision theory in Newcomb-like problems, where choices predict the behavior of copies. Scored by accuracy. | Expert | Checked automatically | 99.2% | System cards only |
| ARC-AGI-2 Models infer a rule from example grid pairs and produce the output grid for new inputs. Scored by tasks solved within two attempts. | Everyday | Checked automatically | 95% | In use |
| ARC-AGI Few-shot grid puzzles: infer a hidden transformation from about three examples and produce the output grid. Semi-private sets verified by ARC Prize. | Everyday | Checked automatically | 98.5% | In use |
| SimpleBench Models answer over 200 multiple-choice trick questions on spatial, temporal, and social reasoning that ordinary people solve. Scored by average accuracy over five runs. | Everyday | Checked automatically | 94.9% | Not in cards |
| ZebraLogic Models solve logic grid puzzles of controlled size, derived from constraint satisfaction problems. Scored by the share of puzzles solved fully. | School | Checked automatically | 97.3% | Fading |
| Misguided Attention Models get slight variations of famous riddles, paradoxes and thought experiments. A model fails when it answers the well-known original instead of the problem as stated. | Everyday | Graded by a model | 73.7% | Not in cards |
| KOR-Bench Models read a new rule set and apply it to questions in five task types: operation, logic, cipher, puzzle and counterfactual. Scored by accuracy against an answer key. | University | Checked automatically | 72.9% | Fading |
| MysteryMechanism Agents recover a hidden mathematical law from two noisy observations and a few experiments they choose. The submitted expression is run on private probes and passes if its error is under a noise-aware threshold. | Expert | Checked automatically | 53.1% | Not in cards |
| General365 Models solve 365 hand-written reasoning problems and 1,095 variants in eight categories that need only K-12 knowledge. Scoring mixes rule-based and model-based checks. | Everyday | Checked automatically | 62.8% | Not in cards |
| 3SAT-Search Models find satisfying assignments for 960 3-SAT formulas across difficulty levels. A checker verifies each assignment. | University | Checked automatically | 96.2% | System cards only |
| TSP-Opt Models find optimal tours for 960 traveling salesman instances across 8 difficulty levels. Answers are checked for optimality. | University | Checked automatically | 56.4% | System cards only |
| AutoLogi Models solve open-ended logic puzzles in English and Chinese. Generated verifier programs check each answer against the puzzle constraints. | School | Checked automatically | 89.5% | Dropped |
| BA-Calendar Models find a common meeting slot for several people under constraints such as time zones, buffer time and priority. 2,000 problems checked by code. | Everyday | Checked automatically |  67.7% | System cards only |
| Novel logical reasoning problems expert evaluation (Z.ai) Human experts score model answers to new multi-step logic problems with a fixed scoring standard for each question. | University | Graded by people | 62.0 | System cards only |
| ProcBench Models follow explicit step-by-step procedures to answer questions, which isolates multi-step inference from knowledge and search. | Everyday | Checked automatically | 96.6% | Fading |
| Sudoku-Bench Models solve Sudoku variants with unusual rule sets that need creative, multi-step deduction. Scored by the share of puzzles solved. | Competition | Checked automatically |  56% | Dropped |
| DeepPHY Agentic vision-language models act in simulated physics environments of varied difficulty that test understanding of basic physical principles. | Everyday | Checked automatically | Not in cards |