CritPt v2
Version 2
The official leaderboard hasn’t scored a model released after DeepSeek V4.1 Flash 1017 (October 2026). Labs report their own numbers for 2 newer models, such as Inkling 2 Small at 55.7%.
Models solve unpublished research-level physics challenges written by working physicists, with answers graded automatically. The v2 release revised the public test challenges.
Scores are averaged over 5 runs on 70 test challenges. The v1 board listed GPT-5 (high) at 5.7 percent, and the v2 board lists GPT-5 at 28.9 percent.
- Made by
- CritPt collaboration
- Released
- October 2026
- Kind
- Research science
- Level
- Research frontier
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- Public, plus a private test set
- Measures
- Capability
- Subject
- Physical sciences
- Language
- English
- Interaction
- Single turn
- Items from
- Written by experts
- Refresh
- New versions replace it
- Run by
- A host
- Based on
- CritPt
- Strongest model tested
- GPT-6 Astra #4 on the capability index
- Last in a launch post
- November 2026
- Last model added
- November 2026
- Scores refreshed
- November 19th, 2026
In the authors’ words
While large language models (LLMs) with reasoning capabilities are progressing rapidly on high-school math competitions and coding, can they reason effectively through complex, open-ended challenges found in frontier physics research? And crucially, what kinds of reasoning tasks do physicists want LLMs to assist with? To address these questions, we present the CritPt (Complex Research using Integrated Thinking - Physics Test, pronounced "critical point"), the first benchmark designed to test LLMs on unpublished, research-level reasoning tasks that broadly covers modern physics research areas, including condensed matter, quantum physics, atomic, molecular & optical physics, astrophysics, high energy physics, mathematical physics, statistical physics, nuclear physics, nonlinear dynamics, fluid dynamics and biophysics. CritPt consists of 71 composite research challenges designed to simulate full-scale research projects at the entry level, which are also decomposed to 190 simpler checkpoint tasks for more fine-grained insights. All problems are newly created by 50+ active physics researchers based on their own research. Every problem is hand-curated to admit a guess-resistant and machine-verifiable answer and is evaluated by an automated grading pipeline heavily customized for advanced physics-specific output formats. We find that while current state-of-the-art LLMs show early promise on isolated checkpoints, they remain far from being able to reliably solve full research-scale challenges: the best average accuracy among base models is only 5.7%, achieved by GPT-5 (high), moderately rising to around 10% when equipped with coding tools. Through the realistic yet standardized evaluation offered by CritPt, we highlight a large disconnect between current model capabilities and realistic physics research demands, offering a foundation to guide the development of scientifically grounded AI tools.
GPT-6 Astra leads with 83.1%
Best score per model from the benchmark's own leaderboard, read on November 19th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.
GPT-6 Astra leads with 83.1%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 1 GPT-6 Astra 83.1%
- 2 GPT-6.1 Sol 82%
- 3 Claude Opus 5.5 77.7%
- 4 GPT-5.6 Sol 74.9%
- 5
Muse Wonder Full 73.1%
- 6
Gemini 4 Argon 72.9%
- 7 Claude Fable 5.1 72%
- 8
Muse Wonder 69.7%
- 9 Grok 4.7 50.6%
- 10
DeepSeek V4.1 Flash 1017 50%
- 11 GPT-5 28.9%
Source: Official CritPt v2 leaderboard
- 1 GPT-6 Astra 83.1%
- 2 GPT-6.1 Sol 82%
- 3 Claude Opus 5.5 77.7%
- 4 GPT-5.6 Sol 74.9%
- 5
Muse Wonder Full 73.1%
- 6
Gemini 4 Argon 72.9%
- 7 Claude Fable 5.1 72%
- 8
Muse Wonder 69.7%
- 9
Inkling 2 Small 55.7%
- 10 Grok 4.7 50.6%
- 11
DeepSeek V4.1 Flash 1017 50%
- 12
Inkling 2 Mini 31.4%
- 13 GPT-5 28.9%
Sources: Official CritPt v2 leaderboard and labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 83.1% | Sep 2026 | Official leaderboard | |
| 2 | GPT-6.1 Sol | 82% | Sep 2026 | Official leaderboard | |
| 3 | Claude Opus 5.5 | 77.7% | Sep 2026 | Official leaderboard | |
| 4 | GPT-5.6 Sol | 74.9% | Jul 2026 | Official leaderboard | |
| 5 | | 73.1% | Oct 2026 | Official leaderboard | |
| 6 | | 72.9% | Sep 2026 | Official leaderboard | |
| 7 | Claude Fable 5.1 | 72% | Sep 2026 | Official leaderboard | |
| 8 | | 69.7% | Oct 2026 | Official leaderboard | |
| 9 | Grok 4.7 | 50.6% | Sep 2026 | Official leaderboard | |
| 10 | | 50% | Oct 2026 | Official leaderboard | |
| 11 | GPT-5 | 28.9% | Aug 2025 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 83.1% | Sep 2026 | Official leaderboard | |
| 2 | GPT-6.1 Sol | 82% | Sep 2026 | Official leaderboard | |
| 3 | Claude Opus 5.5 | 77.7% | Sep 2026 | Official leaderboard | |
| 4 | GPT-5.6 Sol | 74.9% | Jul 2026 | Official leaderboard | |
| 5 | 73.1% | Oct 2026 | Official leaderboard | ||
| 6 | 72.9% | Sep 2026 | Official leaderboard | ||
| 7 | Claude Fable 5.1 | 72% | Sep 2026 | Official leaderboard | |
| 8 | 69.7% | Oct 2026 | Official leaderboard | ||
| 9 | 55.7% | Nov 2026 | Lab model card | ||
| 10 | Grok 4.7 | 50.6% | Sep 2026 | Official leaderboard | |
| 11 | 50% | Oct 2026 | Official leaderboard | ||
| 12 | 31.4% | Nov 2026 | Lab model card | ||
| 13 | GPT-5 | 28.9% | Aug 2025 | Official leaderboard |
Labs report scores for 2 models in their cards. Switch the scores above to rank them together.
The best score, 83.1%, costs $0.19 a task
Runs on the cost frontier score higher than any run that's cheaper. GPT-6.1 Sol comes within 1.1 points of the best for $0.04 a task, 5 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
Source: Official CritPt v2 leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| GPT-6 Astra | 83.1% | $0.19 | |||
| GPT-6.1 Sol | 82% | $0.04 | |||
| Claude Opus 5.5 | 77.7% | $0.38 | |||
| GPT-5.6 Sol | 74.9% | $0.14 | |||
| 73.1% | |||||
| 72.9% | $0.18 | ||||
| Claude Fable 5.1 | 72% | $0.75 | |||
| 69.7% | $0.31 | ||||
| Grok 4.7 | 50.6% | $0.44 | |||
| 50% | $0.04 | ||||
| GPT-5 | 28.9% | $0.07 |
Number one over time Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Sources: Official CritPt v2 leaderboard and labs’ model cards
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official CritPt v2 leaderboard and labs’ model cards
One lab put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Nov 20, 2026 | 31.4 | CritPt v2 (Full, 17.0 on Hard) | |
| Nov 5, 2026 | 55.7 | CritPt v2 (Full; 35.0 on Hard) |
One system card reports it
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Nov 20, 2026 | | 31.4 | CritPt v2 (Full, 17.0 on Hard) | Capabilities |
Versions
We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.
Source: Official leaderboards
More in Science and reasoning
- GPQA Diamond In use 198 graduate-level 'Google-proof' multiple-choice questions in biology, chemistry and physics.
Gemini 4 Carbon 97%
- Terminal-Bench-Science In use 70 scientific research-workflow tasks across five science areas, run in Harbor containers. Hidden tests give the resolution rate. GPT-6 Astra 68.1%
- BioMysteryBench In use Bioinformatics problems on anonymized real datasets that require real analysis with tools. Rubric-graded, with human-solvable and human-difficult splits. Claude Fable 5.2 83.7%
- FrontierScience In use Models solve physics, chemistry, and biology problems in an Olympiad track graded by short answer and a Research track graded by 10-point rubrics.
Intern-S2-397B 87%
- HiPhO In use Models solve theory problems from 2024 and 2025 physics olympiad exams, some with diagrams. Official marking schemes grade answers and steps, mapped to medal levels.
Gemini 3 Pro 78.3%
- LABBench2 In use 11 biology research sub-tasks (literature, figures, tables, patents, trials and more) with tool access. Macro-averaged accuracy. GPT-5.2 60.9%
Data audit
All 11 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access Public tasks, plus a private test set.
- Pass Confidence Checked against a primary source.
Scores 5 pass
- Pass Independent runs Scores from the official leaderboard (11 entries), launch posts (2 scores) and system cards (1 score).
- Pass Models scored 13 models we track.
- Pass Newest score The newest is from November 2026, for Inkling 2 Mini.
- Pass Frontier models 5 of the top 10 have a score, led by GPT-6 Astra (#4).
- Pass Headroom The best is 83.1%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on github.com: 11 rows for models we track at the last read.
- Pass Last read November 19th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.