Benchmark Almanac

CritPt v2

GPT-6 Astra 83.1% leads since October 2026
GPT-6 Astra 83.1% leads since October 2026
In use
Version 2

The official leaderboard hasn’t scored a model released after DeepSeek V4.1 Flash 1017 (October 2026). Labs report their own numbers for 2 newer models, such as Inkling 2 Small at 55.7%.

Models solve unpublished research-level physics challenges written by working physicists, with answers graded automatically. The v2 release revised the public test challenges.

Scores are averaged over 5 runs on 70 test challenges. The v1 board listed GPT-5 (high) at 5.7 percent, and the v2 board lists GPT-5 at 28.9 percent.

Released
October 2026
Kind
Research science
Level
Research frontier
Scored
Checked automatically
Answers
Short answer
Tasks
Public, plus a private test set
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Single turn
Items from
Written by experts
Run by
A host
Based on
CritPt
Strongest model tested
GPT-6 Astra #4 on the capability index
Last in a launch post
November 2026
Last model added
November 2026
Scores refreshed
November 19th, 2026

In the authors’ words

While large language models (LLMs) with reasoning capabilities are progressing rapidly on high-school math competitions and coding, can they reason effectively through complex, open-ended challenges found in frontier physics research? And crucially, what kinds of reasoning tasks do physicists want LLMs to assist with? To address these questions, we present the CritPt (Complex Research using Integrated Thinking - Physics Test, pronounced "critical point"), the first benchmark designed to test LLMs on unpublished, research-level reasoning tasks that broadly covers modern physics research areas, including condensed matter, quantum physics, atomic, molecular & optical physics, astrophysics, high energy physics, mathematical physics, statistical physics, nuclear physics, nonlinear dynamics, fluid dynamics and biophysics. CritPt consists of 71 composite research challenges designed to simulate full-scale research projects at the entry level, which are also decomposed to 190 simpler checkpoint tasks for more fine-grained insights. All problems are newly created by 50+ active physics researchers based on their own research. Every problem is hand-curated to admit a guess-resistant and machine-verifiable answer and is evaluated by an automated grading pipeline heavily customized for advanced physics-specific output formats. We find that while current state-of-the-art LLMs show early promise on isolated checkpoints, they remain far from being able to reliably solve full research-scale challenges: the best average accuracy among base models is only 5.7%, achieved by GPT-5 (high), moderately rising to around 10% when equipped with coding tools. Through the realistic yet standardized evaluation offered by CritPt, we highlight a large disconnect between current model capabilities and realistic physics research demands, offering a foundation to guide the development of scientifically grounded AI tools.

Abstract of Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark (arXiv:2509.26574, CC0)
11
Models scored on its own leaderboard
2
Launch posts list it, from 1 lab
1
System card reports it, from 1 lab

GPT-6 Astra leads with 83.1%

Best score per model from the benchmark's own leaderboard, read on November 19th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

GPT-6 Astra leads with 83.1%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

  1. 1 GPT-6 Astra 83.1%
  2. 2 GPT-6.1 Sol 82%
  3. 3 Claude Opus 5.5 77.7%
  4. 4 GPT-5.6 Sol 74.9%
  5. 5 Muse Wonder Full 73.1%
  6. 6 Gemini 4 Argon 72.9%
  7. 7 Claude Fable 5.1 72%
  8. 8 Muse Wonder 69.7%
  9. 9 Grok 4.7 50.6%
  10. 10 DeepSeek V4.1 Flash 1017 50%
  11. 11 GPT-5 28.9%

Source: Official CritPt v2 leaderboard

#ModelScoreSettingDateSource
1 GPT-6 Astra 83.1% Sep 2026 Official leaderboard
2 GPT-6.1 Sol 82% Sep 2026 Official leaderboard
3 Claude Opus 5.5 77.7% Sep 2026 Official leaderboard
4 GPT-5.6 Sol 74.9% Jul 2026 Official leaderboard
5 Muse Wonder Full 73.1% Oct 2026 Official leaderboard
6 Gemini 4 Argon 72.9% Sep 2026 Official leaderboard
7 Claude Fable 5.1 72% Sep 2026 Official leaderboard
8 Muse Wonder 69.7% Oct 2026 Official leaderboard
9 Grok 4.7 50.6% Sep 2026 Official leaderboard
10 DeepSeek V4.1 Flash 1017 50% Oct 2026 Official leaderboard
11 GPT-5 28.9% Aug 2025 Official leaderboard
#ModelScoreSettingDateSource
1 GPT-6 Astra 83.1% Sep 2026 Official leaderboard
2 GPT-6.1 Sol 82% Sep 2026 Official leaderboard
3 Claude Opus 5.5 77.7% Sep 2026 Official leaderboard
4 GPT-5.6 Sol 74.9% Jul 2026 Official leaderboard
5 Muse Wonder Full 73.1% Oct 2026 Official leaderboard
6 Gemini 4 Argon 72.9% Sep 2026 Official leaderboard
7 Claude Fable 5.1 72% Sep 2026 Official leaderboard
8 Muse Wonder 69.7% Oct 2026 Official leaderboard
9 Inkling 2 Small 55.7% Nov 2026 Lab model card
10 Grok 4.7 50.6% Sep 2026 Official leaderboard
11 DeepSeek V4.1 Flash 1017 50% Oct 2026 Official leaderboard
12 Inkling 2 Mini 31.4% Nov 2026 Lab model card
13 GPT-5 28.9% Aug 2025 Official leaderboard

Labs report scores for 2 models in their cards. Switch the scores above to rank them together.

The best score, 83.1%, costs $0.19 a task

Runs on the cost frontier score higher than any run that's cheaper. GPT-6.1 Sol comes within 1.1 points of the best for $0.04 a task, 5 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.02 $0.05 $0.10 $0.20 $0.50 $1.00 Cost per task, log scale GPT-6 Astra GPT-6.1 Sol 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 Cost per task, log scale GPT-6 Astra GPT-6.1 Sol

Source: Official CritPt v2 leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-6 Astra 83.1% $0.19
GPT-6.1 Sol 82% $0.04
Claude Opus 5.5 77.7% $0.38
GPT-5.6 Sol 74.9% $0.14
Muse Wonder Full 73.1%
Gemini 4 Argon 72.9% $0.18
Claude Fable 5.1 72% $0.75
Muse Wonder 69.7% $0.31
Grok 4.7 50.6% $0.44
DeepSeek V4.1 Flash 1017 50% $0.04
GPT-5 28.9% $0.07

Number one over time Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

OpenAI Thinking Machines
Own leaderboard 1 leader
GPT-6 Astra 83.1%
Lab claims 1 leader

Sources: Official CritPt v2 leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) GPT-6 Astra GPT-5.6 Sol GPT-5 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) GPT-6 Astra GPT-5.6 Sol

Sources: Official CritPt v2 leaderboard and labs’ model cards

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Thinking Machines
DateLaunch postScoreAs the post writes it
Nov 20, 2026 Inkling 2 Mini 31.4 CritPt v2 (Full, 17.0 on Hard)
Nov 5, 2026 Inkling 2 Small 55.7 CritPt v2 (Full; 35.0 on Hard)

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Nov 20, 2026 Inkling 2 Mini model card 31.4 CritPt v2 (Full, 17.0 on Hard) Capabilities

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

OpenAI
CritPt v2 4 leaders
GPT-6 Astra 83.1%
CritPt 3 leaders
GPT-6 Astra 83.1%
CritPt v2 1 leader
GPT-6 Astra 83.1%

Source: Official leaderboards

More in Science and reasoning

All 203 in Science and reasoning
  • GPQA Diamond In use 198 graduate-level 'Google-proof' multiple-choice questions in biology, chemistry and physics. Gemini 4 Carbon 97%
  • Terminal-Bench-Science In use 70 scientific research-workflow tasks across five science areas, run in Harbor containers. Hidden tests give the resolution rate. GPT-6 Astra 68.1%
  • BioMysteryBench In use Bioinformatics problems on anonymized real datasets that require real analysis with tools. Rubric-graded, with human-solvable and human-difficult splits. Claude Fable 5.2 83.7%
  • FrontierScience In use Models solve physics, chemistry, and biology problems in an Olympiad track graded by short answer and a Research track graded by 10-point rubrics. Intern-S2-397B 87%
  • HiPhO In use Models solve theory problems from 2024 and 2025 physics olympiad exams, some with diagrams. Official marking schemes grade answers and steps, mapped to medal levels. Gemini 3 Pro 78.3%
  • LABBench2 In use 11 biology research sub-tasks (literature, figures, tables, patents, trials and more) with tool access. Macro-averaged accuracy. GPT-5.2 60.9%

Data audit

All 11 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access Public tasks, plus a private test set.
  • Pass Confidence Checked against a primary source.
Scores 5 pass
  • Pass Independent runs Scores from the official leaderboard (11 entries), launch posts (2 scores) and system cards (1 score).
  • Pass Models scored 13 models we track.
  • Pass Newest score The newest is from November 2026, for Inkling 2 Mini.
  • Pass Frontier models 5 of the top 10 have a score, led by GPT-6 Astra (#4).
  • Pass Headroom The best is 83.1%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on github.com: 11 rows for models we track at the last read.
  • Pass Last read November 19th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check