Benchmark Almanac

HLE-Diamond

Claude Fable 5.2 77.9% leads since November 2026
Claude Fable 5.2 77.9% leads since November 2026
In use 1 known issue
Version Diamond

The official leaderboard hasn’t scored a model released after GPT-6.2 Astra (November 2026). Labs report their own numbers for 2 newer models, such as Gemini 4 Neon at 71.3%.

Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers.

The makers built it after a year of cleaning as a migration path for when the original set hits its noise ceiling.

Released
September 2026
Kind
Expert-written questions
Level
Research frontier
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Images → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
GPT-6.2 Astra #1 on the capability index
Last in a launch post
November 2026
Last model added
November 2026
Scores refreshed
November 18th, 2026

Known issues

A published audit found a problem in HLE-Diamond. We summarize each finding in our words and link its source. The scores on this page don't include any fixes. See all audits

  1. Leaked answers Reward hacking Still open

    Prime Intellect reports that HLE-Diamond answers leak through mirrors and public rollout logs, so domain blocklists do not stop agents from finding them. In one trace, DeepSeek V4.1-Flash 1017 read a graded answer through OpenReward's public API and submitted it.

    DeepSeek V4.1-Flash 1017 dropped its own answer (0.070) and submitted the leaked answer 0.011 with 98% confidence.

    It went through OpenReward’s public rollout API, found another model’s trajectory on the same question, and read the grader’s verdict: Correct Answer: 0.011.

    Learnings from Auditing Public Evaluations by Prime Intellect, November 3rd, 2026

22
Models scored on its own leaderboard
5
Launch posts list it, from 3 labs
2
System cards report it, from 2 labs

Claude Fable 5.2 leads with 77.9%

Best score per model from the benchmark's own leaderboard, read on November 18th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

Claude Fable 5.2 leads with 77.9%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.

  1. 1 Claude Fable 5.2 max 77.9%
  2. 2 GPT-6.2 Astra max 72.3%
  3. 3 GPT-6 Astra max 65.8%
  4. 4 Gemini 4 Carbon high 63.6%
  5. 5 GPT-6.2 Sol max 62.7%
  6. 6 GPT-6.1 Sol max 61.3%
  7. 7 Claude Opus 5.5 max 60.9%
  8. 8 Gemini 4 Argon high 56.9%
  9. 9 Claude Fable 5.1 max 55.1%
  10. 10 Claude Sonnet 5.5 max 43.3%
  11. 11 GPT-6 Sol max 43.2%
  12. 12 GPT-5.6 Sol max 41%
  13. 13 Muse Wonder max 40.6%
  14. 14 Claude Opus 5 max 40.5%
  15. 15 Kimi K3.1 Pro max 38.7%
  16. 16 Muse Spark 1.3 max 34.6%
  17. 17 Gemini 3.8 Flash high 33.3%
  18. 18 Kimi K3 max 28.3%
  19. 19 Muse Spark Open max 28.2%
  20. 20 GLM-5.3 max 25.1%
  21. 21 Grok 4.7 max 25%
  22. 22 DeepSeek-V4-Pro max 16.6%

Source: Official HLE-Diamond leaderboard

#ModelScoreSettingDateSource
1 Claude Fable 5.2 77.9% max Nov 2026 Official leaderboard
2 GPT-6.2 Astra 72.3% max Nov 2026 Official leaderboard
3 GPT-6 Astra 65.8% max Sep 2026 Official leaderboard
4 Gemini 4 Carbon 63.6% high Oct 2026 Official leaderboard
5 GPT-6.2 Sol 62.7% max Nov 2026 Official leaderboard
6 GPT-6.1 Sol 61.3% max Sep 2026 Official leaderboard
7 Claude Opus 5.5 60.9% max Sep 2026 Official leaderboard
8 Gemini 4 Argon 56.9% high Sep 2026 Official leaderboard
9 Claude Fable 5.1 55.1% max Sep 2026 Official leaderboard
10 Claude Sonnet 5.5 43.3% max Sep 2026 Official leaderboard
11 GPT-6 Sol 43.2% max Sep 2026 Official leaderboard
12 GPT-5.6 Sol 41% max Jul 2026 Official leaderboard
13 Muse Wonder 40.6% max Oct 2026 Official leaderboard
14 Claude Opus 5 40.5% max Jul 2026 Official leaderboard
15 Kimi K3.1 Pro 38.7% max Oct 2026 Official leaderboard
16 Muse Spark 1.3 34.6% max Sep 2026 Official leaderboard
17 Gemini 3.8 Flash 33.3% high Sep 2026 Official leaderboard
18 Kimi K3 28.3% max Jul 2026 Official leaderboard
19 Muse Spark Open 28.2% max Oct 2026 Official leaderboard
20 GLM-5.3 25.1% max Aug 2026 Official leaderboard
21 Grok 4.7 25% max Sep 2026 Official leaderboard
22 DeepSeek-V4-Pro 16.6% max Apr 2026 Official leaderboard
#ModelScoreSettingDateSource
1 Claude Fable 5.2 77.9% max Nov 2026 Official leaderboard
2 GPT-6.2 Astra 72.3% max Nov 2026 Official leaderboard
3 Gemini 4 Neon 71.3% Nov 2026 Lab model card
4 GPT-6 Astra 65.8% max Sep 2026 Official leaderboard
5 Gemini 4 Carbon 63.6% high Oct 2026 Official leaderboard
6 GPT-6.2 Sol 62.7% max Nov 2026 Official leaderboard
7 GPT-6.1 Sol 61.3% max Sep 2026 Official leaderboard
8 Claude Opus 5.5 60.9% max Sep 2026 Official leaderboard
9 Gemini 4 Hydrogen 58.8% Nov 2026 Lab model card
10 Gemini 4 Argon 56.9% high Sep 2026 Official leaderboard
11 Claude Fable 5.1 55.1% max Sep 2026 Official leaderboard
12 Claude Sonnet 5.5 43.3% max Sep 2026 Official leaderboard
13 GPT-6 Sol 43.2% max Sep 2026 Official leaderboard
14 GPT-5.6 Sol 41% max Jul 2026 Official leaderboard
15 Muse Wonder 40.6% max Oct 2026 Official leaderboard
16 Claude Opus 5 40.5% max Jul 2026 Official leaderboard
17 Kimi K3.1 Pro 38.7% max Oct 2026 Official leaderboard
18 Muse Spark 1.3 34.6% max Sep 2026 Official leaderboard
19 Gemini 3.8 Flash 33.3% high Sep 2026 Official leaderboard
20 Inkling 2 Small 28.7% Nov 2026 Lab model card
21 Kimi K3 28.3% max Jul 2026 Official leaderboard
22 Muse Spark Open 28.2% max Oct 2026 Official leaderboard
23 GLM-5.3 25.1% max Aug 2026 Official leaderboard
24 Grok 4.7 25% max Sep 2026 Official leaderboard
25 DeepSeek-V4-Pro 16.6% max Apr 2026 Official leaderboard
26 Inkling 2 Mini 15.2% Nov 2026 Lab model card

Labs report scores for 5 models in their cards. Switch the scores above to rank them together.

More effort adds a median 6 points

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Fable 5.2
high max
77.9%
GPT-6.2 Astra
high max
72.3%
GPT-6 Astra
high max
65.8%
Gemini 4 Carbon
high
63.6%
GPT-6.2 Sol
high max
62.7%
GPT-6.1 Sol
high max
61.3%
Claude Opus 5.5
high max
60.9%
Gemini 4 Argon
high
56.9%
Claude Fable 5.1
high max
55.1%
Claude Sonnet 5.5
high max
43.3%
GPT-6 Sol
high max
43.2%
GPT-5.6 Sol
high max
41%

Source: Official HLE-Diamond leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.2 max 77.9% 35,481
Claude Fable 5.2 high 74.3% 8,831
GPT-6.2 Astra max 72.3% 15,369
GPT-6.2 Astra high 66.6% 3,346
GPT-6 Astra max 65.8% 11,693
Gemini 4 Carbon high 63.6% 59,164
Gemini 4 Carbon max 63.6% 59,164
GPT-6.2 Sol max 62.7% 13,671
GPT-6.1 Sol max 61.3% 12,383
Claude Opus 5.5 max 60.9% 46,945
GPT-6 Astra high 59.9% 5,172
GPT-6.2 Sol high 58.3% 3,730
Gemini 4 Argon high 56.9% 26,087
Gemini 4 Argon max 56.9% 26,087
Claude Fable 5.1 max 55.1% 41,758
Claude Opus 5.5 high 54.6% 9,614
GPT-6.1 Sol high 53.2% 4,492
Claude Fable 5.1 high 51.6% 14,817
Claude Sonnet 5.5 max 43.3% 66,770
GPT-6 Sol max 43.2% 20,512
GPT-5.6 Sol max 41% 20,037
Muse Wonder max 40.6% 40,112
Claude Opus 5 max 40.5% 34,063
Kimi K3.1 Pro max 38.7% 37,725
Claude Opus 5 high 38.1% 19,656
Muse Wonder high 34.7% 16,125
Muse Spark 1.3 max 34.6% 32,818
Gemini 3.8 Flash high 33.3% 26,074
Gemini 3.8 Flash max 33.3% 26,074
Claude Sonnet 5.5 high 33.1% 8,584
GPT-6 Sol high 32.8% 4,597
GPT-5.6 Sol high 30.4% 5,912
Kimi K3.1 Pro high 29.5% 9,800
Kimi K3 max 28.3% 26,855
Muse Spark Open max 28.2% 22,779
GLM-5.3 max 25.1% 47,012
Grok 4.7 max 25% 57,366
Muse Spark Open high 24.9% 19,691
Muse Spark 1.3 high 24.6% 21,304
Grok 4.7 high 22.8% 43,244
GLM-5.3 high 22.3% 44,311
Kimi K3 high 21.7% 13,075
DeepSeek-V4-Pro max 16.6% 35,296
DeepSeek-V4-Pro high 13.4% 34,683

3 models have held number one on the official leaderboard since its release in September 2026 3 models have held number one with claims included since its release in September 2026

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic OpenAI Meta Thinking Machines Google DeepMind
Own leaderboard 3 leaders
Lab claims 4 leaders

Sources: Official HLE-Diamond leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Fable 5.2 GPT-6 Astra Claude Fable 5.1 GPT-5.6 Sol DeepSeek-V4-Pro 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Fable 5.2 GPT-6 Astra

Sources: Official HLE-Diamond leaderboard and labs’ model cards

3 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Meta
Thinking Machines
Google DeepMind
DateLaunch postScoreAs the post writes it
Nov 20, 2026 Inkling 2 Mini 15.2 HLE-Diamond
Nov 18, 2026 Gemini 4 Neon 71.3 HLE-Diamond
Nov 12, 2026 Gemini 4 Hydrogen 58.8 HLE-Diamond
Nov 5, 2026 Inkling 2 Small 28.7 HLE-Diamond
Oct 15, 2026 Muse Spark Open 26.1 HLE-Diamond (no tools)

2 system cards report it, from 2 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Nov 20, 2026 Inkling 2 Mini model card 15.2 HLE-Diamond Capabilities
Nov 18, 2026 Gemini 4 Neon model card 71.3 HLE-Diamond (results table, p.4) Capabilities

One lab claim next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.

Model Lab says Board says Gap
Muse Spark Open 26.1% HLE-Diamond (no tools) 28.2% Leaderboard, max −2.1 pts

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

OpenAI Anthropic Google DeepMind
HLE-Diamond 15 leaders
GPT-6 Astra 54.8%
HLE-Diamond 3 leaders

Source: Official leaderboards

More in Knowledge

All 332 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • MMLU In use 57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy. Seed1.8 92.3%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed. Gemini 3 Pro 48.2%

Data audit

11 of 12 checks pass, 1 problem. Look first at the Known issues check. 1 open finding: Leaked answers and reward hacking. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 5 pass
  • Pass Independent runs Scores from the official leaderboard (22 entries), launch posts (5 scores) and system cards (2 scores).
  • Pass Models scored 26 models we track.
  • Pass Newest score The newest is from November 2026, for Inkling 2 Mini.
  • Pass Frontier models 9 of the top 10 have a score, led by GPT-6.2 Astra (#1).
  • Pass Headroom The best is 77.9%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on lastexam.ai: 44 rows for models we track at the last read.
  • Pass Last read November 18th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 problem 1 pass
  • Pass Lab claims The one pair agrees within 10 points.
  • Problem Known issues 1 open finding: Leaked answers and reward hacking.

How we check