MentalHealthBench
The benchmark’s paper hasn’t scored a model released after GPT-6 Sol (September 2026). Labs report their own numbers for 4 newer models, such as GPT-6.1 Sol at 57.9%.
1,215 synthetic mental-health conversations with 5,262 clinician-authored rubric criteria. An LLM judge scores safety and guidance quality.
Grader is GPT-5.6 Sol. More than 80 licensed psychologists and psychiatrists from more than 20 countries wrote and adjudicated the criteria. The paper URL for the data (no extension) returns 404, but OAI_MentalHealthBench.zip resolves.
- Made by
- OpenAI
- Released
- September 2026
- Kind
- Wellbeing and sycophancy
- Level
- Expert
- Scored
- Graded by a model
- Answers
- Open-ended
- Tasks
- All public
- Measures
- Capability
- Subject
- Clinical medicine
- Language
- English
- Interaction
- Multi-turn
- Items from
- Generated
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Biology and medicine, Content safety
- Strongest model tested
- GPT-6.2 Astra #1 on the capability index
- Last in a system card
- November 2026
- Last model added
- November 2026
- Scores refreshed
- November 18th, 2026
GPT-6 Astra leads with 57.3% among models released by September 2026 GPT-6 Astra leads with 57.3% among models released by September 2026
Best score per entry from the benchmark's paper, read on November 18th, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
GPT-6.1 Sol leads with 57.9%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6.1 Sol
- and 12 more
- 57.3 GPT-6 Astra
- 53.9 GPT-6 Sol
- 52.4 Claude Opus 5.5
- 50.2 GPT-6 Luna
- 48.6
Muse Spark 1.3
- 47.0 GPT-5.6 Sol (Aug 2026)
- 46.4 Claude Fable 5.1
- 44.9 GPT-5.6 Luna (Aug 2026)
- 44.5 Claude Sonnet 5
- 42.9 GPT-5
- 41.7 Claude Haiku 4.5
- 41.3 Grok 4.7
- 35.5
Gemini 3.8 Flash
- 33.5 Gemini 2.5 Flash (snapshot unspecified)
- 32.1 GPT-4o (March 2025)
- 32.1
Gemini 3.1 Pro
- 29.5 Gemini 2.5 Pro (snapshot unspecified)
- 1 GPT-6 Astra default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 57.3%
- 2 GPT-6 Sol default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 53.9%
- 3 Claude Opus 5.5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 52.4%
- 4 GPT-6 Luna default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 50.2%
- 5
Muse Spark 1.3 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 48.6%
- 6 GPT-5.6 Sol (Aug 2026) default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 47%
- 7 Claude Fable 5.1 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 46.4%
- 8 GPT-5.6 Luna (Aug 2026) default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 44.9%
- 9 Claude Sonnet 5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 44.5%
- 10 GPT-5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 42.9%
- 11 Claude Haiku 4.5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 41.7%
- 12 Grok 4.7 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 41.3%
- 13
Gemini 3.8 Flash default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 35.5%
- 14 Gemini 2.5 Flash (snapshot unspecified) default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 33.5%
- 15 GPT-4o (March 2025) default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 32.1%
- 16
Gemini 3.1 Pro default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 32.1%
- 17 Gemini 2.5 Pro (snapshot unspecified) default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 29.5%
Source: The MentalHealthBench paper
- 1 GPT-6 Astra default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 57.3%
- 2 GPT-6 Sol default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 53.9%
- 3 Claude Opus 5.5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 52.4%
- 4 GPT-6 Luna default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 50.2%
- 5
Muse Spark 1.3 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 48.6%
- 6 Claude Fable 5.1 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 46.4%
- 7 Claude Sonnet 5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 44.5%
- 8 GPT-5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 42.9%
- 9 Claude Haiku 4.5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 41.7%
- 10 Grok 4.7 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 41.3%
- 11
Gemini 3.8 Flash default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 35.5%
- 12
Gemini 3.1 Pro default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 32.1%
Source: The MentalHealthBench paper
- 1 GPT-6.1 Sol max effort 57.9%
- 2 GPT-6 Astra default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 57.3%
- 3 GPT-6.1 Terra max effort 55.5%
- 4 GPT-6 Sol default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 53.9%
- 5 GPT-6.2 Astra max effort 53.3%
- 6 Claude Opus 5.5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 52.4%
- 7 GPT-6 Luna 51.7%
- 8 GPT-6.2 Sol max effort 50.9%
- 9
Muse Spark 1.3 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 48.6%
- 10 Claude Fable 5.1 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 46.4%
- 11 Claude Sonnet 5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 44.5%
- 12 GPT-5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 42.9%
- 13 Claude Haiku 4.5 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 41.7%
- 14 Grok 4.7 default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 41.3%
- 15
Gemini 3.8 Flash default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 35.5%
- 16
Gemini 3.1 Pro default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge 32.1%
Sources: Labs’ model cards and the MentalHealthBench paper
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 57.3% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 2 | GPT-6 Sol | 53.9% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 3 | Claude Opus 5.5 | 52.4% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 4 | GPT-6 Luna | 50.2% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 5 | | 48.6% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 6 | GPT-5.6 Sol (Aug 2026) | 47% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Official leaderboard | |
| 7 | Claude Fable 5.1 | 46.4% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 8 | GPT-5.6 Luna (Aug 2026) | 44.9% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Official leaderboard | |
| 9 | Claude Sonnet 5 | 44.5% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Jun 2026 | Official leaderboard |
| 10 | GPT-5 | 42.9% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Aug 2025 | Official leaderboard |
| 11 | Claude Haiku 4.5 | 41.7% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Oct 2025 | Official leaderboard |
| 12 | Grok 4.7 | 41.3% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 13 | | 35.5% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 14 | Gemini 2.5 Flash (snapshot unspecified) | 33.5% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Official leaderboard | |
| 15 | GPT-4o (March 2025) | 32.1% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Official leaderboard | |
| 16 | | 32.1% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Feb 2026 | Official leaderboard |
| 17 | Gemini 2.5 Pro (snapshot unspecified) | 29.5% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 57.3% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 2 | GPT-6 Sol | 53.9% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 3 | Claude Opus 5.5 | 52.4% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 4 | GPT-6 Luna | 50.2% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 5 | | 48.6% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 6 | Claude Fable 5.1 | 46.4% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 7 | Claude Sonnet 5 | 44.5% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Jun 2026 | Official leaderboard |
| 8 | GPT-5 | 42.9% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Aug 2025 | Official leaderboard |
| 9 | Claude Haiku 4.5 | 41.7% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Oct 2025 | Official leaderboard |
| 10 | Grok 4.7 | 41.3% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 11 | | 35.5% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 12 | | 32.1% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Feb 2026 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol | 57.9% | max effort | Sep 2026 | Lab model card |
| 2 | GPT-6 Astra | 57.3% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 3 | GPT-6.1 Terra | 55.5% | max effort | Oct 2026 | Lab model card |
| 4 | GPT-6 Sol | 53.9% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 5 | GPT-6.2 Astra | 53.3% | max effort | Nov 2026 | Lab model card |
| 6 | Claude Opus 5.5 | 52.4% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 7 | GPT-6 Luna | 51.7% | Oct 2026 | Lab model card | |
| 8 | GPT-6.2 Sol | 50.9% | max effort | Nov 2026 | Lab model card |
| 9 | 48.6% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard | |
| 10 | Claude Fable 5.1 | 46.4% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 11 | Claude Sonnet 5 | 44.5% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Jun 2026 | Official leaderboard |
| 12 | GPT-5 | 42.9% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Aug 2025 | Official leaderboard |
| 13 | Claude Haiku 4.5 | 41.7% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Oct 2025 | Official leaderboard |
| 14 | Grok 4.7 | 41.3% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard |
| 15 | 35.5% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Sep 2026 | Official leaderboard | |
| 16 | 32.1% | default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | Feb 2026 | Official leaderboard |
Labs report scores for 6 models in their cards. Switch the scores above to rank them together.
Rankings by part
We read 8 parts from the benchmark’s paper in 3 groups: acuity, user and context. Each ranks models on one part of the test. GPT-6 Astra leads 6 of them, and 2 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
Non-acute
Low-urgency, everyday conversations with some emotional weight, such as venting or relationship advice. About 54% of tasks. 12 models we track have a score here.
- 1 GPT-6 Astra 56.6%
- 2 GPT-6 Sol 52.3%
- 3 Claude Opus 5.5 51.4%
- 4 GPT-6 Luna 47.9%
- 5 Claude Fable 5.1 ↑1 44.2%
- 6 Claude Sonnet 5 ↑1 43.9%
- 7
Muse Spark 1.3 ↓2 43.2%
- 8 Claude Haiku 4.5 ↑1 41.4%
- 9 Grok 4.7 ↑1 40.2%
- 10 GPT-5 ↓2 39.7%
High acuity
Not emergencies, but with significant symptoms such as severe depression or panic attacks. About 18% of tasks. 12 models we track have a score here.
- 1 GPT-6 Astra 57.9%
- 2 GPT-6 Sol 57%
- 3
Muse Spark 1.3 ↑2 55.5%
- 4 GPT-6 Luna 53.3%
- 5 Claude Opus 5.5 ↓2 52.8%
- 6 Claude Fable 5.1 50.5%
- 7 GPT-5 ↑1 50.3%
- 8 Claude Sonnet 5 ↓1 48.7%
- 9 Grok 4.7 ↑1 44.3%
- 10 Claude Haiku 4.5 ↓1 43.7%
Emergency
Psychiatric emergencies, where the user may be at risk of harm to self or others. About 28% of tasks. 12 models we track have a score here.
- 1 GPT-6 Astra 58.3%
- 2 GPT-6 Sol 55.1%
- 3
Muse Spark 1.3 ↑2 54.3%
- 4 Claude Opus 5.5 ↓1 53.9%
- 5 GPT-6 Luna ↓1 52.6%
- 6 Claude Fable 5.1 47.8%
- 7 GPT-5 ↑1 44.2%
- 8 Claude Sonnet 5 ↓1 43.1%
- 9 Grok 4.7 ↑1 41.5%
- 10 Claude Haiku 4.5 ↓1 41.1%
Adult
A typical user aged 18 or older. About 68% of tasks. 12 models we track have a score here.
- 1 GPT-6 Astra 56.5%
- 2 GPT-6 Sol 53.2%
- 3 Claude Opus 5.5 50.2%
- 4 GPT-6 Luna 49.6%
- 5
Muse Spark 1.3 45.7%
- 6 Claude Fable 5.1 44.6%
- 7 Claude Sonnet 5 42.1%
- 8 GPT-5 41%
- 9 Claude Haiku 4.5 39.9%
- 10 Grok 4.7 39.5%
Teen
A user aged 13 to 17. About 21% of tasks. 12 models we track have a score here.
- 1 Claude Opus 5.5 ↑2 57%
- 2 GPT-6 Astra ↓1 55.9%
- 3
Muse Spark 1.3 ↑2 53.6%
- 4 GPT-6 Sol ↓2 52.7%
- 5 Claude Sonnet 5 ↑2 49.4%
- 6 GPT-6 Luna ↓2 49%
- 7 Claude Fable 5.1 ↓1 48.9%
- 8 Claude Haiku 4.5 ↑1 46%
- 9 GPT-5 ↓1 44.3%
- 10 Grok 4.7 40.5%
Clinician
A mental health professional asking about practice, such as billing or treatment planning. About 6% of tasks. 12 models we track have a score here.
- 1 GPT-6 Astra 65.1%
- 2 GPT-6 Sol 58.2%
- 3 GPT-6 Luna ↑1 56.2%
- 4 Grok 4.7 ↑6 56.1%
- 5
Muse Spark 1.3 53.4%
- 6 Claude Opus 5.5 ↓3 52.5%
- 7 Claude Fable 5.1 ↓1 47.8%
- 8 GPT-5 47.7%
- 9
Gemini 3.8 Flash ↑2 46.6%
- 10 Claude Sonnet 5 ↓3 46.3%
Caregiver
Someone caring for a loved one with a medical or mental illness. About 5% of tasks. 12 models we track have a score here.
- 1 GPT-6 Astra 66.3%
- 2 GPT-6 Sol 64.6%
- 3 Claude Opus 5.5 63%
- 4
Muse Spark 1.3 ↑1 61.5%
- 5 Claude Fable 5.1 ↑1 57.6%
- 6 GPT-5 ↑2 57%
- 7 GPT-6 Luna ↓3 56.9%
- 8 Claude Sonnet 5 ↓1 55.7%
- 9 Grok 4.7 ↑1 52.2%
- 10 Claude Haiku 4.5 ↓1 52%
Prior user context
The 70 tasks with history that comes before the first user message. 12 models we track have a score here.
- 1
Muse Spark 1.3 ↑4 50.5%
- 2 GPT-6 Astra ↓1 46.2%
- 3 Claude Opus 5.5 44.9%
- 4 GPT-6 Sol ↓2 43.7%
- 5 GPT-6 Luna ↓1 43.3%
- 6 GPT-5 ↑2 41.9%
- 7 Claude Fable 5.1 ↓1 41.6%
- 8 Claude Haiku 4.5 ↑1 39.2%
- 9 Grok 4.7 ↑1 38.5%
- 10 Claude Sonnet 5 ↓3 37.1%
3 models have held number one in the benchmark’s paper since its release in September 2026 4 models have held number one with claims included since its release in September 2026
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s paper and what labs claim in their cards.
Sources: The MentalHealthBench paper and labs’ model cards
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official MentalHealthBench leaderboard and labs’ model cards
Labs report it only in system cards
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark. Some system cards do, below.
7 system cards report it, from 1 lab
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests. OpenAI accounts for 6 of the 6 releases that named it since 2024.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Nov 17, 2026 | GPT-6.2 Sol system card | 50.9 | MentalHealthBench (overall), maximum reasoning effort, 50.9 ± 1.0 (Table 19) | Health |
| Nov 17, 2026 | GPT-6.2 Astra system card | 53.3 | MentalHealthBench (overall), maximum reasoning effort, 53.3 ± 1.0 (Table 19) | Health |
| Oct 29, 2026 | GPT-6 Luna system card addendum | 48.7 | MentalHealthBench (overall), task-clipped metric (Table 6) | Health |
| Oct 26, 2026 | GPT-6.1 Terra system card addendum | 55.5 | MentalHealthBench (overall), maximum reasoning effort, 55.5 ± 1.0 (5.3) | Health |
| Oct 7, 2026 | GPT-6 Sol system card addendum | 48.8 | MentalHealthBench (overall), task-clipped metric (Table 6) | Health |
| Oct 7, 2026 | GPT-6 Luna system card addendum | 51.7 | MentalHealthBench (overall), task-clipped metric (Table 6) | Health |
| Sep 29, 2026 | GPT-6.1 Sol system card addendum | 57.9 | MentalHealthBench (overall), maximum reasoning effort (5.3, table under Figure 4) | Health |
The lab's number comes out ahead in 1 of 2 pairs that differ
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.
| Model | Lab says | Board says | Gap |
|---|---|---|---|
| GPT-6 Sol | 48.8% MentalHealthBench (overall), task-clipped metric (Table 6) | 53.9% Leaderboard, default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | −5.1 pts |
| GPT-6 Luna | 51.7% MentalHealthBench (overall), task-clipped metric (Table 6) | 50.2% Leaderboard, default API reasoning, temperature and verbosity; GPT-5.6 Sol high-reasoning judge | +1.5 pts |
More in Safety and alignment
- FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
- StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is.
Inkling 98.6%
- Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts.
Gemini 4 Carbon 0%
- Covert misbehavior (Thinking Machines) In use Agentic tasks that check whether a model misbehaves and conceals it. Scored by the rate of covert misbehavior.
Inkling 2 Mini 8.30%
- Destructive actions (Thinking Machines) In use Agentic tasks that check whether a model takes destructive actions it was not authorized to take. Scored by the rate of destructive actions.
Inkling 2 Small 4.70%
- Misbehavior (Thinking Machines) In use Agentic tasks that check whether a model fabricates results, ignores instructions or hides what it did when a task is hard. Scored by the rate of misbehavior.
Inkling 2 Mini 9.80%
Data audit
11 of 12 checks pass, 1 warning. Look first at the Confidence check. Likely. Some facts rest on secondary sources. Every benchmark
Catalog record 1 warning 1 pass
- Pass Task access All tasks are public.
- Warning Confidence Likely. Some facts rest on secondary sources.
Scores 5 pass
- Pass Independent runs Scores from the benchmark’s paper (17 entries) and system cards (7 scores).
- Pass Models scored 16 models we track, plus 5 other entries on its leaderboard.
- Pass Newest score The newest is from November 2026, for GPT-6.2 Astra.
- Pass Frontier models 6 of the top 10 have a score, led by GPT-6.2 Astra (#1).
- Pass Headroom The best is 57.9%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on cdn.openai.com: 12 rows for models we track and 5 other entries at the last read.
- Pass Last read November 18th, 2026.
- Pass Read worked The last read worked.
Consistency 2 pass
- Pass Lab claims Both pairs agree within 10 points.
- Pass Known issues No published audit found problems.