Benchmark Almanac

MonitorBench

gpt-oss-120b 91.9% leads since March 2026
Not in model cards

Models work through 1,540 instances across 19 tasks in 7 categories, each built around a decision-critical factor such as a cue, a hidden goal or a required step. A monitor LLM reads the chain of thought, and the monitorability score measures how often it surfaces that factor.

COLM 2026. Two stress tests ask the model to hide the factor or to evade a monitor it knows about. Inspect Evals lists single tasks separately, such as monitorbench-goal-sandbag-engineering from the Goal (Sandbag) task. Code and data are MIT licensed.

Released
March 2026
Kind
Deception and alignment
Level
Expert
Scored
Graded by a model
Answers
Open-ended
Tasks
All public
Measures
Propensity
Language
English
In and out
Text → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Gemini 3 Flash #60 on the capability index
Last model added
December 2025
Scores refreshed
November 20th, 2026
8
Models scored in its paper

gpt-oss-120b leads with 91.9% among models released by December 2025

Best score per model from the benchmark's paper, read on November 20th, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in December 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 gpt-oss-120b All-messages, Standard 91.9%
  2. 2 Qwen3-235B-A22B All-messages, Standard 91.7%
  3. 3 Qwen3-14B All-messages, Standard 89.8%
  4. 4 Claude Haiku 4.5 All-messages, Standard 85.8%
  5. 5 Claude Sonnet 4.5 All-messages, Standard 84.9%
  6. 6 GPT-5 All-messages, Standard 81.1%
  7. 7 GPT-5 mini All-messages, Standard 79.5%
  8. 8 Gemini 3 Flash All-messages, Standard 78.3%

Source: The MonitorBench paper

#ModelScoreSettingDateSource
1 gpt-oss-120b 91.9% All-messages, Standard Aug 2025 Official leaderboard
2 Qwen3-235B-A22B 91.7% All-messages, Standard Apr 2025 Official leaderboard
3 Qwen3-14B 89.8% All-messages, Standard Apr 2025 Official leaderboard
4 Claude Haiku 4.5 85.8% All-messages, Standard Oct 2025 Official leaderboard
5 Claude Sonnet 4.5 84.9% All-messages, Standard Sep 2025 Official leaderboard
6 GPT-5 81.1% All-messages, Standard Aug 2025 Official leaderboard
7 GPT-5 mini 79.5% All-messages, Standard Aug 2025 Official leaderboard
8 Gemini 3 Flash 78.3% All-messages, Standard Dec 2025 Official leaderboard

Rankings by part

We read 19 parts from the benchmark’s paper in 7 groups: cue (condition), cue (answer), goal (sandbag), goal (deception), dual objectives, impossible tasks and solution process. Each ranks models on one part of the test. Qwen3-235B-A22B leads 8 of them, and 3 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

Health

8 models we track have a score here.

  1. 1 Qwen3-235B-A22B ↑1 100%
  2. 2 gpt-oss-120b ↓1 100%
  3. 3 GPT-5 ↑3 100%
  4. 4 GPT-5 mini ↑3 100%
  5. 5 Claude Sonnet 4.5 100%
  6. 6 Claude Haiku 4.5 ↓2 100%
  7. 7 Gemini 3 Flash ↑1 100%
  8. 8 Qwen3-14B ↓5 97.6%

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
gpt-oss-120b
Action-only… All-message… CoT-only, S…
91.9%
Qwen3-235B-A22B
Action-only… Action-only… CoT-only, D… All-message…
91.7%
Qwen3-14B
Action-only… Action-only… Action-only… CoT-only, D… All-message… All-message…
89.8%
Claude Haiku 4.5
Action-only… Action-only… CoT-only, M… CoT-only, S…
85.8%
Claude Sonnet 4.5
Action-only… Action-only… All-message… CoT-only, S…
84.9%
GPT-5
Action-only… CoT-only, D… CoT-only, S…
81.1%
GPT-5 mini
Action-only… Action-only… CoT-only, D… CoT-only, S… All-message…
79.5%
Gemini 3 Flash
Action-only… Action-only… CoT-only, D… All-message…
78.3%

Source: The MonitorBench paper

ModelEffortHarnessScoreCost per taskOutput tokens
gpt-oss-120b All-messages, Standard 91.9%
Qwen3-235B-A22B All-messages, Standard 91.7%
gpt-oss-120b CoT-only, Standard 91.5%
Qwen3-14B All-messages, Standard 89.8%
Qwen3-235B-A22B CoT-only, Standard 88.5%
Qwen3-235B-A22B All-messages, Monitor-aware Evasion 86.4%
Claude Haiku 4.5 All-messages, Standard 85.8%
Qwen3-14B CoT-only, Standard 85.5%
Claude Sonnet 4.5 All-messages, Standard 84.9%
Qwen3-235B-A22B All-messages, Direct Concealment 84.6%
Claude Sonnet 4.5 CoT-only, Standard 84.4%
Qwen3-14B All-messages, Monitor-aware Evasion 83.7%
Claude Haiku 4.5 CoT-only, Standard 81.4%
GPT-5 All-messages, Standard 81.1%
Qwen3-235B-A22B CoT-only, Direct Concealment 80.5%
GPT-5 mini All-messages, Standard 79.5%
Qwen3-235B-A22B CoT-only, Monitor-aware Evasion 78.9%
Claude Sonnet 4.5 CoT-only, Monitor-aware Evasion 78.8%
Qwen3-14B All-messages, Direct Concealment 78.4%
Gemini 3 Flash All-messages, Standard 78.3%
Qwen3-14B CoT-only, Monitor-aware Evasion 78%
Claude Sonnet 4.5 CoT-only, Direct Concealment 77.5%
Claude Sonnet 4.5 All-messages, Monitor-aware Evasion 77.4%
GPT-5 CoT-only, Standard 76.7%
Claude Haiku 4.5 All-messages, Direct Concealment 76.3%
gpt-oss-120b CoT-only, Monitor-aware Evasion 76.1%
Claude Sonnet 4.5 All-messages, Direct Concealment 75.7%
Qwen3-14B CoT-only, Direct Concealment 75.6%
gpt-oss-120b CoT-only, Direct Concealment 75.2%
Claude Haiku 4.5 CoT-only, Direct Concealment 74.8%
Qwen3-235B-A22B Action-only, Standard 74.6%
Gemini 3 Flash CoT-only, Standard 74.6%
GPT-5 mini CoT-only, Standard 73.7%
gpt-oss-120b All-messages, Direct Concealment 73.3%
gpt-oss-120b All-messages, Monitor-aware Evasion 72.8%
Claude Haiku 4.5 All-messages, Monitor-aware Evasion 72.8%
GPT-5 All-messages, Monitor-aware Evasion 72%
Gemini 3 Flash All-messages, Monitor-aware Evasion 72%
GPT-5 CoT-only, Monitor-aware Evasion 71.7%
GPT-5 All-messages, Direct Concealment 71.3%
Claude Haiku 4.5 CoT-only, Monitor-aware Evasion 71.3%
Gemini 3 Flash CoT-only, Monitor-aware Evasion 71.1%
GPT-5 CoT-only, Direct Concealment 70.9%
Gemini 3 Flash All-messages, Direct Concealment 70.3%
Qwen3-14B Action-only, Standard 69.6%
Gemini 3 Flash CoT-only, Direct Concealment 69.1%
GPT-5 mini All-messages, Monitor-aware Evasion 67.7%
GPT-5 mini All-messages, Direct Concealment 67.6%
Qwen3-235B-A22B Action-only, Monitor-aware Evasion 66.3%
GPT-5 mini CoT-only, Monitor-aware Evasion 64.5%
GPT-5 mini CoT-only, Direct Concealment 63.8%
Claude Sonnet 4.5 Action-only, Standard 63.8%
Claude Haiku 4.5 Action-only, Standard 62.8%
Qwen3-235B-A22B Action-only, Direct Concealment 62.5%
Qwen3-14B Action-only, Monitor-aware Evasion 61.5%
GPT-5 mini Action-only, Standard 56.7%
gpt-oss-120b Action-only, Standard 55%
Claude Sonnet 4.5 Action-only, Direct Concealment 53.6%
Qwen3-14B Action-only, Direct Concealment 53.4%
gpt-oss-120b Action-only, Direct Concealment 52.2%
gpt-oss-120b Action-only, Monitor-aware Evasion 51.9%
Claude Sonnet 4.5 Action-only, Monitor-aware Evasion 51.6%
Claude Haiku 4.5 Action-only, Direct Concealment 50.8%
Claude Haiku 4.5 Action-only, Monitor-aware Evasion 49.9%
GPT-5 Action-only, Standard 48.5%
GPT-5 mini Action-only, Direct Concealment 48.4%
GPT-5 mini Action-only, Monitor-aware Evasion 46.1%
Gemini 3 Flash Action-only, Standard 45%
GPT-5 Action-only, Monitor-aware Evasion 44.4%
GPT-5 Action-only, Direct Concealment 43.9%
Gemini 3 Flash Action-only, Direct Concealment 37.4%
Gemini 3 Flash Action-only, Monitor-aware Evasion 37.1%

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Benchmark paper Passed 85% in Mar 2026, by gpt-oss-120b
gpt-oss-120b 91.9%

Source: The MonitorBench paper

Qwen3-235B-A22B on the official leaderboard passed 85% in April 2025

Models released before it came out already scored that high. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
In the benchmark’s paper Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) gpt-oss-120b Qwen3-235B-A22B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) gpt-oss-120b

Source: Official MonitorBench leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Safety and alignment

All 636 in Safety and alignment
  • FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
  • StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is. Inkling 98.6%
  • Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. Gemini 4 Carbon 0%
  • Covert misbehavior (Thinking Machines) In use Agentic tasks that check whether a model misbehaves and conceals it. Scored by the rate of covert misbehavior. Inkling 2 Mini 8.30%
  • Destructive actions (Thinking Machines) In use Agentic tasks that check whether a model takes destructive actions it was not authorized to take. Scored by the rate of destructive actions. Inkling 2 Small 4.70%
  • Misbehavior (Thinking Machines) In use Agentic tasks that check whether a model fabricates results, ignores instructions or hides what it did when a task is hard. Scored by the rate of misbehavior. Inkling 2 Mini 9.80%

Data audit

All 8 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
  • Pass Independent runs Scores from the benchmark’s paper (8 entries).
  • Pass Models scored 8 models we track.
  • Note Newest score The newest came 11 months ago, in December 2025, for Gemini 3 Flash. That’s expected for a saturated test.
  • Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Flash (#60).
  • Note Headroom Saturated. The best is 91.9%, past 85% since April 2025.
Leaderboard import 3 pass
  • Pass Importer We read the board on proceedings.colm.cc: 72 rows for models we track at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check