Benchmark Almanac

Image input safety evaluations

Only in system cards

Models answer harmful prompts that combine text and images in categories such as hate, extremism and self-harm. Scored as safe completion rate.

In-house OpenAI evaluation. Cards report category scores with no overall number.

Made by
OpenAI
Released
December 2024
Kind
Refusing harmful requests
Level
Everyday
Scored
Graded by a model
Answers
Open-ended
Tasks
Kept private
Measures
Propensity
Language
English
In and out
Text, Images → Text
Interaction
Single turn
Refresh
One fixed set
Strongest model tested
o1 #118 on the capability index
Last in a system card
November 2026
Last model added
November 2026
2
Models scored in lab cards
33
System cards report it, from 1 lab

GPT-4.5 leads with 99%

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

  1. 1 GPT-4.5 99%
  2. 2 o1 96.1%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 GPT-4.5 99% Feb 2025 Lab model card
2 o1 96.1% Dec 2024 Lab model card

OpenAI’s own number for o1 passed 85% in December 2024

Models released before it came out already scored that high. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 Near the ceiling (85%) GPT-4.5 o1 0% 20% 40% 60% 80% 100% 2025 Near the ceiling (85%) GPT-4.5

Source: Labs’ model cards

Labs report it only in system cards

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark. Some system cards do, below.

33 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests. OpenAI accounts for 24 of the 24 releases that named it since 2024.

DateDocumentScoreAs the document writes itSection
Nov 17, 2026 GPT-6.2 Sol system card Harmful Image Inputs, 4 categories, no overall score (Table 16) Safety
Nov 17, 2026 GPT-6.2 Astra system card Harmful Image Inputs, 4 categories, no overall score (Table 16) Safety
Oct 29, 2026 GPT-6 Luna system card addendum Image input evaluations, safe completion, 4 categories, no overall score (Table 3) Safety
Oct 26, 2026 GPT-6.1 Terra system card addendum Image input evaluations, safe completion, 4 categories, no overall score (second Table 24, 3.2) Safety
Oct 7, 2026 GPT-6 Sol system card addendum Image input evaluations, safe completion, 4 categories, no overall score (Table 3) Safety
Oct 7, 2026 GPT-6 Luna system card addendum Image input evaluations, safe completion, 4 categories, no overall score (Table 3) Safety
Sep 29, 2026 GPT-6.1 Sol system card addendum Image input evaluations, safe completion, 4 categories, no overall score (Table 4) Safety
Sep 22, 2026 GPT-6 Sol system card Image input evaluations, safe completion, 4 categories, no overall score (Table 26, A.2.2) Safety
Sep 22, 2026 GPT-6 Luna system card Image input evaluations, safe completion, 4 categories, no overall score (Table 26, A.2.2) Safety
Sep 3, 2026 GPT-6 Astra system card Image input evaluations, safe completion, 4 categories, no overall score (Table 4) Safety
Aug 6, 2026 GPT-5.6 Sol system card addendum Image input evaluations, not_unsafe, 4 categories, no overall score (3.3) Safety
Aug 6, 2026 GPT-5.6 Luna system card addendum Image input evaluations, not_unsafe, 4 categories, no overall score (3.3) Safety
Jul 9, 2026 GPT-5.6 Terra system card Image input evaluations, not_unsafe, 4 categories, no overall score (3.2) Safety
Jul 9, 2026 GPT-5.6 Sol system card Image input evaluations, not_unsafe, 4 categories, no overall score (3.2) Safety
Jul 9, 2026 GPT-5.6 Luna system card Image input evaluations, not_unsafe, 4 categories, no overall score (3.2) Safety
Jun 25, 2026 GPT-5.6 Terra system card Image input evaluations, not_unsafe, 4 categories, no overall score (3.2) Safety
Jun 25, 2026 GPT-5.6 Sol system card Image input evaluations, not_unsafe, 4 categories, no overall score (3.2) Safety
Jun 25, 2026 GPT-5.6 Luna system card Image input evaluations, not_unsafe, 4 categories, no overall score (3.2) Safety
May 4, 2026 GPT-5.5 Instant system card Image input evaluations, not_unsafe, 4 categories, no overall score: hate 0.979, extremism 0.969, self-harm 0.982, harms-erotic 0.990 (Table 2) Safety
Apr 23, 2026 GPT-5.5 system card Image input evaluations, not_unsafe, 4 categories, no overall score (Table 2) Safety
Mar 5, 2026 GPT-5.4 system card Image input evaluations, not_unsafe, 6 categories, no overall score (Table 5) Safety
Dec 11, 2025 GPT-5.2 system card addendum Image input evaluations, not_unsafe, gpt-5.2-thinking, 6 categories, no overall score (Table 4) Safety
Nov 18, 2025 GPT-5.1-Codex-Max system card Image input evaluations, not_unsafe, 6 categories, no overall score (Table 3) Safety
Nov 12, 2025 GPT-5.1 system card addendum Image input evaluations, not_unsafe, gpt-5.1-thinking, 6 categories, no overall score (Table 3) Safety
Oct 27, 2025 GPT-5 system card addendum Image input evaluations, not_unsafe, 6 categories, no overall score (Table 3) Safety
Aug 7, 2025 GPT-5 system card Image input evaluations, not_unsafe, 6 categories, no overall score (Table 10) Safety
Aug 7, 2025 GPT-5 nano system card Image input, gpt-5-thinking-nano, 6 categories, no overall score (Table 26) Safety
Aug 7, 2025 GPT-5 mini system card Image input, gpt-5-thinking-mini, 6 categories, no overall score (Table 26) Safety
Jul 17, 2025 ChatGPT agent system card Image input evaluations, not_unsafe, 6 categories, no overall score (Table 5) Safety
Apr 16, 2025 o4-mini system card Multimodal refusal evaluations, vision sexual and self-harm rows, not_unsafe, no overall score (Table 5) Safety
Apr 16, 2025 o3 system card Multimodal refusal evaluations, vision sexual and self-harm rows, not_unsafe, no overall score (Table 5) Safety
Feb 27, 2025 GPT-4.5 system card 99 Multimodal Refusal Evaluation, text and image input, not_unsafe, not_overrefuse 0.31 (Table 2) Safety
Dec 5, 2024 o1 system card 96.1 Multimodal Refusal Evaluation - Text and Image Input, not_unsafe overall sample weighted (Table 22, not_overrefuse 0.96 in Table 2) Safety

More in Safety and alignment

All 636 in Safety and alignment
  • FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
  • StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is. Inkling 98.6%
  • Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. Gemini 4 Carbon 0%
  • Covert misbehavior (Thinking Machines) In use Agentic tasks that check whether a model misbehaves and conceals it. Scored by the rate of covert misbehavior. Inkling 2 Mini 8.30%
  • Destructive actions (Thinking Machines) In use Agentic tasks that check whether a model takes destructive actions it was not authorized to take. Scored by the rate of destructive actions. Inkling 2 Small 4.70%
  • Misbehavior (Thinking Machines) In use Agentic tasks that check whether a model fabricates results, ignores instructions or hides what it did when a task is hard. Scored by the rate of misbehavior. Inkling 2 Mini 9.80%

Data audit

All 4 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access The tasks are private.
  • Pass Confidence Checked against a primary source.
Scores 1 pass 4 notes
  • Note Independent runs Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores).
  • Note Models scored Only 2 models we track.
  • Pass Newest score The newest is from November 2026, for GPT-6.2 Astra.
  • Note Frontier models None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118).
  • Note Headroom Saturated. The best is 99%, past 85% since December 2024.
Leaderboard import 1 note
  • Note Importer No public leaderboard to read.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check