Benchmark Almanac

AILuminate Safety 1.2

Meta Llama Guard 2 8B 100.0 tops its leaderboard Tencent Hy4 preview 96.0 leads the models we track
Not in model cards

Models answer 3,900 hidden English prompts, 325 in each of 12 hazard categories, picked with item response theory. A calibrated evaluator gives a 0 to 100 score.

Released September 9th, 2026 with a new judge, prompt sample and scale. MLCommons says its scores are not comparable with earlier Poor-to-Excellent grades. Version 1.2.1 aligned scoring with Jailbreak 1.1.

Made by
MLCommons
Released
September 2026
Kind
Refusing harmful requests
Level
Everyday
Scored
Graded by a model
Answers
Open-ended
Tasks
Kept private
Strongest model tested
DeepSeek-V4-Flash #86 on the capability index
Last model added
August 2026
Scores refreshed
November 12th, 2026
27
Entries on its own leaderboard, 5 models we track

Meta Llama Guard 2 8B tops the leaderboard with 100.0

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Tencent Hy4 preview, at 96.0.

The newest model with a score here came out in August 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

27 entries. Scroll sideways for the rest.

  1. 100.0 Meta Llama Guard 2 8B
  2. 100.0 Meta Llama Guard 3 8B
  3. 100.0 Meta Llama Guard 4 12B
  4. 100.0 NVIDIA Nemotron 3.5 Content Safety
  5. 98.0 Deep Cogito Cogito v2.1 671B
  6. 96.0 Tencent Hy4 preview
  7. 94.0 HKUST JailJudge Guard
  8. 93.0 NVIDIA Nemotron 3 Ultra 550B A55B
  9. 86.0 Google Gemma 3n E4B IT
  10. 85.0 IBM Granite 4.2 30B
  11. 85.0 IBM Granite 4.2 3B
  12. 85.0 OpenAI gpt-oss-120b
  13. 78.0 Xiaomi MiMo V2.5 Pro
  14. 75.0 IBM Granite 4.2 8B
  15. 64.0 DeepSeek Flash Latest (OpenRouter)
  16. 57.0 InclusionAI Ling 3.0 Flash Fin
  17. 52.0 Z.ai GLM-5.3 Flash BF16
  18. 50.0 Qwen3 235B A22B Instruct 2507
  19. 44.0 InclusionAI Ling 3.0 Flash
  20. 43.0 InclusionAI Ling 3.0 Flash VL
  21. 43.0 DeepSeek-V4-Flash
  22. 40.0 InclusionAI Ling 2.6 1T
  23. 38.0 DeepSeek-V4-Flash-Vision-Exp
  24. 33.0 Moonshot AI Kimi K2.7 Code
  25. 16.0 Mistral Medium 3.5
  26. 11.0 Reka Edge
  27. 4.00 SpeakLeash Bielik 11B v3.0 Instruct
  1. 1 Meta Llama Guard 2 8B AILuminate Safety 1.2.1; English (US); overall percentile 100.0
  2. 2 Meta Llama Guard 3 8B AILuminate Safety 1.2.1; English (US); overall percentile 100.0
  3. 3 Meta Llama Guard 4 12B AILuminate Safety 1.2.1; English (US); overall percentile 100.0
  4. 4 NVIDIA Nemotron 3.5 Content Safety AILuminate Safety 1.2.1; English (US); overall percentile 100.0
  5. 5 Deep Cogito Cogito v2.1 671B AILuminate Safety 1.2.1; English (US); overall percentile 98.0
  6. 6 Tencent Hy4 preview AILuminate Safety 1.2.1; English (US); overall percentile 96.0
  7. 7 HKUST JailJudge Guard AILuminate Safety 1.2.1; English (US); overall percentile 94.0
  8. 8 NVIDIA Nemotron 3 Ultra 550B A55B AILuminate Safety 1.2.1; English (US); overall percentile 93.0
  9. 9 Google Gemma 3n E4B IT AILuminate Safety 1.2.1; English (US); overall percentile 86.0
  10. 10 IBM Granite 4.2 30B AILuminate Safety 1.2.1; English (US); overall percentile 85.0
  11. 11 IBM Granite 4.2 3B AILuminate Safety 1.2.1; English (US); overall percentile 85.0
  12. 12 OpenAI gpt-oss-120b AILuminate Safety 1.2.1; English (US); overall percentile 85.0
  13. 13 Xiaomi MiMo V2.5 Pro AILuminate Safety 1.2.1; English (US); overall percentile 78.0
  14. 14 IBM Granite 4.2 8B AILuminate Safety 1.2.1; English (US); overall percentile 75.0
  15. 15 DeepSeek Flash Latest (OpenRouter) AILuminate Safety 1.2.1; English (US); overall percentile 64.0
  16. 16 InclusionAI Ling 3.0 Flash Fin AILuminate Safety 1.2.1; English (US); overall percentile 57.0
  17. 17 Z.ai GLM-5.3 Flash BF16 AILuminate Safety 1.2.1; English (US); overall percentile 52.0
  18. 18 Qwen3 235B A22B Instruct 2507 AILuminate Safety 1.2.1; English (US); overall percentile 50.0
  19. 19 InclusionAI Ling 3.0 Flash AILuminate Safety 1.2.1; English (US); overall percentile 44.0
  20. 20 InclusionAI Ling 3.0 Flash VL AILuminate Safety 1.2.1; English (US); overall percentile 43.0
  21. 21 DeepSeek-V4-Flash AILuminate Safety 1.2.1; English (US); overall percentile 43.0
  22. 22 InclusionAI Ling 2.6 1T AILuminate Safety 1.2.1; English (US); overall percentile 40.0
  23. 23 DeepSeek-V4-Flash-Vision-Exp AILuminate Safety 1.2.1; English (US); overall percentile 38.0
  24. 24 Moonshot AI Kimi K2.7 Code AILuminate Safety 1.2.1; English (US); overall percentile 33.0
  25. 25 Mistral Medium 3.5 AILuminate Safety 1.2.1; English (US); overall percentile 16.0
  26. 26 Reka Edge AILuminate Safety 1.2.1; English (US); overall percentile 11.0
  27. 27 SpeakLeash Bielik 11B v3.0 Instruct AILuminate Safety 1.2.1; English (US); overall percentile 4.00

Source: Official AILuminate Safety 1.2 leaderboard

#EntryScoreSettingDateSource
1 Meta Llama Guard 2 8B 100.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
2 Meta Llama Guard 3 8B 100.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
3 Meta Llama Guard 4 12B 100.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
4 NVIDIA Nemotron 3.5 Content Safety 100.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
5 Deep Cogito Cogito v2.1 671B 98.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
6 Tencent Hy4 preview 96.0 AILuminate Safety 1.2.1; English (US); overall percentile Aug 2026 Official leaderboard
7 HKUST JailJudge Guard 94.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
8 NVIDIA Nemotron 3 Ultra 550B A55B 93.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
9 Google Gemma 3n E4B IT 86.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
10 IBM Granite 4.2 30B 85.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
11 IBM Granite 4.2 3B 85.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
12 OpenAI gpt-oss-120b 85.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
13 Xiaomi MiMo V2.5 Pro 78.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
14 IBM Granite 4.2 8B 75.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
15 DeepSeek Flash Latest (OpenRouter) 64.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
16 InclusionAI Ling 3.0 Flash Fin 57.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
17 Z.ai GLM-5.3 Flash BF16 52.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
18 Qwen3 235B A22B Instruct 2507 50.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
19 InclusionAI Ling 3.0 Flash 44.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
20 InclusionAI Ling 3.0 Flash VL 43.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
21 DeepSeek-V4-Flash 43.0 AILuminate Safety 1.2.1; English (US); overall percentile Apr 2026 Official leaderboard
22 InclusionAI Ling 2.6 1T 40.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
23 DeepSeek-V4-Flash-Vision-Exp 38.0 AILuminate Safety 1.2.1; English (US); overall percentile Aug 2026 Official leaderboard
24 Moonshot AI Kimi K2.7 Code 33.0 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard
25 Mistral Medium 3.5 16.0 AILuminate Safety 1.2.1; English (US); overall percentile Apr 2026 Official leaderboard
26 Reka Edge 11.0 AILuminate Safety 1.2.1; English (US); overall percentile Mar 2026 Official leaderboard
27 SpeakLeash Bielik 11B v3.0 Instruct 4.00 AILuminate Safety 1.2.1; English (US); overall percentile Official leaderboard

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Tencent
Own leaderboard Not tested since Sep 2026
Tencent Hy4 preview 96.0

Source: Official AILuminate Safety 1.2 leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0.00 20.0 40.0 60.0 80.0 100.0 2026 0.00 20.0 40.0 60.0 80.0 100.0 2026

Source: Official AILuminate Safety 1.2 leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

Replaces MLCommons AILuminate

More in Safety and alignment

All 465 in Safety and alignment
  • FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
  • StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is. Inkling 98.6%
  • Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. Gemini 4 Carbon 0%
  • DistressBench In use Models reply as a chatbot to 718 simulated users in psychological crisis, in single-turn and multi-turn chats. Three judge models score replies against clinician-written rubrics. Muse Spark 1.2 88.4%
  • Lakera B3 (Backbone Breaker Benchmark) In use Crowdsourced prompt-injection attacks hit backbone LLMs inside agent threat snapshots at three defense levels. The score is attack resistance, 100 minus the board's vulnerability percentage. Claude Opus 4.5 89%
  • IHEval In use Models face conflicts between system, user and tool instructions. Reports a composite score for following the higher-priority instruction. GPT-4o 70%