Benchmark Almanac

MLCommons AILuminate

Meta Llama Guard 2 8B 100% tops its leaderboard Tencent Hy4 preview 96% leads the models we track
Not in model cards

Industry-standard hazard benchmark over 12 risk categories with graded ratings.

Made by
MLCommons
Released
December 2024
Kind
Refusing harmful requests
Scored
Graded by a model
Answers
Open-ended
Tasks
Public, plus a private test set
Strongest model tested
DeepSeek-V4-Flash #86 on the capability index
Last model added
August 2026
Scores refreshed
November 12th, 2026
27
Entries on its own leaderboard, 5 models we track

Meta Llama Guard 2 8B tops the leaderboard with 100%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Tencent Hy4 preview, at 96%.

The newest model with a score here came out in August 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

27 entries. Scroll sideways for the rest.

  1. 100.0 Meta Llama Guard 2 8B
  2. 100.0 Meta Llama Guard 3 8B
  3. 100.0 Meta Llama Guard 4 12B
  4. 100.0 NVIDIA Nemotron 3.5 Content Safety
  5. 98.0 Deep Cogito Cogito v2.1 671B
  6. 96.0 Tencent Hy4 preview
  7. 94.0 HKUST JailJudge Guard
  8. 93.0 NVIDIA Nemotron 3 Ultra 550B A55B
  9. 86.0 Google Gemma 3n E4B IT
  10. 85.0 IBM Granite 4.2 30B
  11. 85.0 IBM Granite 4.2 3B
  12. 85.0 OpenAI gpt-oss-120b
  13. 78.0 Xiaomi MiMo V2.5 Pro
  14. 75.0 IBM Granite 4.2 8B
  15. 64.0 DeepSeek Flash Latest (OpenRouter)
  16. 57.0 InclusionAI Ling 3.0 Flash Fin
  17. 52.0 Z.ai GLM-5.3 Flash BF16
  18. 50.0 Qwen3 235B A22B Instruct 2507
  19. 44.0 InclusionAI Ling 3.0 Flash
  20. 43.0 InclusionAI Ling 3.0 Flash VL
  21. 43.0 DeepSeek-V4-Flash
  22. 40.0 InclusionAI Ling 2.6 1T
  23. 38.0 DeepSeek-V4-Flash-Vision-Exp
  24. 33.0 Moonshot AI Kimi K2.7 Code
  25. 16.0 Mistral Medium 3.5
  26. 11.0 Reka Edge
  27. 4.00 SpeakLeash Bielik 11B v3.0 Instruct
  1. 1 Meta Llama Guard 2 8B Safety 1.2.1; English (US); overall percentile 100%
  2. 2 Meta Llama Guard 3 8B Safety 1.2.1; English (US); overall percentile 100%
  3. 3 Meta Llama Guard 4 12B Safety 1.2.1; English (US); overall percentile 100%
  4. 4 NVIDIA Nemotron 3.5 Content Safety Safety 1.2.1; English (US); overall percentile 100%
  5. 5 Deep Cogito Cogito v2.1 671B Safety 1.2.1; English (US); overall percentile 98%
  6. 6 Tencent Hy4 preview Safety 1.2.1; English (US); overall percentile 96%
  7. 7 HKUST JailJudge Guard Safety 1.2.1; English (US); overall percentile 94%
  8. 8 NVIDIA Nemotron 3 Ultra 550B A55B Safety 1.2.1; English (US); overall percentile 93%
  9. 9 Google Gemma 3n E4B IT Safety 1.2.1; English (US); overall percentile 86%
  10. 10 IBM Granite 4.2 30B Safety 1.2.1; English (US); overall percentile 85%
  11. 11 IBM Granite 4.2 3B Safety 1.2.1; English (US); overall percentile 85%
  12. 12 OpenAI gpt-oss-120b Safety 1.2.1; English (US); overall percentile 85%
  13. 13 Xiaomi MiMo V2.5 Pro Safety 1.2.1; English (US); overall percentile 78%
  14. 14 IBM Granite 4.2 8B Safety 1.2.1; English (US); overall percentile 75%
  15. 15 DeepSeek Flash Latest (OpenRouter) Safety 1.2.1; English (US); overall percentile 64%
  16. 16 InclusionAI Ling 3.0 Flash Fin Safety 1.2.1; English (US); overall percentile 57%
  17. 17 Z.ai GLM-5.3 Flash BF16 Safety 1.2.1; English (US); overall percentile 52%
  18. 18 Qwen3 235B A22B Instruct 2507 Safety 1.2.1; English (US); overall percentile 50%
  19. 19 InclusionAI Ling 3.0 Flash Safety 1.2.1; English (US); overall percentile 44%
  20. 20 InclusionAI Ling 3.0 Flash VL Safety 1.2.1; English (US); overall percentile 43%
  21. 21 DeepSeek-V4-Flash Safety 1.2.1; English (US); overall percentile 43%
  22. 22 InclusionAI Ling 2.6 1T Safety 1.2.1; English (US); overall percentile 40%
  23. 23 DeepSeek-V4-Flash-Vision-Exp Safety 1.2.1; English (US); overall percentile 38%
  24. 24 Moonshot AI Kimi K2.7 Code Safety 1.2.1; English (US); overall percentile 33%
  25. 25 Mistral Medium 3.5 Safety 1.2.1; English (US); overall percentile 16%
  26. 26 Reka Edge Safety 1.2.1; English (US); overall percentile 11%
  27. 27 SpeakLeash Bielik 11B v3.0 Instruct Safety 1.2.1; English (US); overall percentile 4.00%

Source: Official MLCommons AILuminate leaderboard

#EntryScoreSettingDateSource
1 Meta Llama Guard 2 8B 100% Safety 1.2.1; English (US); overall percentile Official leaderboard
2 Meta Llama Guard 3 8B 100% Safety 1.2.1; English (US); overall percentile Official leaderboard
3 Meta Llama Guard 4 12B 100% Safety 1.2.1; English (US); overall percentile Official leaderboard
4 NVIDIA Nemotron 3.5 Content Safety 100% Safety 1.2.1; English (US); overall percentile Official leaderboard
5 Deep Cogito Cogito v2.1 671B 98% Safety 1.2.1; English (US); overall percentile Official leaderboard
6 Tencent Hy4 preview 96% Safety 1.2.1; English (US); overall percentile Aug 2026 Official leaderboard
7 HKUST JailJudge Guard 94% Safety 1.2.1; English (US); overall percentile Official leaderboard
8 NVIDIA Nemotron 3 Ultra 550B A55B 93% Safety 1.2.1; English (US); overall percentile Official leaderboard
9 Google Gemma 3n E4B IT 86% Safety 1.2.1; English (US); overall percentile Official leaderboard
10 IBM Granite 4.2 30B 85% Safety 1.2.1; English (US); overall percentile Official leaderboard
11 IBM Granite 4.2 3B 85% Safety 1.2.1; English (US); overall percentile Official leaderboard
12 OpenAI gpt-oss-120b 85% Safety 1.2.1; English (US); overall percentile Official leaderboard
13 Xiaomi MiMo V2.5 Pro 78% Safety 1.2.1; English (US); overall percentile Official leaderboard
14 IBM Granite 4.2 8B 75% Safety 1.2.1; English (US); overall percentile Official leaderboard
15 DeepSeek Flash Latest (OpenRouter) 64% Safety 1.2.1; English (US); overall percentile Official leaderboard
16 InclusionAI Ling 3.0 Flash Fin 57% Safety 1.2.1; English (US); overall percentile Official leaderboard
17 Z.ai GLM-5.3 Flash BF16 52% Safety 1.2.1; English (US); overall percentile Official leaderboard
18 Qwen3 235B A22B Instruct 2507 50% Safety 1.2.1; English (US); overall percentile Official leaderboard
19 InclusionAI Ling 3.0 Flash 44% Safety 1.2.1; English (US); overall percentile Official leaderboard
20 InclusionAI Ling 3.0 Flash VL 43% Safety 1.2.1; English (US); overall percentile Official leaderboard
21 DeepSeek-V4-Flash 43% Safety 1.2.1; English (US); overall percentile Apr 2026 Official leaderboard
22 InclusionAI Ling 2.6 1T 40% Safety 1.2.1; English (US); overall percentile Official leaderboard
23 DeepSeek-V4-Flash-Vision-Exp 38% Safety 1.2.1; English (US); overall percentile Aug 2026 Official leaderboard
24 Moonshot AI Kimi K2.7 Code 33% Safety 1.2.1; English (US); overall percentile Official leaderboard
25 Mistral Medium 3.5 16% Safety 1.2.1; English (US); overall percentile Apr 2026 Official leaderboard
26 Reka Edge 11% Safety 1.2.1; English (US); overall percentile Mar 2026 Official leaderboard
27 SpeakLeash Bielik 11B v3.0 Instruct 4.00% Safety 1.2.1; English (US); overall percentile Official leaderboard

3 models have held number one on the official leaderboard since its release in December 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Reka DeepSeek Tencent
Own leaderboard Passed 85% in Aug 2026, by Tencent Hy4 preview
Tencent Hy4 preview 96%

Source: Official MLCommons AILuminate leaderboard

Tencent Hy4 preview on the official leaderboard passed 85% in August 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Tencent Hy4 preview DeepSeek-V4-Flash Reka Edge 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Tencent Hy4 preview DeepSeek-V4-Flash

Source: Official MLCommons AILuminate leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

Replaced by AILuminate Safety 1.2

More in Safety and alignment

All 465 in Safety and alignment
  • FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
  • StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is. Inkling 98.6%
  • Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. Gemini 4 Carbon 0%
  • DistressBench In use Models reply as a chatbot to 718 simulated users in psychological crisis, in single-turn and multi-turn chats. Three judge models score replies against clinician-written rubrics. Muse Spark 1.2 88.4%
  • Lakera B3 (Backbone Breaker Benchmark) In use Crowdsourced prompt-injection attacks hit backbone LLMs inside agent threat snapshots at three defense levels. The score is attack resistance, 100 minus the board's vulnerability percentage. Claude Opus 4.5 89%
  • IHEval In use Models face conflicts between system, user and tool instructions. Reports a composite score for following the higher-priority instruction. GPT-4o 70%