S-Eval
Models respond to Chinese and English harmful prompts and attack variants. A trained safety judge measures the safe-response fraction and attack success rate.
The released benchmark contains 20,000 base risk prompts and 200,000 attack prompts. Safety score and attack success rate have opposite directions.
- Made by
- Zhejiang University and Alibaba Group
- Released
- May 2024
- Kind
- Refusing harmful requests
- Level
- Everyday
- Scored
- Graded by a model
- Answers
- Open-ended
- Tasks
- All public
- Measures
- Propensity
- Behavior
- Harmful use
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Content safety
- Strongest model tested
-
DeepSeek-V3.2 #91 on the capability index
- Last model added
- December 2025
- Scores refreshed
- November 20th, 2026
In the authors’ words
Generative large language models (LLMs) have revolutionized natural language processing with their transformative and emergent capabilities. However, recent evidence indicates that LLMs can produce harmful content that violates social norms, raising significant concerns regarding the safety and ethical ramifications of deploying these advanced models. Thus, it is both critical and imperative to perform a rigorous and comprehensive safety evaluation of LLMs before deployment. Despite this need, owing to the extensiveness of LLM generation space, it still lacks a unified and standardized risk taxonomy to systematically reflect the LLM content safety, as well as automated safety assessment techniques to explore the potential risk efficiently. To bridge the striking gap, we propose S-Eval, a novel LLM-based automated Safety Evaluation framework with a newly defined comprehensive risk taxonomy. S-Eval incorporates two key components, i.e., an expert testing LLM Mt and a novel safety critique LLM Mc. Mt is responsible for automatically generating test cases in accordance with the proposed risk taxonomy. Mc can provide quantitative and explainable safety evaluations for better risk awareness of LLMs. In contrast to prior works, S-Eval is efficient and effective in test generation and safety evaluation. Moreover, S-Eval can be flexibly configured and adapted to the rapid evolution of LLMs and accompanying new safety threats, test generation methods and safety critique methods thanks to the LLM-based architecture. S-Eval has been deployed in our industrial partner for the automated safety evaluation of multiple LLMs serving millions of users, demonstrating its effectiveness in real-world scenarios. Our benchmark is publicly available at https://github.com/IS2Lab/S-Eval.
gpt-oss-120b leads with 88.8% among models released by December 2025 gpt-oss-120b leads with 88.8% among models released by December 2025
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. The board reports each language separately, so this is the plain mean of Chinese and English base risk safety scores. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in December 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 88.8 gpt-oss-120b
- 85.7 gpt-oss-20b
- 83.7 ErnieBot-4.0
- 82.8
DeepSeek-V3.2
- 80.8
Qwen3-235B-A22B-Thinking
- 79.9
Qwen3-235B-A22B-Instruct
- 79.0 Qwen3-Max-0923
- 72.3 Qwen-72B-Chat
- 72.0 Baichuan2-13B-Chat
- 70.0
Qwen-14B
- 70.0 DeepSeek-V3 (snapshot unspecified)
- 65.2 Qwen2.5-32B-Instruct
- 64.1 Qwen2.5-14B-Instruct
- 59.5 Qwen-7B-Chat
- 59.0
Qwen2.5-Max
- 59.0
Mistral Small 3.2
- 58.9 GPT-4-Turbo (snapshot unspecified)
- 58.7 ChatGLM3-6B
- 55.7 Gemma-7B-it
- 54.9 Gemma-2B-it
- 54.4
Doubao-Seed-1.6
- 54.0 Doubao-1.5-Pro-32k
- 54.0 Qwen-1.8B-Chat
- 53.0 GPT-4o (snapshot unspecified)
- 52.1
Qwen3-235B-A22B
- 52.0
Qwen3-14B
- 47.9
Gemini 1.0 Pro
- 43.0 Yi-34B-Chat
- 38.5 DeepSeek-R1 (snapshot unspecified)
- 37.7
QwQ-32B
- 37.6 Gemini-2.5-Pro (snapshot unspecified)
- 34.8
Gemma 3 27B
- 34.1
DeepSeek-R1-Distill-Qwen-14B
- 34.1
DeepSeek-R1-Distill-Qwen-32B
- 24.6 Qwen3-0.6B (Instruct)
- 20.1
Qwen3-0.6B
- 1 gpt-oss-120b 88.8%
- 2 gpt-oss-20b 85.7%
- 3 ErnieBot-4.0 83.7%
- 4
DeepSeek-V3.2 thinking 82.8%
- 5
Qwen3-235B-A22B-Thinking Jul 2025 80.8%
- 6
Qwen3-235B-A22B-Instruct Jul 2025 79.9%
- 7 Qwen3-Max-0923 79%
- 8 Qwen-72B-Chat 72.3%
- 9 Baichuan2-13B-Chat 72%
- 10
Qwen-14B 70%
- 11 DeepSeek-V3 (snapshot unspecified) 70%
- 12 Qwen2.5-32B-Instruct 65.2%
- 13 Qwen2.5-14B-Instruct 64.1%
- 14 Qwen-7B-Chat 59.5%
- 15
Qwen2.5-Max 59%
- 16
Mistral Small 3.2 59%
- 17 GPT-4-Turbo (snapshot unspecified) 58.9%
- 18 ChatGLM3-6B 58.7%
- 19 Gemma-7B-it 55.7%
- 20 Gemma-2B-it 54.9%
- 21
Doubao-Seed-1.6 thinking 54.4%
- 22 Doubao-1.5-Pro-32k 54%
- 23 Qwen-1.8B-Chat 54%
- 24 GPT-4o (snapshot unspecified) 53%
- 25
Qwen3-235B-A22B thinking 52.1%
- 26
Qwen3-14B thinking 52%
- 27
Gemini 1.0 Pro 47.9%
- 28 Yi-34B-Chat 43%
- 29 DeepSeek-R1 (snapshot unspecified) 38.5%
- 30
QwQ-32B 37.7%
- 31 Gemini-2.5-Pro (snapshot unspecified) 37.6%
- 32
Gemma 3 27B 34.8%
- 33
DeepSeek-R1-Distill-Qwen-14B 34.1%
- 34
DeepSeek-R1-Distill-Qwen-32B 34.1%
- 35 Qwen3-0.6B (Instruct) 24.6%
- 36
Qwen3-0.6B thinking 20.1%
Source: Official S-Eval leaderboard
- 88.8 gpt-oss-120b
- 85.7 gpt-oss-20b
- 82.8
DeepSeek-V3.2
- 80.8
Qwen3-235B-A22B-Thinking
- 79.9
Qwen3-235B-A22B-Instruct
- 72.0 Baichuan2-13B-Chat
- 70.0
Qwen-14B
- 59.0
Qwen2.5-Max
- 59.0
Mistral Small 3.2
- 58.7 ChatGLM3-6B
- 54.4
Doubao-Seed-1.6
- 52.1
Qwen3-235B-A22B
- 52.0
Qwen3-14B
- 47.9
Gemini 1.0 Pro
- 37.7
QwQ-32B
- 34.8
Gemma 3 27B
- 34.1
DeepSeek-R1-Distill-Qwen-14B
- 34.1
DeepSeek-R1-Distill-Qwen-32B
- 20.1
Qwen3-0.6B
- 1 gpt-oss-120b 88.8%
- 2 gpt-oss-20b 85.7%
- 3
DeepSeek-V3.2 thinking 82.8%
- 4
Qwen3-235B-A22B-Thinking Jul 2025 80.8%
- 5
Qwen3-235B-A22B-Instruct Jul 2025 79.9%
- 6 Baichuan2-13B-Chat 72%
- 7
Qwen-14B 70%
- 8
Qwen2.5-Max 59%
- 9
Mistral Small 3.2 59%
- 10 ChatGLM3-6B 58.7%
- 11
Doubao-Seed-1.6 thinking 54.4%
- 12
Qwen3-235B-A22B thinking 52.1%
- 13
Qwen3-14B thinking 52%
- 14
Gemini 1.0 Pro 47.9%
- 15
QwQ-32B 37.7%
- 16
Gemma 3 27B 34.8%
- 17
DeepSeek-R1-Distill-Qwen-14B 34.1%
- 18
DeepSeek-R1-Distill-Qwen-32B 34.1%
- 19
Qwen3-0.6B thinking 20.1%
Source: Official S-Eval leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | gpt-oss-120b | 88.8% | Aug 2025 | Official leaderboard | |
| 2 | gpt-oss-20b | 85.7% | Aug 2025 | Official leaderboard | |
| 3 | ErnieBot-4.0 | 83.7% | Official leaderboard | ||
| 4 | | 82.8% | thinking | Dec 2025 | Official leaderboard |
| 5 | | 80.8% | Jul 2025 | Official leaderboard | |
| 6 | | 79.9% | Jul 2025 | Official leaderboard | |
| 7 | Qwen3-Max-0923 | 79% | Official leaderboard | ||
| 8 | Qwen-72B-Chat | 72.3% | Official leaderboard | ||
| 9 | Baichuan2-13B-Chat | 72% | Sep 2023 | Official leaderboard | |
| 10 | | 70% | Sep 2023 | Official leaderboard | |
| 11 | DeepSeek-V3 (snapshot unspecified) | 70% | Official leaderboard | ||
| 12 | Qwen2.5-32B-Instruct | 65.2% | Official leaderboard | ||
| 13 | Qwen2.5-14B-Instruct | 64.1% | Official leaderboard | ||
| 14 | Qwen-7B-Chat | 59.5% | Official leaderboard | ||
| 15 | | 59% | Jan 2025 | Official leaderboard | |
| 16 | | 59% | Jun 2025 | Official leaderboard | |
| 17 | GPT-4-Turbo (snapshot unspecified) | 58.9% | Official leaderboard | ||
| 18 | ChatGLM3-6B | 58.7% | Oct 2023 | Official leaderboard | |
| 19 | Gemma-7B-it | 55.7% | Official leaderboard | ||
| 20 | Gemma-2B-it | 54.9% | Official leaderboard | ||
| 21 | | 54.4% | thinking | Jun 2025 | Official leaderboard |
| 22 | Doubao-1.5-Pro-32k | 54% | Official leaderboard | ||
| 23 | Qwen-1.8B-Chat | 54% | Official leaderboard | ||
| 24 | GPT-4o (snapshot unspecified) | 53% | Official leaderboard | ||
| 25 | | 52.1% | thinking | Apr 2025 | Official leaderboard |
| 26 | | 52% | thinking | Apr 2025 | Official leaderboard |
| 27 | | 47.9% | Dec 2023 | Official leaderboard | |
| 28 | Yi-34B-Chat | 43% | Official leaderboard | ||
| 29 | DeepSeek-R1 (snapshot unspecified) | 38.5% | Official leaderboard | ||
| 30 | | 37.7% | Mar 2025 | Official leaderboard | |
| 31 | Gemini-2.5-Pro (snapshot unspecified) | 37.6% | Official leaderboard | ||
| 32 | | 34.8% | Mar 2025 | Official leaderboard | |
| 33 | | 34.1% | Jan 2025 | Official leaderboard | |
| 34 | | 34.1% | Jan 2025 | Official leaderboard | |
| 35 | Qwen3-0.6B (Instruct) | 24.6% | Official leaderboard | ||
| 36 | | 20.1% | thinking | Apr 2025 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | gpt-oss-120b | 88.8% | Aug 2025 | Official leaderboard | |
| 2 | gpt-oss-20b | 85.7% | Aug 2025 | Official leaderboard | |
| 3 | | 82.8% | thinking | Dec 2025 | Official leaderboard |
| 4 | | 80.8% | Jul 2025 | Official leaderboard | |
| 5 | | 79.9% | Jul 2025 | Official leaderboard | |
| 6 | Baichuan2-13B-Chat | 72% | Sep 2023 | Official leaderboard | |
| 7 | | 70% | Sep 2023 | Official leaderboard | |
| 8 | | 59% | Jan 2025 | Official leaderboard | |
| 9 | | 59% | Jun 2025 | Official leaderboard | |
| 10 | ChatGLM3-6B | 58.7% | Oct 2023 | Official leaderboard | |
| 11 | | 54.4% | thinking | Jun 2025 | Official leaderboard |
| 12 | | 52.1% | thinking | Apr 2025 | Official leaderboard |
| 13 | | 52% | thinking | Apr 2025 | Official leaderboard |
| 14 | | 47.9% | Dec 2023 | Official leaderboard | |
| 15 | | 37.7% | Mar 2025 | Official leaderboard | |
| 16 | | 34.8% | Mar 2025 | Official leaderboard | |
| 17 | | 34.1% | Jan 2025 | Official leaderboard | |
| 18 | | 34.1% | Jan 2025 | Official leaderboard | |
| 19 | | 20.1% | thinking | Apr 2025 | Official leaderboard |
Rankings by part
We read 42 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as Chinese, Chinese: 违法犯罪 and Chinese: 仇恨言论. gpt-oss-120b leads 13 of them, and 7 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
Chinese
19 models we track have a score here.
- 1 gpt-oss-120b 91.2%
- 2
DeepSeek-V3.2 ↑1 89%
- 3 gpt-oss-20b ↓1 88.4%
- 4
Qwen3-235B-A22B-Instruct Jul 2025 ↑1 84.3%
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 83.8%
- 6
Doubao-Seed-1.6 ↑5 66.8%
- 7 Baichuan2-13B-Chat ↓1 66.6%
- 8
Qwen-14B ↓1 66.5%
- 9
Qwen2.5-Max ↓1 64.3%
- 10
Qwen3-14B ↑3 62.2%
Chinese: 违法犯罪
19 models we track have a score here.
- 1
DeepSeek-V3.2 ↑2 92.8%
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 92.2%
- 3
Qwen3-235B-A22B-Instruct Jul 2025 ↑2 91.7%
- 4 gpt-oss-120b ↓3 90.6%
- 5 gpt-oss-20b ↓3 89.4%
- 6
Qwen-14B ↑1 75%
- 7 Baichuan2-13B-Chat ↓1 74.4%
- 8
Qwen3-14B ↑5 71.7%
- 9
Doubao-Seed-1.6 ↑2 70%
- 10
Qwen2.5-Max ↓2 67.2%
Chinese: 仇恨言论
19 models we track have a score here.
- 1 gpt-oss-20b ↑1 94.3%
- 2 gpt-oss-120b ↓1 92.9%
- 3
DeepSeek-V3.2 87.1%
- 4
Qwen3-235B-A22B-Instruct Jul 2025 ↑1 83.6%
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 78.6%
- 6
Qwen-14B ↑1 76.4%
- 7 ChatGLM3-6B ↑3 72.1%
- 8 Baichuan2-13B-Chat ↓2 70%
- 9
Doubao-Seed-1.6 ↑2 69.3%
- 10
Qwen3-14B ↑3 65%
Chinese: 身心健康
19 models we track have a score here.
- 1 gpt-oss-120b 94%
- 2
DeepSeek-V3.2 ↑1 94%
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 88%
- 4
Qwen3-235B-A22B-Instruct Jul 2025 ↑1 87%
- 5 gpt-oss-20b ↓3 84%
- 6
Qwen-14B ↑1 80%
- 7 Baichuan2-13B-Chat ↓1 79%
- 8
Doubao-Seed-1.6 ↑3 75%
- 9
Qwen3-14B ↑4 70%
- 10 ChatGLM3-6B 68%
Chinese: 伦理道德
19 models we track have a score here.
- 1 gpt-oss-120b 94%
- 2
DeepSeek-V3.2 ↑1 94%
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 90%
- 4 gpt-oss-20b ↓2 90%
- 5
Qwen3-235B-A22B-Instruct Jul 2025 86%
- 6
Qwen2.5-Max ↑2 68%
- 7
Doubao-Seed-1.6 ↑4 54%
- 8
Gemini 1.0 Pro ↑6 50%
- 9 Baichuan2-13B-Chat ↓3 47%
- 10
Qwen3-14B ↑3 42%
Chinese: 数据隐私
19 models we track have a score here.
- 1 gpt-oss-120b 80%
- 2
Mistral Small 3.2 ↑7 79%
- 3 gpt-oss-20b ↓1 78%
- 4 Baichuan2-13B-Chat ↑2 77%
- 5
Qwen-14B ↑2 77%
- 6
DeepSeek-V3.2 ↓3 74%
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓3 73%
- 8
Qwen3-235B-A22B-Instruct Jul 2025 ↓3 67%
- 9
Qwen2.5-Max ↓1 63%
- 10
Doubao-Seed-1.6 ↑1 63%
Chinese: 网络安全
19 models we track have a score here.
- 1 gpt-oss-120b 97.9%
- 2 gpt-oss-20b 97.1%
- 3
DeepSeek-V3.2 97.1%
- 4
Qwen3-235B-A22B-Instruct Jul 2025 ↑1 90.7%
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 85.7%
- 6
Qwen2.5-Max ↑2 78.6%
- 7
Mistral Small 3.2 ↑2 73.6%
- 8
Qwen3-14B ↑5 71.4%
- 9 Baichuan2-13B-Chat ↓3 65%
- 10
Doubao-Seed-1.6 ↑1 64.3%
Chinese: 极端主义
19 models we track have a score here.
- 1 gpt-oss-120b 97.5%
- 2 gpt-oss-20b 97.5%
- 3
DeepSeek-V3.2 97.5%
- 4
Qwen3-235B-A22B-Thinking Jul 2025 96.7%
- 5
Qwen3-235B-A22B-Instruct Jul 2025 95.8%
- 6
Qwen2.5-Max ↑2 82.5%
- 7
Doubao-Seed-1.6 ↑4 75.8%
- 8
Qwen-14B ↓1 74.2%
- 9
Qwen3-14B ↑4 73.3%
- 10 Baichuan2-13B-Chat ↓4 68.3%
Chinese: 不当建议
19 models we track have a score here.
- 1 gpt-oss-120b 80.8%
- 2
DeepSeek-V3.2 ↑1 78.3%
- 3 gpt-oss-20b ↓1 71.7%
- 4
Qwen3-235B-A22B-Instruct Jul 2025 ↑1 65.8%
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 62.5%
- 6
Doubao-Seed-1.6 ↑5 60%
- 7
Qwen-14B 55%
- 8 ChatGLM3-6B ↑2 50%
- 9 Baichuan2-13B-Chat ↓3 48.3%
- 10
Mistral Small 3.2 ↓1 45.8%
English
19 models we track have a score here.
- 1 gpt-oss-120b 86.4%
- 2 gpt-oss-20b 82.9%
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 77.7%
- 4 Baichuan2-13B-Chat ↑2 77.4%
- 5
DeepSeek-V3.2 ↓2 76.6%
- 6
Qwen3-235B-A22B-Instruct Jul 2025 ↓1 75.5%
- 7
Qwen-14B 73.5%
- 8
Mistral Small 3.2 ↑1 57.8%
- 9 ChatGLM3-6B ↑1 57.7%
- 10
Qwen2.5-Max ↓2 53.7%
English: Illegal Activities
19 models we track have a score here.
- 1 gpt-oss-120b 83.9%
- 2 gpt-oss-20b 82.2%
- 3 Baichuan2-13B-Chat ↑3 81.1%
- 4
Qwen3-235B-A22B-Thinking Jul 2025 76.7%
- 5
Qwen3-235B-A22B-Instruct Jul 2025 75.6%
- 6
DeepSeek-V3.2 ↓3 73.9%
- 7
Qwen-14B 69.4%
- 8
Qwen3-235B-A22B ↑4 55.6%
- 9
Mistral Small 3.2 53.9%
- 10
Qwen2.5-Max ↓2 52.8%
English: Hate Speech
19 models we track have a score here.
- 1 gpt-oss-120b 91.4%
- 2 gpt-oss-20b 91.4%
- 3 Baichuan2-13B-Chat ↑3 80.7%
- 4
DeepSeek-V3.2 ↓1 78.6%
- 5
Qwen3-235B-A22B-Instruct Jul 2025 76.4%
- 6
Qwen-14B ↑1 75.7%
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓3 75%
- 8 ChatGLM3-6B ↑2 74.3%
- 9
Qwen3-235B-A22B ↑3 57.9%
- 10
Doubao-Seed-1.6 ↑1 56.4%
English: Physical & Mental Health
19 models we track have a score here.
- 1
Qwen3-235B-A22B-Thinking Jul 2025 ↑3 89%
- 2 gpt-oss-120b ↓1 88%
- 3
DeepSeek-V3.2 88%
- 4 Baichuan2-13B-Chat ↑2 86%
- 5
Qwen3-235B-A22B-Instruct Jul 2025 84%
- 6
Qwen-14B ↑1 83%
- 7 gpt-oss-20b ↓5 77%
- 8 ChatGLM3-6B ↑2 76%
- 9
Qwen3-14B ↑4 72%
- 10
Mistral Small 3.2 ↓1 69%
English: Ethics & Morality
19 models we track have a score here.
- 1 gpt-oss-120b 91%
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 89%
- 3
DeepSeek-V3.2 88%
- 4 gpt-oss-20b ↓2 85%
- 5
Qwen3-235B-A22B-Instruct Jul 2025 84%
- 6 Baichuan2-13B-Chat 74%
- 7
Qwen-14B 72%
- 8
Qwen2.5-Max 56%
- 9 ChatGLM3-6B ↑1 55%
- 10
Qwen3-235B-A22B ↑2 52%
English: Data Privacy
19 models we track have a score here.
- 1
Qwen-14B ↑6 88%
- 2 Baichuan2-13B-Chat ↑4 85%
- 3
Mistral Small 3.2 ↑6 82%
- 4
DeepSeek-V3.2 ↓1 77%
- 5 gpt-oss-120b ↓4 76%
- 6 ChatGLM3-6B ↑4 75%
- 7 gpt-oss-20b ↓5 73%
- 8
Qwen3-235B-A22B-Thinking Jul 2025 ↓4 72%
- 9
Qwen3-235B-A22B-Instruct Jul 2025 ↓4 70%
- 10
Qwen2.5-Max ↓2 68%
English: Cybersecurity
19 models we track have a score here.
- 1 gpt-oss-120b 96.4%
- 2 gpt-oss-20b 94.3%
- 3 Baichuan2-13B-Chat ↑3 82.9%
- 4
DeepSeek-V3.2 ↓1 82.9%
- 5
Qwen3-235B-A22B-Instruct Jul 2025 78.6%
- 6
Qwen3-235B-A22B-Thinking Jul 2025 ↓2 74.3%
- 7
Qwen-14B 71.4%
- 8
Mistral Small 3.2 ↑1 67.9%
- 9
Qwen2.5-Max ↓1 65%
- 10
Qwen3-235B-A22B ↑2 46.4%
English: Extremism
19 models we track have a score here.
- 1 gpt-oss-20b ↑1 88.3%
- 2 gpt-oss-120b ↓1 86.7%
- 3
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 83.3%
- 4
Qwen3-235B-A22B-Instruct Jul 2025 ↑1 80.8%
- 5
Qwen-14B ↑2 78.3%
- 6
DeepSeek-V3.2 ↓3 75%
- 7 Baichuan2-13B-Chat ↓1 73.3%
- 8
Qwen2.5-Max 51.7%
- 9
Mistral Small 3.2 50.8%
- 10
Qwen3-235B-A22B ↑2 50%
English: Inappropriate Advice
19 models we track have a score here.
- 1 gpt-oss-120b 75.8%
- 2
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 66.7%
- 3 gpt-oss-20b ↓1 66.7%
- 4
DeepSeek-V3.2 ↓1 62.5%
- 5
Qwen-14B ↑2 55.8%
- 6
Qwen3-235B-A22B-Instruct Jul 2025 ↓1 55.8%
- 7 Baichuan2-13B-Chat ↓1 55%
- 8 ChatGLM3-6B ↑2 45.8%
- 9
Mistral Small 3.2 45.8%
- 10
Gemini 1.0 Pro ↑4 40%
Chinese attack success rate
4 models we track have a score here.
- 1 Baichuan2-13B-Chat ↑5 38.1%
- 2 ChatGLM3-6B ↑8 46%
- 3
Gemini 1.0 Pro ↑11 47%
- 4
Qwen-14B ↑3 48.4%
Chinese attack success rate: 自适应攻击
4 models we track have a score here.
- 1 Baichuan2-13B-Chat ↑5 0.20%
- 2
Qwen-14B ↑5 0.30%
- 3 ChatGLM3-6B ↑7 0.60%
- 4
Gemini 1.0 Pro ↑10 0.80%
Chinese attack success rate: 正面诱导
4 models we track have a score here.
- 1 Baichuan2-13B-Chat ↑5 13.6%
- 2
Qwen-14B ↑5 27.9%
- 3 ChatGLM3-6B ↑7 33.1%
- 4
Gemini 1.0 Pro ↑10 42.1%
Chinese attack success rate: 反面诱导
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 16.4%
- 2 Baichuan2-13B-Chat ↑4 22.1%
- 3
Qwen-14B ↑4 27.9%
- 4 ChatGLM3-6B ↑6 29.1%
Chinese attack success rate: 代码注入
4 models we track have a score here.
- 1 Baichuan2-13B-Chat ↑5 80%
- 2 ChatGLM3-6B ↑8 90.2%
- 3
Qwen-14B ↑4 95.2%
- 4
Gemini 1.0 Pro ↑10 97.9%
Chinese attack success rate: 指令越狱
4 models we track have a score here.
- 1 Baichuan2-13B-Chat ↑5 21%
- 2
Qwen-14B ↑5 32%
- 3 ChatGLM3-6B ↑7 37.6%
- 4
Gemini 1.0 Pro ↑10 44.1%
Chinese attack success rate: 目标劫持
4 models we track have a score here.
- 1 Baichuan2-13B-Chat ↑5 63.8%
- 2 ChatGLM3-6B ↑8 66.1%
- 3
Qwen-14B ↑4 81.2%
- 4
Gemini 1.0 Pro ↑10 81.8%
Chinese attack success rate: 指令加密
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 96.4%
- 2 Baichuan2-13B-Chat ↑4 97.8%
- 3
Qwen-14B ↑4 99.5%
- 4 ChatGLM3-6B ↑6 99.5%
Chinese attack success rate: 深度催眠
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 30.4%
- 2 Baichuan2-13B-Chat ↑4 35.2%
- 3 ChatGLM3-6B ↑7 39.3%
- 4
Qwen-14B ↑3 48.2%
Chinese attack success rate: 上下文攻击
4 models we track have a score here.
- 1 Baichuan2-13B-Chat ↑5 30.6%
- 2
Gemini 1.0 Pro ↑12 33.1%
- 3 ChatGLM3-6B ↑7 35.2%
- 4
Qwen-14B ↑3 51.5%
Chinese attack success rate: 话语链
4 models we track have a score here.
- 1 Baichuan2-13B-Chat ↑5 2.00%
- 2
Qwen-14B ↑5 9.90%
- 3
Gemini 1.0 Pro ↑11 19.4%
- 4 ChatGLM3-6B ↑6 20%
Chinese attack success rate: 组合指令攻击
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 8.00%
- 2 ChatGLM3-6B ↑8 10.4%
- 3
Qwen-14B ↑4 10.5%
- 4 Baichuan2-13B-Chat ↑2 15.3%
English attack success rate
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 41.2%
- 2 ChatGLM3-6B ↑8 46.8%
- 3 Baichuan2-13B-Chat ↑3 47.6%
- 4
Qwen-14B ↑3 52.4%
English attack success rate: Adaptive Attack
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 0.50%
- 2 ChatGLM3-6B ↑8 1.10%
- 3
Qwen-14B ↑4 1.20%
- 4 Baichuan2-13B-Chat ↑2 2.70%
English attack success rate: Positive Induction
4 models we track have a score here.
- 1 ChatGLM3-6B ↑9 25.8%
- 2
Gemini 1.0 Pro ↑12 31.5%
- 3 Baichuan2-13B-Chat ↑3 32.6%
- 4
Qwen-14B ↑3 59.6%
English attack success rate: Reverse Induction
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 18.3%
- 2 ChatGLM3-6B ↑8 29.8%
- 3
Qwen-14B ↑4 33.5%
- 4 Baichuan2-13B-Chat ↑2 37.6%
English attack success rate: Code Injection
4 models we track have a score here.
- 1
Qwen-14B ↑6 88.6%
- 2 Baichuan2-13B-Chat ↑4 89.1%
- 3 ChatGLM3-6B ↑7 89.1%
- 4
Gemini 1.0 Pro ↑10 93.8%
English attack success rate: Instruction Jailbreak
4 models we track have a score here.
- 1
Qwen-14B ↑6 18%
- 2
Gemini 1.0 Pro ↑12 21.6%
- 3 ChatGLM3-6B ↑7 34%
- 4 Baichuan2-13B-Chat ↑2 34.1%
English attack success rate: Goal Hijacking
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 70.4%
- 2 ChatGLM3-6B ↑8 71.5%
- 3 Baichuan2-13B-Chat ↑3 74.7%
- 4
Qwen-14B ↑3 82.2%
English attack success rate: Instruction Encryption
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 97.3%
- 2 Baichuan2-13B-Chat ↑4 98.9%
- 3
Qwen-14B ↑4 99.9%
- 4 ChatGLM3-6B ↑6 99.9%
English attack success rate: DeepInception
4 models we track have a score here.
- 1 Baichuan2-13B-Chat ↑5 21.2%
- 2
Gemini 1.0 Pro ↑12 27.4%
- 3 ChatGLM3-6B ↑7 48.1%
- 4
Qwen-14B ↑3 53%
English attack success rate: In-Context Attack
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 31%
- 2 ChatGLM3-6B ↑8 39.8%
- 3 Baichuan2-13B-Chat ↑3 44.5%
- 4
Qwen-14B ↑3 49.3%
English attack success rate: Chain of Utterances
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 10.4%
- 2 ChatGLM3-6B ↑8 17%
- 3
Qwen-14B ↑4 18.2%
- 4 Baichuan2-13B-Chat ↑2 28.4%
English attack success rate: Compositional Instruction Attack
4 models we track have a score here.
- 1
Gemini 1.0 Pro ↑13 9.90%
- 2 ChatGLM3-6B ↑8 13.3%
- 3 Baichuan2-13B-Chat ↑3 14.5%
- 4
Qwen-14B ↑3 21.9%
3 models have held number one on the official leaderboard since its release in May 2024
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official S-Eval leaderboard
Gpt-oss-120b on the official leaderboard passed 85% in August 2025
That was 15 months after its release. By the same age, 27% of the benchmarks from 2024 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official S-Eval leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Safety and alignment
- FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
- StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is.
Inkling 98.6%
- Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts.
Gemini 4 Carbon 0%
- Covert misbehavior (Thinking Machines) In use Agentic tasks that check whether a model misbehaves and conceals it. Scored by the rate of covert misbehavior.
Inkling 2 Mini 8.30%
- Destructive actions (Thinking Machines) In use Agentic tasks that check whether a model takes destructive actions it was not authorized to take. Scored by the rate of destructive actions.
Inkling 2 Small 4.70%
- Misbehavior (Thinking Machines) In use Agentic tasks that check whether a model fabricates results, ignores instructions or hides what it did when a task is hard. Scored by the rate of misbehavior.
Inkling 2 Mini 9.80%
Data audit
All 8 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
- Pass Independent runs Scores from the official leaderboard (36 entries).
- Pass Models scored 19 models we track, plus 17 other entries on its leaderboard.
- Note Newest score The newest came 11 months ago, in December 2025, for DeepSeek-V3.2. That’s expected for a saturated test.
- Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3.2 (#91).
- Note Headroom Saturated. The best is 88.8%, past 85% since August 2025.
Leaderboard import 3 pass
- Pass Importer We read the board on s.alibaba.com: 21 rows for models we track and 17 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.