LongCat safety evaluation (Meituan)
LongCat-Flash-Thinking 95.7% Models answer private test queries in four risk categories: harmful content, criminal activity, misinformation and privacy. The score is the share of safe responses in each category.
In-house test built on the framework of Mu et al. 2024 and Meituan content guidelines. Queries were reviewed by hand.
- Made by
- Meituan LongCat Team
- Released
- September 2025
- Kind
- Refusing harmful requests
- Level
- Everyday
- Scored
- Graded by a model
- Answers
- Open-ended
- Tasks
- Kept private
- Last in a launch post
- September 2025, dropped after it saturated
- Last model added
- September 2025
LongCat-Flash-Thinking leads with 95.7%
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
- 1 LongCat-Flash-Thinking 95.7%
- 2 LongCat-Flash-Chat 87.7%
Source: Labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | LongCat-Flash-Thinking | 95.7% | Sep 2025 | Lab model card | |
| 2 | LongCat-Flash-Chat | 87.7% | Aug 2025 | Lab model card |
Meituan’s own number for LongCat-Flash-Chat passed 85% in August 2025
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Labs’ model cards
Labs dropped it from launch posts once the best score passed 85%
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Sep 21, 2025 | LongCat-Flash-Thinking | 95.7 | Mean of Safety Harmful 93.7 / Criminal 97.1 / Misinformation 93.0 / Privacy 98.8 |
| Aug 29, 2025 | LongCat-Flash-Chat | 87.7 | Mean of Safety Harmful 83.98 / Criminal 91.24 / Misinformation 81.72 / Privacy 93.98 |
2 system cards report it, from 1 lab
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Sep 23, 2025 | LongCat-Flash-Thinking technical report | Safety accuracy Harmful 93.7, Criminal 97.1, Misinformation 93.0, Privacy 98.8, no overall score (Table 2) | Safety | |
| Sep 1, 2025 | LongCat-Flash-Chat technical report | Safety Harmful 83.98, Criminal 91.24, Misinformation 81.72, Privacy 93.98, no overall score (Table 3) | Safety |
More in Safety and alignment
- FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
- StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is.
Inkling 98.6% - Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts.
Gemini 4 Carbon 0% - DistressBench In use Models reply as a chatbot to 718 simulated users in psychological crisis, in single-turn and multi-turn chats. Three judge models score replies against clinician-written rubrics.
Muse Spark 1.2 88.4% - Lakera B3 (Backbone Breaker Benchmark) In use Crowdsourced prompt-injection attacks hit backbone LLMs inside agent threat snapshots at three defense levels. The score is attack resistance, 100 minus the board's vulnerability percentage. Claude Opus 4.5 89%
- IHEval In use Models face conflicts between system, user and tool instructions. Reports a composite score for following the higher-priority instruction. GPT-4o 70%