Benchmark Almanac

LongCat-Flash-Chat

LongCat-Flash-Chat places in the top three on Arena-Hard v2.0.

Meituan Released August 29th, 2025 Family LongCat-Flash The lab's model card Compare with others
34
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Meituan Aug 2025

LongCat-Flash-Chat

Open weights Flagship

132.1 #259 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • VITA-Bench 22.8% #9 /18
  • AMO-Bench 14.6% #15 /21
  • VitaBench 2.0 30.2% #15 /17

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
Arena-Hard v2.0 86.5% #2 of 24 Lab model card
MMLU 89.7% #17 of 131 Lab model card
IFEval 89.7% #36 of 135 Lab model card
MMLU-Pro 82.7% #54 of 167 Lab model card
LMArena (Elo) 1,400 #131 of 335 Official leaderboard
C-Eval 90.4% #18 of 46 Lab model card
Meeseeks 43% #1 of 2 Lab model card
CMMLU 84.3% #10 of 17 Lab model card
DROP 79.1% #27 of 41 Lab model card
COLLIE 57.1% #9 of 12 Lab model card
GraphWalks 51% #24 of 27 Lab model card
Longform Creative Writing (EQ-Bench) 35.1 #106 of 119 Official leaderboard

Math and science

BenchmarkScoreRankSource
ZebraLogic 89.3% #7 of 23 Lab model card
MATH-500 96.4% #29 of 70 Lab model card
GPQA Diamond 73.2% #158 of 302 Lab model card
AMO-Bench 14.6% #15 of 21 Official leaderboard
AIME 2024 70.4% #63 of 88 Lab model card
AIME 2025 61.3% #126 of 143 Lab model card
Beyond AIME 43% #14 of 14 Lab model card

Coding

BenchmarkScoreRankSource
MBPP+ 79.6% #6 of 15 Lab model card
HumanEval+ 88.4% #11 of 27 Lab model card
LiveCodeBench 48% #31 of 54 Lab model card
SWE-bench Verified 60.4% #112 of 155 Lab model card
CATArena OIBench-5 0.15 #4 of 5 Official leaderboard
LiveCodeBench v6 48% #83 of 90 Lab model card
CATArena SWE-Perf-10 0.14× #5 of 5 Official leaderboard

Agents and tools

BenchmarkScoreRankSource
VITA-Bench 22.8% #9 of 18 Official leaderboard
Terminal-Bench 39.5% #12 of 22 Lab model card
SnakeBench 20 #133 of 188 Official leaderboard
tau2-bench 67.7% #62 of 79 Lab model card
VitaBench 2.0 30.2% #15 of 17 Official leaderboard
AceBench 76.1% #3 of 3 Lab model card

Safety and security

BenchmarkScoreRankSource
LiveSecBench 57.1 #15 of 34 Official leaderboard
LongCat safety evaluation (Meituan) 87.7% #2 of 2 Lab model card

Its lab's numbers put it at 132.1, number 259 of 404

The Self-Reported Index fits one score to the 23 capability results Meituan reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 130.0 to 134.3. How we compute it.

Its lab's numbers next to 2 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark Lab says Board says Gap
VITA-Bench 24.3% VitaBench(avg@4) 22.8% Leaderboard, non-thinking +1.5 pts
MMLU-Pro 82.7% MMLU-Pro(acc) 82.7% Leaderboard Same

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

LongCat-Flash-Chat model card (Hugging Face) Aug 29, 2025

Its system card

Every result in the longer documents the lab published with it. 1 of them isn't in the launch post.