Benchmark Almanac

Kimi k1.5

Kimi k1.5 ranks first on MME-CoT.

Moonshot AI Released January 20th, 2025 Family Kimi k1.5 The lab's model card Compare with others
12
Benchmarks with a score
1
Ranked first, out of five or more models
3
In the top three
Moonshot AI Jan 2025

Kimi k1.5

Flagship

131.6 #264 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • MME-CoT 64.2% #1 /10
  • HumanEval Multi 81.5% #2 /5
  • IFEval 87.2% #62 /135

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
CLUEWSC 91.7% #2 of 8 Lab model card
MMLU 87.4% #34 of 131 Lab model card
IFEval 87.2% #62 of 135 Lab model card
C-Eval 88.3% #24 of 46 Lab model card

Math and science

BenchmarkScoreRankSource
MATH-500 96.2% #31 of 70 Lab model card
AIME 2024 77.5% #48 of 88 Lab model card

Coding

BenchmarkScoreRankSource
HumanEval Multi 81.5% #2 of 5 Lab model card
LiveCodeBench 62.5% #22 of 54 Lab model card

Vision, audio and robotics

BenchmarkScoreRankSource
MME-CoT 64.2% #1 of 10 Official leaderboard
MathVista 74.9% #36 of 70 Lab model card
MMMU 70% #63 of 103 Lab model card
MATH-Vision (MathVision) 38.6% #47 of 51 Lab model card

Its lab's numbers put it at 131.6, number 264 of 404

The Self-Reported Index fits one score to the 10 capability results Moonshot AI reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 128.4 to 134.7. How we compute it.

Its lab's numbers next to 2 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark Lab says Board says Gap
MATH-Vision (MathVision) 38.6% MathVision-Full (Pass@1), long-CoT 38.6% Leaderboard Same
MathVista 74.9% MathVista-Test (Pass@1), long-CoT 74.9% Leaderboard Same

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Kimi k1.5: Scaling Reinforcement Learning with LLMs (tech report, arXiv 2501.12599, Tables 2 and 3) Jan 20, 2025

Its system card

Every result in the longer documents the lab published with it.