Benchmark Almanac

Moonlight-16B-A3B

Moonlight-16B-A3B.

Moonshot AI Released February 22nd, 2025 Family Moonlight The lab's model card Compare with others
11
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Moonshot AI Feb 2025

Moonlight-16B-A3B

Open weights Small

109.3 #378 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • MMLU-Pro 42.4% #161 /167
  • C-Eval 77.2% #38 /46
  • MMLU 70% #100 /131

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
BIG-Bench Hard 65.2% #54 of 73 Lab model card
MMLU 70% #100 of 131 Lab model card
TriviaQA 66.3% #32 of 40 Lab model card
CMMLU 78.2% #14 of 17 Lab model card
C-Eval 77.2% #38 of 46 Lab model card
MMLU-Pro 42.4% #161 of 167 Lab model card

Math and science

BenchmarkScoreRankSource
GSM8K 77.4% #69 of 92 Lab model card
MATH 45.3% #75 of 99 Lab model card
CMATH 81.1% #5 of 6 Lab model card

Coding

BenchmarkScoreRankSource
MBPP 63.8% #52 of 71 Lab model card
HumanEval 48.1% #86 of 101 Lab model card

Its lab's numbers put it at 109.3, number 378 of 404

The Self-Reported Index fits one score to the 10 capability results Moonshot AI reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 107.0 to 111.5. How we compute it.

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.