Benchmark Almanac

Mercury Coder Small

Mercury Coder Small places in the top three on EvalPlus.

Inception Labs Released February 26th, 2025 Family Mercury The lab's model card Compare with others
7
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Inception Labs Feb 2025

Mercury Coder Small

Closed weights Flagship

122.2 #333 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • BigCodeBench 45.5% #10 /14
  • LiveCodeBench 25% #46 /54
  • HumanEval 90% #13 /101

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Coding

BenchmarkScoreRankSource
HumanEval 90% #13 of 101 Lab model card
MBPP 76.6% #30 of 71 Lab model card
MultiPL-E 76.2% #8 of 18 Lab model card
EvalPlus 80.4% #3 of 6 Lab model card
BigCodeBench 45.5% #10 of 14 Lab model card
LiveCodeBench 25% #46 of 54 Lab model card
HumanEval Infilling 84.8% #1 of 1 Lab model card

Its lab's numbers put it at 122.2, number 333 of 404

The Self-Reported Index fits one score to the 5 capability results Inception Labs reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 117.8 to 126.6. How we compute it.

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Its system card

Every result in the longer documents the lab published with it. 1 of them isn't in the launch post.