Benchmark Almanac

Llemma 7B

Llemma 7B.

EleutherAI Released October 16th, 2023 Family Llemma The lab's model card Compare with others
3
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
EleutherAI Oct 2023

Llemma 7B

Open weights Small

No Epoch Capabilities Index score yet

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
MMLU-Pro 23.4% #191 of 212 Official leaderboard

Math and science

BenchmarkScoreRankSource
TheoremQA 17.2% #4 of 4 Official leaderboard

Other

BenchmarkScoreRankSource
Open LLM Leaderboard v1 48.8 #56 of 76 Official leaderboard

Who measured it?

Independent runs cover all 3 of its benchmarks.

  • An independent run only 3
Official leaderboards scored it on 3.

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards