Benchmark Almanac

Falcon-H1-34B

Falcon-H1-34B places in the top three on 3 benchmarks.

Technology Innovation Institute Released May 20th, 2025 Family Falcon-H1 The lab's model card Compare with others
27
Benchmarks with a score
0
Ranked first, out of five or more models
3
In the top three
Technology Innovation Institute May 2025

Falcon-H1-34B

Open weights Flagship

118.3 #355 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • Alyah 71.1% #4 /10
  • AraGen 45.7% #24 /42
  • Okapi MMLU 77.8% #3 /8

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
MT-Bench 9.20 #3 of 23 Lab model card
IFEval 89.4% #41 of 135 Lab model card
TruthfulQA 65.3% #11 of 30 Lab model card
Okapi MMLU 77.8% #3 of 8 Lab model card
Alyah 71.1% #4 of 10 Official leaderboard
MMLU 84% #59 of 131 Lab model card
MMLU-stem 83.6% #1 of 2 Lab model card
AraGen 45.7% #24 of 42 Official leaderboard
BIG-Bench Hard 70.7% #46 of 73 Lab model card
HellaSwag 81.9% #35 of 53 Lab model card
MGSM 76.3% #38 of 57 Lab model card
AlpacaEval 2.0 48.3% #11 of 16 Lab model card
ARC-Challenge 61% #37 of 49 Lab model card
MMLU-Pro 58.7% #146 of 167 Lab model card

Math and science

BenchmarkScoreRankSource
AMC 69.4% #1 of 4 Lab model card
GPQA Main 41.5% #4 of 7 Lab model card
GSM8K 83.6% #60 of 92 Lab model card
MATH-500 83.8% #57 of 70 Lab model card
GPQA Diamond 49.7% #247 of 302 Lab model card
AIME 2024 23.8% #83 of 88 Lab model card
AIME 2025 16.7% #140 of 143 Lab model card

Coding

BenchmarkScoreRankSource
MBPP 83.9% #14 of 71 Lab model card
HumanEval 87.2% #32 of 101 Lab model card
CRUXEval 73.1% #3 of 9 Lab model card
MBPP+ 71.4% #9 of 15 Lab model card
HumanEval+ 81.7% #19 of 27 Lab model card

Other

BenchmarkScoreRankSource
LiveBench 46.3% #22 of 27 Lab model card

Its lab's numbers put it at 118.3, number 355 of 404

The Self-Reported Index fits one score to the 20 capability results Technology Innovation Institute reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 115.8 to 120.8. How we compute it.

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Its system card

Every result in the longer documents the lab published with it. 6 of them aren't in the launch post.