Benchmark Almanac

Phi-4 Mini

Phi-4 Mini.

Microsoft Released February 27th, 2025 Compare with others
44
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Microsoft Feb 2025

Phi-4 Mini

Open weights

113.0 #368 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • BigCodeBench 43% #12 /14
  • StructEval 57% #26 /28
  • AlGhafa 69.3% #12 /39

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
SIQA 72.5% #6 of 24 Lab model card
AlGhafa 69.3% #12 of 39 Official leaderboard
TruthfulQA 66.4% #10 of 30 Lab model card
OpenBookQA 79.2% #8 of 24 Lab model card
Vectara Hallucination Leaderboard (HHEM) 3.40% #54 of 148 Official leaderboard
Open Arabic LLM Leaderboard v2 55.8% #15 of 39 Official leaderboard
MadinahQA 47.1% #17 of 39 Official leaderboard
ArabicMMLU 49.5% #18 of 39 Official leaderboard
ArbMMLU-HT 42.2% #20 of 39 Official leaderboard
ARC-Challenge 83.7% #26 of 49 Lab model card
BoolQ 81.2% #15 of 28 Lab model card
ALRAGE 69.1% #22 of 39 Official leaderboard
BIG-Bench Hard 70.4% #48 of 73 Lab model card
Qasper 40.4% #2 of 3 Lab model card
MMLU 67.3% #106 of 131 Lab model card
FaithJudge 38.4% #30 of 37 Official leaderboard
MGSM 63.9% #48 of 57 Lab model card
PIQA 77.6% #33 of 37 Lab model card
MMLU-Pro 52.8% #152 of 167 Lab model card
WinoGrande 67% #43 of 47 Lab model card
IFEval 70.1% #124 of 135 Lab model card
MMMLU 49.3% #66 of 71 Lab model card
HellaSwag 69.1% #50 of 53 Lab model card
Wer wird Millionär? LLM Benchmark 58.0 #75 of 77 Official leaderboard
Vectara Hallucination Leaderboard (2025 dataset) 23.5% #103 of 104 Official leaderboard
Arena-Hard-Auto 32.8% #39 of 39 Lab model card
SQuALITY 22.8% #3 of 3 Lab model card

Math and science

BenchmarkScoreRankSource
GSM8K 88.6% #49 of 92 Lab model card
MATH 64% #54 of 99 Lab model card
MATH-500 71.8% #65 of 70 Lab model card
GPQA Diamond 30.4% #294 of 302 Lab model card
AIME 2024 10% #86 of 88 Lab model card

Coding

BenchmarkScoreRankSource
HumanEval 74.4% #64 of 101 Lab model card
MBPP 65.3% #50 of 71 Lab model card
BigCodeBench 43% #12 of 14 Lab model card
LiveCodeBench 19.9% #49 of 54 Lab model card
HumanEval+ 68.3% #25 of 27 Lab model card
StructEval 57% #26 of 28 Official leaderboard
MBPP+ 63.8% #14 of 15 Lab model card
Spider 42.2% #4 of 4 Lab model card

Agents and tools

BenchmarkScoreRankSource
Berkeley Function Calling Leaderboard (BFCL) 70.3% #17 of 41 Lab model card

Safety and security

BenchmarkScoreRankSource
XSTest 20.8% #6 of 16 Lab model card
AraTrust 77.8% #17 of 39 Official leaderboard
Harmful content defect rate (Azure AI Evaluation SDK, Microsoft) 3.75% #2 of 3 Lab model card

Its lab's numbers put it at 113.0, number 368 of 404

The Self-Reported Index fits one score to the 27 capability results Microsoft reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 110.8 to 115.3. How we compute it.

Its lab's numbers next to 2 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark Lab says Board says Gap
Berkeley Function Calling Leaderboard (BFCL) 70.3% BFCL (0-shot), version not stated (Table 7) 31.3% Leaderboard, prompted +39 pts
MMLU-Pro 52.8% MMLU-Pro (0-shot, CoT) 52.8% Leaderboard Same

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Its system card

Every result in the longer documents the lab published with it. 16 of them aren't in the launch post.

Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs Mar 3, 2025

Capabilities

Hallucination

Safety