Phi-4 ranks first on Microsoft RAI benchmark: Jailbreak (DR-1).
Microsoft Dec 2024 Open weights Small
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (2025 dataset) | 3.70% | #3 of 104 | Official leaderboard | |
| MuSR | 50.3% | #3 of 98 | Official leaderboard | |
| MMLU-CF | 68.5% | #8 of 37 | Official leaderboard | |
| ALRAGE | 77.2% | #14 of 39 | Official leaderboard | |
| QIMMA | 56.6% | #9 of 25 | Official leaderboard | |
| Global PIQA | 74.2% | #16 of 44 | Official leaderboard | |
| FaithJudge | 17% | #16 of 37 | Official leaderboard | |
| MMLU | 84.8% | #57 of 131 | Lab model card | |
| ArbMMLU-HT | 39.1% | #21 of 39 | Official leaderboard | |
| Arena-Hard-Auto | 75.4% | #22 of 39 | Lab model card | |
| MGSM | 80.6% | #33 of 57 | Lab model card | |
| Vectara Hallucination Leaderboard (HHEM) | 4.70% | #88 of 148 | Official leaderboard | |
| MadinahQA | 39.2% | #24 of 39 | Official leaderboard | |
| ArabicMMLU | 39.8% | #25 of 39 | Official leaderboard | |
| SuperGPQA | 37.6% | #40 of 59 | Official leaderboard | |
| Wer wird Millionär? LLM Benchmark | 349.0 | #53 of 77 | Official leaderboard | |
| Open Arabic LLM Leaderboard v2 | 45.7% | #28 of 39 | Official leaderboard | |
| MMLU-Pro | 70.4% | #121 of 167 | Lab model card | |
| AraGen | 30% | #31 of 42 | Official leaderboard | |
| BuzzBench | 38.7 | #15 of 20 | Official leaderboard | |
| LMArena (Elo) | 1,256 | #261 of 335 | Official leaderboard | |
| DROP | 75.5% | #32 of 41 | Lab model card | |
| Stick to Your Role! | 0.22 | #12 of 15 | Official leaderboard | |
| AlGhafa | 42.3% | #32 of 39 | Official leaderboard | |
| Structured Output Benchmark | 83.1% | #17 of 19 | Official leaderboard | |
| LLM Confabulation Leaderboard | 29.4% | #53 of 58 | Official leaderboard | |
| IFEval | 63% | #129 of 135 | Lab model card | |
| SimpleQA | 3.00% | #59 of 60 | Lab model card | |
| Microsoft RAI benchmark: Grounding | 4.62 | #1 of 1 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PHYSICS | 29.1% | #4 of 20 | Official leaderboard | |
| MATH | 80.4% | #26 of 99 | Lab model card | |
| MATH Level 5 | 64.9% | #38 of 94 | Epoch AI run | |
| U-MATH | 54.5% | #13 of 30 | Official leaderboard | |
| GPQA Diamond | 56.1% | #144 of 229 | Epoch AI run | |
| 3SAT-Search | 17.1% | #8 of 11 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 13.8% | #160 of 212 | Epoch AI run | |
| PolyMath | 17.4% | #35 of 38 | Official leaderboard | |
| TSP-Opt | 8.15% | #11 of 11 | Official leaderboard | |
| AMC 10/12 (November 2024) | 91.8 | #1 of 1 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench-Hard | 27.4% | #30 of 96 | Official leaderboard | |
| BigCodeBench | 50.5% | #22 of 69 | Official leaderboard | |
| HumanEval | 82.6% | #49 of 101 | Lab model card | |
| HumanEval+ | 82.8% | #17 of 27 | Lab model card | |
| BIRD-CRITIC | 22% | #6 of 9 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BFCL V4 | 28.8% | #41 of 65 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 47% | #55 of 67 | Official leaderboard | |
| SnakeBench | 18 | #156 of 188 | Official leaderboard | |
| Elimination Game | 3 | #48 of 56 | Official leaderboard | |
| BALROG | 11.6% | #29 of 33 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CSVQA | 11.5% | #7 of 12 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Microsoft RAI benchmark: Jailbreak (DR-1) | 7.30% | #1 of 5 | Lab model card | |
| Microsoft RAI benchmark: Harmful Content Summarization (DR-3) | 10.2% | #2 of 5 | Lab model card | |
| Microsoft RAI benchmark: Third Party Harm (DR-1) | 12.1% | #2 of 5 | Lab model card | |
| AraTrust | 47.6% | #30 of 39 | Official leaderboard | |
| nocot-bench (NCRI) | 58.3 | #145 of 185 | Official leaderboard | |
| Microsoft RAI benchmark: Harmful Content Continuation (DR-3) | 3.60% | #5 of 5 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 30.4 | #25 of 99 | Official leaderboard | |
| LiveBench | 47.6% | #21 of 27 | Lab model card | |
| PhiBench | 56.2% | #3 of 3 | Lab model card |
The Self-Reported Index fits one score to the 11 capability results Microsoft reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 121.9 to 128.3. How we compute it.
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
Microsoft's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
The benchmarks the lab chose to headline, with the scores it gave.
Every result in the longer documents the lab published with it. 8 of them aren't in the launch post.