granite-3.2-8b-instruct model card (Hugging Face) Feb 26, 2025
- ArenaHard 55.3
- Alpaca-Eval-2 61.2
- MMLU 66.8
- PopQA 28
- TruthfulQA 66.9
- BigBenchHard 64.8
- DROP 51
- GSM8K 81.7
- HumanEval 89.3
- HumanEval+ 85.7
- IFEval 74.3
- AttaQ 85.4
Granite 3.2 8B Instruct
Granite 3.2 8B Instruct.
16
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
IBM Feb 2025
Granite 3.2 8B Instruct
Open weights Flagship
116.3
#360 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
- PopQA 28% #5 /7
- IFEval 74.3% #121 /135
- MMLU 66.8% #107 /131
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MuSR | 45.6% | #11 of 98 | Official leaderboard | |
| TruthfulQA | 66.9% | #8 of 30 | Lab model card | |
| AlpacaEval 2.0 | 61.2% | #6 of 16 | Lab model card | |
| PopQA | 28% | #5 of 7 | Lab model card | |
| BIG-Bench Hard | 64.8% | #55 of 73 | Lab model card | |
| MMLU | 66.8% | #107 of 131 | Lab model card | |
| Arena-Hard-Auto | 55.3% | #32 of 39 | Lab model card | |
| Vectara Hallucination Leaderboard (HHEM) | 8.70% | #126 of 148 | Official leaderboard | |
| IFEval | 74.3% | #121 of 135 | Lab model card | |
| DROP | 51% | #39 of 41 | Lab model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 81.7% | #62 of 92 | Lab model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 89.3% | #15 of 101 | Lab model card | |
| HumanEval+ | 85.7% | #14 of 27 | Lab model card |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BFCL V4 | 26.9% | #49 of 65 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AttaQ | 85.4% | #4 of 5 | Lab model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 30.8 | #23 of 99 | Official leaderboard |
Its lab's numbers put it at 116.3, number 360 of 404
The Self-Reported Index fits one score to the 10 capability results IBM reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 112.1 to 120.5. How we compute it.
- HumanEval 89.3
- HumanEval+ 85.7
- GSM8K 81.7
- IFEval 74.3
- TruthfulQA 66.9
- MMLU 66.8
- BIG-Bench Hard 64.8
- AlpacaEval 2.0 61.2
- Arena-Hard-Auto 55.3
- DROP 51
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.