Benchmark Almanac

Granite 3.3 8B Instruct

Granite 3.3 8B Instruct ranks first on AttaQ.

IBM Released April 16th, 2025 Family Granite 3.3 The lab's model card Compare with others
23
Benchmarks with a score
1
Ranked first, out of five or more models
1
In the top three
IBM Apr 2025

Granite 3.3 8B Instruct

Open weights Flagship

115.5 #366 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
  • PopQA 26.2% #6 /7
  • GPQA Main 32.5% #54 /60
  • QIMMA 46.5% #17 /25

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
TruthfulQA 66.9% #9 of 30 Lab model card
AlpacaEval 2.0 62.7% #5 of 16 Lab model card
Vectara Hallucination Leaderboard (2025 dataset) 10.6% #66 of 104 Official leaderboard
BIG-Bench Hard 69.1% #49 of 73 Lab model card
QIMMA 46.5% #17 of 25 Official leaderboard
Arena-Hard-Auto 57.6% #31 of 39 Lab model card
MMLU 65.5% #110 of 131 Lab model card
PopQA 26.2% #6 of 7 Lab model card
IFEval 74.8% #119 of 135 Lab model card
DROP 59.4% #38 of 41 Lab model card

Math and science

BenchmarkScoreRankSource
GSM8K 80.9% #63 of 92 Lab model card
GPQA Main 32.5% #54 of 60 Official leaderboard
MATH-500 69% #66 of 70 Lab model card
AIME 2024 8.12% #87 of 88 Lab model card

Coding

BenchmarkScoreRankSource
HumanEval 89.7% #14 of 101 Lab model card
HumanEval+ 86.1% #13 of 27 Lab model card

Professional work

BenchmarkScoreRankSource
AssetOpsBench 35% #3 of 3 Official leaderboard

Safety and security

BenchmarkScoreRankSource
AttaQ 88.5% #1 of 5 Lab model card
AIR-Bench 2024 82.5% #19 of 82 Official leaderboard
Anthropic Red Team 99.1% #33 of 75 Official leaderboard
HELM Safety 90.5 #45 of 75 Official leaderboard
SimpleSafetyTests 97.5% #51 of 75 Official leaderboard

Other

BenchmarkScoreRankSource
HELM Capabilities 46.3 #57 of 60 Official leaderboard

Its lab's numbers put it at 115.5, number 366 of 404

The Self-Reported Index fits one score to the 12 capability results IBM reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 112.3 to 118.6. How we compute it.

Its lab's number next to a leaderboard

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark Lab says Board says Gap
IFEval 74.8% IFEval 72.9% Leaderboard, strict, share of instructions followed +1.9 pts

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.