Benchmark Almanac

BLIP-2 FLAN-T5-XL

BLIP-2 FLAN-T5-XL ranks first on SciMMIR.

Salesforce Released February 6th, 2023 Family BLIP-2 The lab's model card Compare with others
6
Benchmarks with a score
1
Ranked first, out of five or more models
1
In the top three
Salesforce Feb 2023

BLIP-2 FLAN-T5-XL

Open weights Small

No Epoch Capabilities Index score yet
  • CoreCognition 50% #97 /121
  • MMCircuitEval 14.3% #16 /21

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Math and science

BenchmarkScoreRankSource
TEM-VQA 12.3 #8 of 8 Official leaderboard

Professional work

BenchmarkScoreRankSource
OmniMedVQA 50.7% #1 of 2 Official leaderboard
MMCircuitEval 14.3% #16 of 21 Official leaderboard

Vision, audio and robotics

BenchmarkScoreRankSource
SciMMIR 3.35 #1 of 5 Official leaderboard
CoreCognition 50% #97 of 121 Official leaderboard
MMMU 34.4% #132 of 137 Official leaderboard

Who measured it?

Independent runs cover all 6 of its benchmarks.

  • An independent run only 6
Official leaderboards scored it on 6.

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards