Benchmark Almanac

Aya Vision 32B

Aya Vision 32B.

Cohere Released March 4th, 2025 Family Aya Vision The lab's model card Compare with others
14
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Cohere Mar 2025

Aya Vision 32B

Open weights Mid-size

No Epoch Capabilities Index score yet
  • MTVQA-Test 23.5% #6 /6
  • Kaleidoscope 39.3% #4 /7
  • IFEval 78.5% #109 /135

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
Kaleidoscope 39.3% #4 of 7 Official leaderboard
MGSM 79.5% #37 of 57 Lab model card
Flores 37.8 #6 of 8 Lab model card
IFEval 78.5% #109 of 135 Lab model card
Global MMLU Lite 63.6% #14 of 16 Lab model card
CVQA 74.1% #1 of 1 Lab model card

Vision, audio and robotics

BenchmarkScoreRankSource
MMMU 45.1% #104 of 136 Official leaderboard
Vision Arena 1,001 #128 of 136 Official leaderboard
MTVQA-Test 23.5% #6 of 6 Lab model card
AyaVisionBench 64% #1 of 1 Lab model card
m-WildVision 72% #1 of 1 Lab model card
MaXM 62.3% #1 of 1 Lab model card
xChatBench 70.1 #1 of 1 Lab model card
xMMMU 45.1% #1 of 1 Lab model card

Its lab's number next to a leaderboard

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark Lab says Board says Gap
Kaleidoscope 41.7% Kaleidoscope, Aya Vision languages (Table 4) 39.3% Leaderboard +2.5 pts

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Its system card

Every result in the longer documents the lab published with it. 11 of them aren't in the launch post.