AyaVisionBench

Aya Vision 32B 64%

Pixtral 12B 41.8% leads

Aya Vision 32B 64% leads

Dropped

Labs’ own numbers All independent runs Include claimed scores

Vision-language models answer image questions in 23 languages across 9 task categories, 135 image-question pairs per language. A model judge compares two models' answers and reports win rates.

Images come from the Cauldron held-out test set. Questions were human-checked in English and machine-translated into 22 other languages.

Made by: Cohere For AI (Cohere Labs)
Released: March 2025
Kind: General visual questions
Level: Everyday
Scored: Graded by a model
Answers: Open-ended
Tasks: All public

Last in a launch post: March 2025, dropped before it saturated
Last model added: May 2025
Scores refreshed: November 12th, 2026

Model scored in lab cards

Launch post lists it, from 1 lab

System card reports it, from 1 lab

Aya Vision 32B leads with 64%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Pixtral 12B leads with 41.8% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Aya Vision 32B leads with 64%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

Bars Table

64.0 Aya Vision 32B

1 Aya Vision 32B 64%

Source: Labs’ model cards

41.8 Pixtral 12B

1 Pixtral 12B reference Aya Vision 8B 41.8%

Source: Official AyaVisionBench leaderboard

1 Aya Vision 32B 64%
2 Pixtral 12B reference Aya Vision 8B 41.8%

Sources: Labs’ model cards and official AyaVisionBench leaderboard

#	Model	Score	Setting	Date	Source
1	Aya Vision 32B	64%		Mar 2025	Lab model card

#	Model	Score	Setting	Date	Source
1	Pixtral 12B	41.8%	reference Aya Vision 8B	Sep 2024	Official leaderboard

#	Model	Score	Setting	Date	Source
1	Aya Vision 32B	64%		Mar 2025	Lab model card
2	Pixtral 12B	41.8%	reference Aya Vision 8B	Sep 2024	Official leaderboard

The benchmark's own leaderboard has 6 entries, 1 of them models we track. Switch the scores above to rank them together.

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far

Sources: Official AyaVisionBench leaderboard and labs’ model cards

No launch post has listed it since March 2025

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Cohere

Date	Launch post	Score	As the post writes it
Mar 4, 2025	Aya Vision 32B	64	Up to 64% win rate on AyaVisionBench (best pairwise win rate against Llama-3.2 90B Vision, Molmo 72B, Qwen2-VL 72B)

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

Date	Document	Score	As the document writes it	Section
May 13, 2025	Aya Vision 32B technical report		AyaVisionBench, pairwise win rates averaged over 23 languages, no overall: vs Llama-3.2-90B-Vision 65.9, vs Molmo-72B 61.2, vs Qwen-2.5-VL-72B 48.5 (Figure 7, printed labels)	Capabilities