Qwen2.5-Omni-7B

Qwen2.5-Omni-7B ranks first on 4 of the 55 benchmarks with five or more models scored.

Qwen Released March 27th, 2025 Family Qwen2.5-Omni The lab's model card Compare with others

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Qwen Mar 2025

Qwen2.5-Omni-7B

Open weights Small

120.3 #344 of 404

Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.

MMMU-Pro 36.6% #86 /90
MVBench 70.3% #19 /23
ODinW-13 42.4% #10 /13

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

Benchmark	Score		Rank	Source
MMLU-Pro	47%		#158 of 167	Lab model card
MMLU-Redux	71%		#44 of 45	Lab model card

Math and science

Benchmark	Score	Rank	Source
MATH	71.5%	#43 of 99	Lab model card
GSM8K	88.7%	#48 of 92	Lab model card
GPQA Diamond	30.8%	#293 of 302	Lab model card

Coding

Benchmark	Score	Rank	Source
MBPP	73.2%	#38 of 71	Lab model card
HumanEval	78.7%	#57 of 101	Lab model card
MultiPL-E	65.8%	#15 of 18	Lab model card
LiveCodeBench	24.6%	#48 of 54	Lab model card

Agents and tools

Benchmark	Score		Rank	Source
OmniGAIA	3.60%		#5 of 5	Official leaderboard

Professional work

Benchmark	Score		Rank	Source
MultiFinBen	33.5		#1 of 9	Official leaderboard

Vision, audio and robotics

Benchmark	Score	Rank	Source
OmniBench	56.1%	#1 of 16	Official leaderboard
DECIDE	66.6	#1 of 8	Official leaderboard
TextVQA	84.4%	#3 of 17	Lab model card
MMSU	60.6%	#1 of 5	Official leaderboard
DocVQA	95.2%	#11 of 36	Lab model card
JointAVBench	56.5%	#2 of 6	Official leaderboard
MMBench-Video	1.65	#12 of 28	Official leaderboard
SpatialViz-Bench	31.4%	#7 of 14	Official leaderboard
WorldSense (omnimodal)	45.4%	#6 of 12	Official leaderboard
ChartQA	85.3%	#22 of 37	Lab model card
EmoBench-M	53.7%	#9 of 15	Official leaderboard
AHELM	49.5	#6 of 10	Official leaderboard
MMAU-Pro	52.2%	#6 of 10	Official leaderboard
LVOmniBench	32%	#4 of 6	Official leaderboard
MME	2340.0	#4 of 6	Lab model card
CrossVid	24.6%	#9 of 13	Official leaderboard
UNO-Bench	30.2%	#5 of 7	Official leaderboard
RefCOCO / RefCOCO+ / RefCOCOg	87.7%	#21 of 29	Lab model card
MORSE-500	11.4%	#8 of 11	Official leaderboard
MathVista	67.9%	#51 of 70	Lab model card
MuirBench	59.2%	#9 of 12	Lab model card
MM-Vet	66.8%	#6 of 8	Lab model card
CRPE	76.5%	#3 of 4	Lab model card
ODinW-13	42.4%	#10 of 13	Lab model card
OCRBench v2	57.8%	#17 of 22	Lab model card
MMAU	65.6%	#12 of 15	Lab model card
AV-SpeakerBench	42.3%	#4 of 5	Official leaderboard
Video-Holmes	24.4%	#4 of 5	Official leaderboard
Video-MME	72.4%	#41 of 51	Lab model card
RealWorldQA	70.3%	#31 of 38	Lab model card
MVBench	70.3%	#19 of 23	Lab model card
AI2D	83.2%	#43 of 52	Lab model card
EgoSchema	68.6%	#10 of 12	Lab model card
MMMU	59.2%	#88 of 103	Lab model card
MMBench-EN v1.1	81.8%	#20 of 23	Lab model card
MMStar	64%	#32 of 36	Lab model card
CoreCognition	40.9%	#110 of 119	Official leaderboard
MMMU-Pro	36.6%	#86 of 90	Lab model card
MATH-Vision (MathVision)	25%	#51 of 51	Lab model card
Audio MC	11.9%	#18 of 18	Official leaderboard
CoVoST2	34.7	#14 of 14	Lab model card
VoiceBench	74.1%	#12 of 12	Lab model card
MM-MT-Bench	6.00	#6 of 6	Lab model card
OmniCap-IF	49.2%	#6 of 6	Official leaderboard
MME-RealWorld	61.6%	#4 of 4	Lab model card
MusTBench	24.3%	#4 of 4	Official leaderboard
QIVD	45.9%	#4 of 4	Official leaderboard

Other

Benchmark	Score		Rank	Source
LiveBench	29.6%		#27 of 27	Lab model card

Its lab's numbers put it at 120.3, number 344 of 404

The Self-Reported Index fits one score to the 29 capability results Qwen reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 118.1 to 122.5. How we compute it.

Its lab's number next to a leaderboard

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark	Lab says	Board says	Gap
OmniBench	56.1% OmniBench Avg (Speech 55.25 / Sound Event 60.00 / Music 52.83)	56.1% Leaderboard, image-audio	Same

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Qwen2.5-Omni-7B model card (Hugging Face) Mar 27, 2025

OmniBench Avg (Speech 55.25 / Sound Event 60.00 / Music 52.83) 56.1
Mean of Librispeech test-clean 1.8 / test-other 3.4 (dev-clean 1.6, dev-other 3.5 not used) 2.6
Mean of Common Voice 15 en 7.6 / zh 5.2 / yue 7.3 / fr 7.5 6.9
Mean of Fleurs zh 3.0 / en 4.1 3.55
Mean of CoVoST2 en-de 30.2 / de-en 37.7 / en-zh 41.4 / zh-en 29.4 34.7
MMAU Avg (Sound 67.87 / Music 69.16 / Speech 59.76) 65.6
VoiceBench Avg 74.1
SEED test-en content consistency, Qwen2.5-Omni-7B_RL (test-zh 1.42, test-hard 6.54) 2.32
MMMUval 59.2
MMMU-Prooverall 36.6
MathVistatestmini 67.9
MathVisionfull 25
MMBench-V1.1-ENtest 81.8
MMVetturbo 66.8
MMStar 64
MMEsum 2340
MuirBench 59.2
CRPErelation 76.5
RealWorldQAavg 70.3
MME-RealWorlden 61.6
MM-MT-Bench 6
AI2D 83.2
TextVQAval 84.4
DocVQAtest 95.2
ChartQAtest Avg 85.3
OCRBench_V2en 57.8
Mean of Refcoco val/testA/testB, Refcoco+ val/testA/testB, Refcocog val/test (90.5/93.5/86.6/85.4/91.0/79.3/87.4/87.9) 87.7
ODinW 42.4
Best of Video-MME w sub 72.4 / w/o sub 64.3 72.4
MVBench 70.3
EgoSchematest 68.6
MMLU-Pro 47
MMLU-redux 71
LiveBench0831 29.6
GPQA 30.8
MATH 71.5
GSM8K 88.7
HumanEval 78.7
MBPP 73.2
MultiPL-E 65.8
LiveCodeBench2305-2409 24.6
Mean of Wenetspeech test-net 5.9 / test-meeting 7.7 6.8

Its system card

Every result in the longer documents the lab published with it. 1 of them isn't in the launch post.

Qwen2.5-Omni Technical Report Mar 26, 2025

Capabilities

Hallucination

CRPE 76.5