Benchmark Almanac

Open ASR Leaderboard

Phi-4-multimodal 6.14%
Cohere Transcribe 5.42% leads since March 2026
Fading

Speech recognition models transcribe the test sets of several English and multilingual corpora. Scored by average word error rate, with speed reported next to it.

Built on the ESB test sets. ASR model cards often report its average WER.

Released
September 2023
Kind
Speech recognition and translation
Level
Everyday
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Last in a launch post
March 2026, dropped before it saturated
Last model added
October 2026
Scores refreshed
November 12th, 2026

In the authors’ words

We present the Open ASR Leaderboard, a reproducible benchmarking platform with community contributions from academia and industry. It compares 86 open-source and proprietary systems across 12 datasets, with English short- and long-form and multilingual short-form tracks. We standardize word error rate (WER) and inverse real-time factor (RTFx) evaluation for consistent accuracy-efficiency comparisons across model architectures and toolkits (e.g., ESPNet, NeMo, SpeechBrain, Transformers). We observe that Conformer-based encoders paired with transformer-based decoders achieve the best average WER, while connectionist temporal classification (CTC) and token-and-duration transducer (TDT) decoders offer superior RTFx, making them better suited for long-form and batched processing. All code and dataset loaders are open-sourced to support transparent, extensible evaluation. We present our evaluation methodology to facilitate community-driven benchmarking in ASR and other tasks.

Abstract of Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual and Long-Form Speech Recognition Evaluation (arXiv:2510.06961, CC0)
1
Model scored in lab cards
1
Launch post lists it, from 1 lab
3
System cards report it, from 2 labs

Phi-4-multimodal leads with 6.14%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Cohere Transcribe leads with 5.42% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

  1. 1 Phi-4-multimodal 6.14%

Source: Labs’ model cards

Source: Official Open ASR Leaderboard leaderboard

#ModelScoreSettingDateSource
1 Phi-4-multimodal 6.14% Mar 2025 Lab model card
#ModelScoreSettingDateSource
1 Cohere Transcribe 5.42% Mar 2026 Official leaderboard
2 Qwen3-ASR-1.7B 5.76% Jan 2026 Official leaderboard
3 Phi-4-multimodal 6.02% Feb 2025 Official leaderboard
4 Voxtral Small 6.62% Jul 2025 Official leaderboard

The benchmark's own leaderboard has 25 entries, 4 of them models we track. Switch the scores above to rank them together.

WER versus RTFx

The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

6% 8% 10% 02,0004,0006,0008,000 Real-time factor Word error rate Phi-4-multimodal: Word error rate 6.02%, real-time factor 151 Voxtral Small: Word error rate 6.62%, real-time factor 54.1 Qwen3-ASR-1.7B: Word error rate 5.76%, real-time factor 148 Meta Omnilingual ASR LLM 7B v2: Word error rate 8.14%, real-time factor 66. Not in our model list OpenAI Whisper Large v3 Turbo: Word error rate 7.83%, real-time factor 200. Not in our model list OpenAI Whisper Large v3: Word error rate 7.44%, real-time factor 146. Not in our model list NVIDIA Parakeet CTC 1.1B: Word error rate 7.4%, real-time factor 2,730. Not in our model list Distil-Whisper Large v3.5: Word error rate 7.21%, real-time factor 202. Not in our model list NVIDIA Canary 1B v2: Word error rate 7.15%, real-time factor 749. Not in our model list Nyra Health CrisperWhisper: Word error rate 6.67%, real-time factor 84.1. Not in our model list NVIDIA Canary 1B: Word error rate 6.5%, real-time factor 235. Not in our model list NVIDIA Parakeet TDT 0.6B v3: Word error rate 6.32%, real-time factor 3,330. Not in our model list IBM Granite Speech 3.3 2B: Word error rate 6%, real-time factor 271. Not in our model list IBM Granite Speech 3.3 8B: Word error rate 5.76%, real-time factor 145. Not in our model list NVIDIA Canary Qwen 2.5B: Word error rate 5.63%, real-time factor 418. Not in our model list IBM Granite Speech 4.0 1B: Word error rate 5.52%, real-time factor 280. Not in our model list Cohere Transcribe: Word error rate 5.42%, real-time factor 525 NVIDIA FastConformer CTC Large: Word error rate 8.96%, real-time factor 6,400. Not in our model list NVIDIA Parakeet TDT 0.6B v2: Word error rate 6.05%, real-time factor 3,390. Not in our model list NVIDIA FastConformer CTC LargeNVIDIA Parakeet TDT 0.6B v2Cohere Transcribe 6% 8% 10% 02,0004,0006,0008,000 Real-time factor Word error rate Phi-4-multimodal: Word error rate 6.02%, real-time factor 151 Voxtral Small: Word error rate 6.62%, real-time factor 54.1 Qwen3-ASR-1.7B: Word error rate 5.76%, real-time factor 148 Meta Omnilingual ASR LLM 7B v2: Word error rate 8.14%, real-time factor 66. Not in our model list OpenAI Whisper Large v3 Turbo: Word error rate 7.83%, real-time factor 200. Not in our model list OpenAI Whisper Large v3: Word error rate 7.44%, real-time factor 146. Not in our model list NVIDIA Parakeet CTC 1.1B: Word error rate 7.4%, real-time factor 2,730. Not in our model list Distil-Whisper Large v3.5: Word error rate 7.21%, real-time factor 202. Not in our model list NVIDIA Canary 1B v2: Word error rate 7.15%, real-time factor 749. Not in our model list Nyra Health CrisperWhisper: Word error rate 6.67%, real-time factor 84.1. Not in our model list NVIDIA Canary 1B: Word error rate 6.5%, real-time factor 235. Not in our model list NVIDIA Parakeet TDT 0.6B v3: Word error rate 6.32%, real-time factor 3,330. Not in our model list IBM Granite Speech 3.3 2B: Word error rate 6%, real-time factor 271. Not in our model list IBM Granite Speech 3.3 8B: Word error rate 5.76%, real-time factor 145. Not in our model list NVIDIA Canary Qwen 2.5B: Word error rate 5.63%, real-time factor 418. Not in our model list IBM Granite Speech 4.0 1B: Word error rate 5.52%, real-time factor 280. Not in our model list Cohere Transcribe: Word error rate 5.42%, real-time factor 525 NVIDIA FastConformer CTC Large: Word error rate 8.96%, real-time factor 6,400. Not in our model list NVIDIA Parakeet TDT 0.6B v2: Word error rate 6.05%, real-time factor 3,390. Not in our model list NVIDIA FastConformer CTC LargeNVIDIA Parakeet TDT 0.6B v2Cohere Transcribe
ModelSettingWord error rateReal-time factor
Cohere Transcribe 5.4% 525
IBM Granite Speech 4.0 1B 5.5% 280
NVIDIA Canary Qwen 2.5B 5.6% 418
IBM Granite Speech 3.3 8B 5.8% 145
Qwen3-ASR-1.7B 5.8% 148
IBM Granite Speech 3.3 2B 6% 271
Phi-4-multimodal 6% 151
NVIDIA Parakeet TDT 0.6B v2 6.1% 3,390
NVIDIA Parakeet TDT 0.6B v3 6.3% 3,330
NVIDIA Canary 1B 6.5% 235
Voxtral Small 6.6% 54.1
Nyra Health CrisperWhisper 6.7% 84.1
NVIDIA Canary 1B v2 7.2% 749
Distil-Whisper Large v3.5 7.2% 202
NVIDIA Parakeet CTC 1.1B 7.4% 2,730
OpenAI Whisper Large v3 7.4% 146
OpenAI Whisper Large v3 Turbo 7.8% 200
Meta Omnilingual ASR LLM 7B v2 8.1% 66
NVIDIA FastConformer CTC Large 9% 6,400

Number one over time 3 models have held number one across all independent runs since its release in September 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Microsoft Qwen Cohere
Own leaderboard Not tested since Mar 2026
Cohere Transcribe 5.42%

Source: Official Open ASR Leaderboard leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Cohere Transcribe Phi-4-multimodal 0% 20% 40% 60% 80% 100% 2025 2026 Cohere Transcribe Phi-4-multimodal

Sources: Official Open ASR Leaderboard leaderboard and labs’ model cards

Labs headline it less often now, last in March 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Cohere
DateLaunch postScoreAs the post writes it
Mar 26, 2026 Cohere Transcribe 5.42 Hugging Face Open ASR Leaderboard average WER, as of 2026-03-26

3 system cards report it, from 2 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 23, 2026 Nemotron 3.5 Bolt E4B technical report 9.28 OpenASR (WER), lower is better, non-reasoning (Table 8) Capabilities
Apr 27, 2026 Nemotron 3 Nano Omni 30B-A3B technical report 5.95 OpenASR Avg WER, English subset, lower is better, reasoning off (Table 8). AMI 11.09, Earnings22 11.27, GigaSpeech 9.66, LibriSpeech clean 1.57, other 2.96, SPGISpeech 1.98, TED-LIUM 3.44, VoxPopuli 5.6 Capabilities
Mar 3, 2025 Phi-4-multimodal technical report 6.14 OpenASR, average WER over 8 sets, lower is better (Tables 3 and 4) Capabilities

One lab claim next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Phi-4-multimodal 6.14% OpenASR, average WER over 8 sets, lower is better (Tables 3 and 4) 6.02% Leaderboard Same

More in Audio and video

All 232 in Audio and video
  • Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy. Qwen 3.8 Max 88.7%
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better. Phi-4-multimodal 4.00%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%