Benchmark Almanac

Whisper large-v3-turbo

Whisper large-v3-turbo places in the top three on 2 benchmarks.

OpenAI Released October 1st, 2024 Family Whisper The lab's model card Compare with others
13
Benchmarks with a score
0
Ranked first, out of five or more models
2
In the top three
OpenAI Oct 2024

Whisper large-v3-turbo

Open weights Small

No Epoch Capabilities Index score yet
  • VoiceCodeBench 57% #3 /9

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Vision, audio and robotics

BenchmarkScoreRankSource
PazaBench v2 73.3% #1 of 4 Official leaderboard
GAIK Ajokortti transcription test 5.00% #3 of 9 Official leaderboard
VoiceCodeBench 57% #3 of 9 Official leaderboard
Finnish Parliament ASR test 9.69% #2 of 4 Official leaderboard
GAIK LA-murre dialect test 62.7% #3 of 4 Official leaderboard
GAIK SKN dialect test 42.5% #3 of 4 Official leaderboard
SPGISpeech 2.79% #7 of 9 Official leaderboard
ContextASR-Bench 11% #7 of 7 Official leaderboard
Open ASR Leaderboard 7.83% #6 of 6 Official leaderboard
CoSE-E 17.6% #3 of 3 Official leaderboard
Contextual Earnings-22 14.7% #2 of 2 Official leaderboard
HALAS 29.4% #2 of 2 Official leaderboard
Swivuriso 15% #1 of 1 Official leaderboard

What does it cost to run?

On the one leaderboard that publishes a price, its best run costs about 1/5 of the median model's. No model comes close to its score for less.

Benchmark Score Price From cheapest to priciest Against the median Close to its score for less Close for less
VoiceCodeBench 57% $0.0007 0.2× the median 0.2× None

Who measured it?

Independent runs cover all 13 of its benchmarks.

  • An independent run only 13
Official leaderboards scored it on 13.

Sources: Official leaderboards, Epoch AI (CC BY 4.0) and labs’ model cards