Benchmark Almanac

VoiceCodeBench

GPT-Live-Transcribe 67.7% leads since August 2026
Not in model cards

Speech-to-text models transcribe 300 English workplace recordings. They are scored on whether every target entity, such as an address or email, survives exactly.

5.6 hours of audio with 1,482 audited entities across 26 types and 8 workflow domains. Metrics are task success rate, CTEM and WER. 20 systems on September 24th, 2026.

Released
August 2026
Kind
Speech recognition and translation
Level
Everyday
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Last model added
September 2026
Scores refreshed
November 12th, 2026
20
Entries on Vals AI’s leaderboard, 9 models we track

GPT-Live-Transcribe leads with 67.7% among models released by September 2026

Best score per entry from Vals AI’s leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

20 entries. Scroll sideways for the rest.

  1. 67.7 GPT-Live-Transcribe
  2. 65.3 Grok Voice Transcribe 2.0
  3. 62.0 Cartesia Ink 2
  4. 59.7 gpt-4o-transcribe
  5. 59.3 Deepgram Nova-3 (Streaming)
  6. 59.0 Reson8 Resonant-1
  7. 57.3 Inworld STT 1
  8. 57.0 Whisper large-v3-turbo
  9. 57.0 Muse Voice Transcribe 1.0
  10. 55.7 Deepgram Flux General English
  11. 55.3 ElevenLabs Scribe v2 Realtime
  12. 55.3 Google Cloud Chirp 3
  13. 54.3 Whisper large-v3
  14. 53.0 Voxtral Mini Transcribe 2
  15. 52.0 Grok Voice Transcribe 1.0
  16. 51.7 gpt-4o-mini-transcribe
  17. 46.0 Voxtral Mini Transcribe Realtime (2602)
  18. 44.7 Cohere Transcribe
  19. 37.3 Azure Speech Universal Language Model
  20. 30.3 AssemblyAI Universal-3.5 Pro
  1. 1 GPT-Live-Transcribe 67.7%
  2. 2 Grok Voice Transcribe 2.0 65.3%
  3. 3 Cartesia Ink 2 62%
  4. 4 gpt-4o-transcribe 59.7%
  5. 5 Deepgram Nova-3 (Streaming) 59.3%
  6. 6 Reson8 Resonant-1 59%
  7. 7 Inworld STT 1 57.3%
  8. 8 Whisper large-v3-turbo 57%
  9. 9 Muse Voice Transcribe 1.0 57%
  10. 10 Deepgram Flux General English 55.7%
  11. 11 ElevenLabs Scribe v2 Realtime 55.3%
  12. 12 Google Cloud Chirp 3 55.3%
  13. 13 Whisper large-v3 54.3%
  14. 14 Voxtral Mini Transcribe 2 53%
  15. 15 Grok Voice Transcribe 1.0 52%
  16. 16 gpt-4o-mini-transcribe 51.7%
  17. 17 Voxtral Mini Transcribe Realtime (2602) 46%
  18. 18 Cohere Transcribe 44.7%
  19. 19 Azure Speech Universal Language Model 37.3%
  20. 20 AssemblyAI Universal-3.5 Pro 30.3%

Source: Vals AI’s leaderboard

#EntryScoreSettingDateSource
1 GPT-Live-Transcribe 67.7% Jul 2026 Official leaderboard
2 Grok Voice Transcribe 2.0 65.3% Official leaderboard
3 Cartesia Ink 2 62% Official leaderboard
4 gpt-4o-transcribe 59.7% Mar 2025 Official leaderboard
5 Deepgram Nova-3 (Streaming) 59.3% Official leaderboard
6 Reson8 Resonant-1 59% Official leaderboard
7 Inworld STT 1 57.3% Official leaderboard
8 Whisper large-v3-turbo 57% Oct 2024 Official leaderboard
9 Muse Voice Transcribe 1.0 57% Sep 2026 Official leaderboard
10 Deepgram Flux General English 55.7% Official leaderboard
11 ElevenLabs Scribe v2 Realtime 55.3% Official leaderboard
12 Google Cloud Chirp 3 55.3% Official leaderboard
13 Whisper large-v3 54.3% Nov 2023 Official leaderboard
14 Voxtral Mini Transcribe 2 53% Feb 2026 Official leaderboard
15 Grok Voice Transcribe 1.0 52% Official leaderboard
16 gpt-4o-mini-transcribe 51.7% Mar 2025 Official leaderboard
17 Voxtral Mini Transcribe Realtime (2602) 46% Feb 2026 Official leaderboard
18 Cohere Transcribe 44.7% Mar 2026 Official leaderboard
19 Azure Speech Universal Language Model 37.3% Official leaderboard
20 AssemblyAI Universal-3.5 Pro 30.3% Official leaderboard

TSR against latency

The Vals scatter chart with its x axis switched from Cost to Latency: mean time per task against task success rate, on a log scale. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on Vals AI’s leaderboard.

40% 60% 80% 110100 Latency (mean per test) (s), log scale, lower is better Task success rate (TSR) Whisper large-v3: Task success rate (TSR) 54.3%, latency (mean per test) 2.2 s Whisper large-v3-turbo: Task success rate (TSR) 57%, latency (mean per test) 2.95 s gpt-4o-mini-transcribe: Task success rate (TSR) 51.7%, latency (mean per test) 2.22 s Voxtral Mini Transcribe 2: Task success rate (TSR) 53%, latency (mean per test) 1.89 s Voxtral Mini Transcribe Realtime (2602): Task success rate (TSR) 46%, latency (mean per test) 15.4 s Cohere Transcribe: Task success rate (TSR) 44.7%, latency (mean per test) 1.95 s Muse Voice Transcribe 1.0: Task success rate (TSR) 57%, latency (mean per test) 67.2 s Cartesia Ink 2: Task success rate (TSR) 62%, latency (mean per test) 69.2 s. Not in our model list Deepgram Nova-3 (Streaming): Task success rate (TSR) 59.3%, latency (mean per test) 67.5 s. Not in our model list Inworld STT 1: Task success rate (TSR) 57.3%, latency (mean per test) 70.5 s. Not in our model list Deepgram Flux General English: Task success rate (TSR) 55.7%, latency (mean per test) 68.8 s. Not in our model list ElevenLabs Scribe v2 Realtime: Task success rate (TSR) 55.3%, latency (mean per test) 77.7 s. Not in our model list Google Cloud Chirp 3: Task success rate (TSR) 55.3%, latency (mean per test) 68.1 s. Not in our model list Grok Voice Transcribe 1.0: Task success rate (TSR) 52%, latency (mean per test) 67.8 s. Not in our model list Azure Speech Universal Language Model: Task success rate (TSR) 37.3%, latency (mean per test) 30.6 s. Not in our model list AssemblyAI Universal-3.5 Pro: Task success rate (TSR) 30.3%, latency (mean per test) 68 s. Not in our model list GPT-Live-Transcribe: Task success rate (TSR) 67.7%, latency (mean per test) 70.3 s gpt-4o-transcribe: Task success rate (TSR) 59.7%, latency (mean per test) 2.66 s Grok Voice Transcribe 2.0: Task success rate (TSR) 65.3%, latency (mean per test) 67.5 s. Not in our model list Reson8 Resonant-1: Task success rate (TSR) 59%, latency (mean per test) 1.45 s. Not in our model list GPT-Live-TranscribeGrok Voice Transcribe 2.0gpt-4o-transcribeReson8 Resonant-1 40% 60% 80% 110100 Latency (mean per test) (s), log scale, lower is better Task success rate (TSR) Whisper large-v3: Task success rate (TSR) 54.3%, latency (mean per test) 2.2 s Whisper large-v3-turbo: Task success rate (TSR) 57%, latency (mean per test) 2.95 s gpt-4o-mini-transcribe: Task success rate (TSR) 51.7%, latency (mean per test) 2.22 s Voxtral Mini Transcribe 2: Task success rate (TSR) 53%, latency (mean per test) 1.89 s Voxtral Mini Transcribe Realtime (2602): Task success rate (TSR) 46%, latency (mean per test) 15.4 s Cohere Transcribe: Task success rate (TSR) 44.7%, latency (mean per test) 1.95 s Muse Voice Transcribe 1.0: Task success rate (TSR) 57%, latency (mean per test) 67.2 s Cartesia Ink 2: Task success rate (TSR) 62%, latency (mean per test) 69.2 s. Not in our model list Deepgram Nova-3 (Streaming): Task success rate (TSR) 59.3%, latency (mean per test) 67.5 s. Not in our model list Inworld STT 1: Task success rate (TSR) 57.3%, latency (mean per test) 70.5 s. Not in our model list Deepgram Flux General English: Task success rate (TSR) 55.7%, latency (mean per test) 68.8 s. Not in our model list ElevenLabs Scribe v2 Realtime: Task success rate (TSR) 55.3%, latency (mean per test) 77.7 s. Not in our model list Google Cloud Chirp 3: Task success rate (TSR) 55.3%, latency (mean per test) 68.1 s. Not in our model list Grok Voice Transcribe 1.0: Task success rate (TSR) 52%, latency (mean per test) 67.8 s. Not in our model list Azure Speech Universal Language Model: Task success rate (TSR) 37.3%, latency (mean per test) 30.6 s. Not in our model list AssemblyAI Universal-3.5 Pro: Task success rate (TSR) 30.3%, latency (mean per test) 68 s. Not in our model list GPT-Live-Transcribe: Task success rate (TSR) 67.7%, latency (mean per test) 70.3 s gpt-4o-transcribe: Task success rate (TSR) 59.7%, latency (mean per test) 2.66 s Grok Voice Transcribe 2.0: Task success rate (TSR) 65.3%, latency (mean per test) 67.5 s. Not in our model list Reson8 Resonant-1: Task success rate (TSR) 59%, latency (mean per test) 1.45 s. Not in our model list GPT-Live-TranscribeGrok Voice Transcribe 2.0gpt-4o-transcribeReson8 Resonant-1
ModelSettingTask success rate (TSR)Latency (mean per test)
GPT-Live-Transcribe 67.7% 70.34 s
Grok Voice Transcribe 2.0 65.3% 67.48 s
Cartesia Ink 2 62% 69.2 s
gpt-4o-transcribe 59.7% 2.66 s
Deepgram Nova-3 (Streaming) 59.3% 67.54 s
Reson8 Resonant-1 59% 1.45 s
Inworld STT 1 57.3% 70.47 s
Whisper large-v3-turbo 57% 2.95 s
Muse Voice Transcribe 1.0 57% 67.17 s
Deepgram Flux General English 55.7% 68.8 s
Google Cloud Chirp 3 55.3% 68.1 s
ElevenLabs Scribe v2 Realtime 55.3% 77.7 s
Whisper large-v3 54.3% 2.2 s
Voxtral Mini Transcribe 2 53% 1.89 s
Grok Voice Transcribe 1.0 52% 67.79 s
gpt-4o-mini-transcribe 51.7% 2.22 s
Voxtral Mini Transcribe Realtime (2602) 46% 15.38 s
Cohere Transcribe 44.7% 1.95 s
Azure Speech Universal Language Model 37.3% 30.62 s
AssemblyAI Universal-3.5 Pro 30.3% 67.95 s

The best score, 67.7%, costs $0.02 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Cost frontier One model at each effort level
0% 20% 40% 60% 80% $0.0001 $0.001 $0.01 $0.10 Cost per task, log scale GPT-Live-Transcribe gpt-4o-transcribe Whisper large-v3-turbo 0% 20% 40% 60% 80% $0.0001 $0.001 $0.01 $0.10 Cost per task, log scale GPT-Live-Transcribe gpt-4o-transcribe

Source: Vals AI’s leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-Live-Transcribe 67.7% $0.02
gpt-4o-transcribe 59.7% $0.003
Whisper large-v3-turbo 57% $0.0007
Muse Voice Transcribe 1.0 57% $0.003
Whisper large-v3 54.3% $0.002
Voxtral Mini Transcribe 2 53% $0.003
gpt-4o-mini-transcribe 51.7% $0.002
Voxtral Mini Transcribe Realtime (2602) 46% $0.007
Cohere Transcribe 44.7%

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Vals AI leaderboard Not tested since Sep 2026
GPT-Live-Transcribe 67.7%

Source: Vals AI’s leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On Vals AI’s leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) GPT-Live-Transcribe gpt-4o-transcribe Whisper large-v3 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) GPT-Live-Transcribe gpt-4o-transcribe

Source: Vals AI’s leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Audio and video

All 232 in Audio and video
  • Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy. Qwen 3.8 Max 88.7%
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better. Phi-4-multimodal 4.00%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%