Audio and speech
Models listen or speak. We count 533 benchmarks here, 34 of them arenas. A benchmark belongs here when input includes audio. Or output includes audio.
How fast it grows
128 were released since July, more than the 80 in the first half of 2026.
How much is still open
No model has passed 85% on 102 of the 116 benchmarks here that score in percent, 88% of them.
Every benchmark in this view
Newest first. Facets come from each benchmark’s own documents, and our methodology says how we tag them.
| Benchmark | Level | Released | Best score | Status |
|---|---|---|---|---|
| AssemblyAI Speech Benchmarks Speech recognition systems transcribe clean, accented, multilingual and code-switched audio, identify entities and diarize speakers. Error rates compare production systems across public speech sets. | Expert | Nov 2026 | | Not in cards |
| AudioVideoBench Agents produce and edit audio and video in containers. The tasks include compositing, mixing, repairing media, reframing shots, and transcribing recordings. | University | Nov 2026 | Not in cards | |
| AVCC Next-Speaker Prediction Models predict who will speak after a pause in a conversation. The authors compare two models with human listeners on the same 1,730 events. | Everyday | Nov 2026 | Not in cards | |
| DECIDE Models select decisions from short voice-call clips and conversation history across 4,867 questions, 1,081 clips, and 32 tasks. Human reference labels determine accuracy. The default board ranks by DECIDE Balance Score, which combines accuracy, cost, and latency on a zero-to-100 scale. | Everyday | Nov 2026 | | Not in cards |
| Joi-Bench-Task Voice agents use tools while a simulated caller changes a request or interrupts a reply. The evaluator tracks completion, responsiveness, and omitted information. | Everyday | Nov 2026 | Not in cards | |
| Musical Expectation Benchmark Models predict how surprising a musical passage will sound. The benchmark compares predictions from scores or recordings with perceptual measurements. | University | Nov 2026 | Not in cards | |
| Omnilingua-AudioMC Models follow requirements across spoken conversations in ten languages. The authors compare each model's spoken and text responses against the same 1,404 rubric items. | Everyday | Nov 2026 | | Not in cards |
| Open TTS Leaderboard 1.4.0 Speech models read multilingual text aloud. The board compares transcriptions with the reference text and measures voice similarity. | Everyday | Nov 2026 | | Not in cards |
| Open TTS Leaderboard 1.5.0 Speech models read multilingual text aloud. The board compares transcriptions with the reference text and measures voice similarity. | Everyday | Nov 2026 | | Not in cards |
| Open TTS Leaderboard 1.6.0 Speech models read multilingual text aloud. The board compares transcriptions with the reference text and measures voice similarity. | Everyday | Nov 2026 | | Not in cards |
| PianoBench Models answer questions about piano performances from recordings or symbolic transcriptions. The authors compare accuracy with and without access to the music. | University | Nov 2026 | 93.2% | Not in cards |
| Social Signal Bench Models identify overlapping social signals in conversations from audio or video. The score weights each label by annotator agreement and gives credit for related predictions. | Everyday | Nov 2026 | | Not in cards |
| VIPBench Models predict whether listeners hear the same voice in two recordings. The authors compare eleven speech models against judgments from 1,290 listeners. | Everyday | Nov 2026 | Not in cards | |
| WorkIndex Agents complete everyday WorkBuddy tasks. Tencent combines eight tests covering office files, audio and video work, and visual questions. | University | Nov 2026 | Not in cards | |
| AD4Edu-Bench Models write audio descriptions for university lectures. A rubric checks accessibility rules, and the authors compare the rankings with blind and low-vision listeners' preferences. | University | Oct 2026 | | Not in cards |
| Addis Scribe Benchmark Speech models transcribe recordings in seven African languages. Addis AI compares transcripts with human references and measures how quickly some systems return text. | Everyday | Oct 2026 | | Not in cards |
| AfriBED Models transcribe Bemba and Nyanja speech. The benchmark compares recognition errors across gender and speaker groups. | Everyday | Oct 2026 | Not in cards | |
| AMemOps Models track instruments and their changing attributes in long audio streams. Questions test whether they recall the latest value and recognize absent sources. | Everyday | Oct 2026 | Not in cards | |
| ANCHOR Models explain answers to audiovisual questions and locate the supporting evidence. The benchmark includes time intervals, image regions, and relevant sounds. | Everyday | Oct 2026 | | Not in cards |
| AVPS-360 Models locate and track the person making a sound in panoramic video. The annotations connect each sound source to a person across frames. | Everyday | Oct 2026 | Not in cards | |
| Co-Design Bench Agents revise existing applications from multimodal user requests. A judge checks browser and repository evidence against 2,331 expert criteria across 273 tasks. | Expert | Oct 2026 | 69.9% | Not in cards |
| Cozzo AI Receptionist Benchmark (Round 4) Voice agents answer dental office calls about booking, rescheduling, insurance, and emergencies. Cozzo checks the spoken answers and times how quickly agents stop after an interruption. | Everyday | Oct 2026 | Not in cards | |
| Cozzo AI Receptionist Benchmark (Round 5) Voice agents answer calls for a roofer, a gym, and a career coach. Cozzo checks quotes, bookings, urgent requests, after-hours answers, and interruptions. | Everyday | Oct 2026 | Not in cards | |
| Cozzo AI Receptionist Benchmark (Round 6) Voice agents quote a dental price, confirm appointments, and handle urgent calls. Cozzo checks spoken details and scores responses to interruptions. | Everyday | Oct 2026 | Not in cards | |
| Cresta-Bench Contact-center agents take simulated calls for lending, health and airline lines. Tool calls get deterministic checks and LLM judges grade the rest. | Everyday | Oct 2026 | 82.6% | Not in cards |
| DRAMA-Eval Models generate dialogue with several speakers across multiple turns. The authors check word accuracy and use a model judge to evaluate delivery in context. | Everyday | Oct 2026 | | Not in cards |
| DuplexLedger Speech agents track facts that speakers introduce, revise, or leave unchanged during conversation. The evaluator asks which facts still apply. | Everyday | Oct 2026 | | Not in cards |
| EvaOmni Models answer questions about videos and identify the audio and visual evidence behind their answers. The score accounts for both correctness and evidence timing. | Everyday | Oct 2026 | | Not in cards |
| G2 Voice AI Agent Platform Evaluations Voice platforms run the same agent on simulated business calls. Bluejay checks required tool arguments and the database after hangup, then measures transcription and latency. | University | Oct 2026 | Not in cards | |
| GAIK LA-murre dialect test Models transcribe 583 Finnish dialect interview segments from ten localities. The reference preserves dialect forms, so standard Finnish spellings can count as errors. | Everyday | Oct 2026 | | Not in cards |
| GAIK SKN dialect test Models transcribe 600 Finnish dialect interview segments from ten municipalities. The reference uses standard Finnish word forms. | Everyday | Oct 2026 | | Not in cards |
| GSR-Bench Models turn speech into transcripts, meeting notes, or other requested text. Judges check whether the output preserves the meaning and stays grounded in the recording. | Everyday | Oct 2026 | | Not in cards |
| HMU-Bench Models answer Chinese and English questions about musical notation and recordings. The authors score responses against reference answers. | University | Oct 2026 | | Not in cards |
| LP-TTS-Bench Models synthesize speech from extended recordings of a speaker. The benchmark measures how well they reproduce timing, emphasis, and intonation on 2,728 held-out utterances. | Everyday | Oct 2026 | Not in cards | |
| Mind the Gap Models describe visible events during natural pauses in a video's soundtrack. The evaluator renders the narration and checks its timing and coverage. | University | Oct 2026 | | Not in cards |
| MUSEBENCH Models detect computer-generated music and identify artifacts. The authors compare eleven detectors on original recordings and on transformed tracks. | University | Oct 2026 | Not in cards | |
| Music Understanding Benchmark Models estimate tempo, key, meter, and song structure from recordings with identifying metadata removed. The evaluator compares predictions with existing annotations. | University | Oct 2026 | Not in cards | |
| Nufo Models transcribe Ewe speech from a fixed test split. The authors compare transcriptions with reference text using word error rate. | Everyday | Oct 2026 | Not in cards | |
| Omni-InteractScope Omnimodal models watch continuous video and audio streams and must respond at the right moment with answers, alerts, warnings, tool calls or step guidance. Scoring joins content and timing. | Everyday | Oct 2026 | | Not in cards |
| Omni-InteractScope Explicit Request Models answer explicit requests during audio and video streams. The score is the published average across tasks T1 through T7. | Everyday | Oct 2026 | | Not in cards |
| Omni-InteractScope Proactive Models provide timely proactive assistance during audio and video streams. The score is the published T8 assistance recall. | Everyday | Oct 2026 | | Not in cards |
| Open TTS Leaderboard 1.1.0 Speech models read multilingual text aloud. The board compares transcriptions with the reference text and measures voice similarity. | Everyday | Oct 2026 | | Not in cards |
| Open TTS Leaderboard 1.2.0 Speech models read multilingual text aloud. The board compares transcriptions with the reference text and measures voice similarity. | Everyday | Oct 2026 | | Not in cards |
| Open TTS Leaderboard 1.3.0 Speech models read multilingual text aloud. The board compares transcriptions with the reference text and measures voice similarity. | Everyday | Oct 2026 | | Not in cards |
| People Making Conversation Speech models continue real conversations at recorded pauses and turn endings. The benchmark compares their timing, response length, and recall with the original speakers. | Everyday | Oct 2026 | | Not in cards |
| RealCommBench Models distinguish human and synthetic speech after transmission over real phone calls. Paired recordings measure how the call changes detection error. | Everyday | Oct 2026 | Not in cards | |
| RMS-AQA Models identify sound events in domestic spaces, then answer questions about their positions and timing. The benchmark scores both stages. | Everyday | Oct 2026 | | Not in cards |
| Roark Live Bench v1.3 Voice agents answer calls over the public phone network and use simulated business tools. Roark scores task completion, tool use, speech quality, and response time. | University | Oct 2026 | Not in cards | |
| RVG-Bench Models locate the video interval that answers a question about sounds and images. The evaluator measures overlap with the annotated interval. | Everyday | Oct 2026 | | Not in cards |
| SAC-Bench Models identify sounds and their order when a spoken claim contradicts the recording. Matched clips test whether answers follow the acoustic evidence. | Everyday | Oct 2026 | | Not in cards |
| SAVU-Bench Models answer spatial questions about real scenes using sound and video. The authors report accuracy across seven tasks. | Everyday | Oct 2026 | | Not in cards |
| SEA-HELM Audio Models answer the same reasoning and instruction-following tasks from speech and text in five languages. Identical checks measure the accuracy lost with speech input. | School | Oct 2026 | Not in cards | |
| StyleInstructCaps Models describe a speaker's voice according to different instructions. The evaluator grades instruction following, accuracy about the recording, and consistency across the speaker's clips. | Everyday | Oct 2026 | Not in cards | |
| VibeVideoBench Agents write code to render a finished video with sound, with no image or video generation model, in their native harness within one hour. Program checks, task rubrics and a judge panel score the result. | Expert | Oct 2026 | 4.11 | Not in cards |
| VocalGrad Models decide which way an acoustic property changes within a recording. The authors compare model accuracy with human listeners on the same binary questions. | Everyday | Oct 2026 | 51.8% | Not in cards |
| VoxAnon Models anonymize speech or try to identify the original speaker. Balanced trials compare reidentification across age, gender, pitch, and accent groups. | Everyday | Oct 2026 | Not in cards | |
| τ-bench 2.0 Agents serve simulated customers in airline, retail, telecom and banking domains in six languages, by text or voice. Release 2.0 adds frozen task manifests and an intake domain for entity capture. | University | Oct 2026 | 79.9% | Not in cards |
| Almieyar Models transcribe newly recorded speech in 17 Arabic dialects. The authors report word and character error rates across six dialect families. | Everyday | Sep 2026 | 34.9% | Not in cards |
| Amharic ASR Benchmark Speech models transcribe 1,548 recordings of people reading Amharic prompts. The organizers report character and word error rates. | Everyday | Sep 2026 | 151.1% | Not in cards |
| AudioJevBench Models listen to a clip and assign probabilities to answers about speech or sound. The maker checks accuracy and calibration across 988 scored items. | Everyday | Sep 2026 | | Not in cards |
| AV-SafetyBench Models generate videos with sound from written prompts. Judges review each clip with sound, in silence, and as audio alone to identify unsafe outputs. | Everyday | Sep 2026 | 49.4% | Not in cards |
| CoSE-E Models transcribe business conversations that switch between languages. The main metric checks whether transcription errors change the answers to questions about the conversation. | Everyday | Sep 2026 | | Not in cards |
| CutCraft Benchmark Models generate multi-shot videos under editing instructions. The evaluator checks transitions, shot structure, and the timing of audio relative to cuts. | University | Sep 2026 | | Not in cards |
| Duplex-MPE Full-duplex speech models join conversations of three or four people and must decide when to answer, stay silent or stop speaking. The set has 2,000 scenarios. | Everyday | Sep 2026 | | Not in cards |
| Huayu Awards AI Cockpit Evaluation 2026 In-car assistants interpret requests and operate vehicle features. Tests include planning three-stop trips and adjusting several cabin controls from one request. | Everyday | Sep 2026 | Not in cards | |
| Hume AI Overall Quality Index Listeners rate generated speech on reliability and expressiveness. The benchmark reports an overall quality index. | Everyday | Sep 2026 | | In use |
| Hume AI Voice Design Benchmark Listeners rate generated voices for English voice design, multilingual control, accents and voice qualities. | Everyday | Sep 2026 | | In use |
| JusticeAxis Models assess evidence and produce judgments for 256 criminal cases from 18 countries. The evaluator checks charges and outcomes against lawyer-written references. | Expert | Sep 2026 | | Not in cards |
| Last Translation Benchmark Models translate inputs that other systems struggled with. A model checks each translation against human-written rules. A translation passes only if it meets every rule. | Research frontier | Sep 2026 | 45.3% | Not in cards |
| Medical Speech-to-Text Benchmark Speech models transcribe 1,513 English medical recordings. A common scorer measures word errors, medical-term errors, drug-name errors, and dosage matches. Separate tests measure batch response time and final streaming latency. | Expert | Sep 2026 | 1.32% | Not in cards |
| MISHAP-Bench Models answer open-ended questions about audio clips. A model judge uses human annotations to identify claims the recording does not support. | Everyday | Sep 2026 | 32.8% | Not in cards |
| MM-FinEval Models forecast financial outcomes from earnings call transcripts, slides, and audio. The authors compare 19 models using directional accuracy and absolute error. | University | Sep 2026 | 88% | Not in cards |
| OmniEchoBench Models answer 2,972 spatial questions from recorded audio and video or navigate toward sound sources in 900 episodes. Rules score extracted spatial answers and multiple-choice letters. Navigation metrics measure success within one meter and path efficiency. | Everyday | Sep 2026 | | Not in cards |
| Omnilingua-Bench Models handle long-audio transcription, multilingual long-audio question answering, multi-speaker attribution and complex spoken instructions. WER, diarization, accuracy and rubric pass rates score the tracks. | University | Sep 2026 | Not in cards | |
| Open TTS Leaderboard 1.0.0 Speech models read multilingual text aloud. The board compares transcriptions with the reference text and measures voice similarity. | Everyday | Sep 2026 | | Not in cards |
| ORAV Bench Models combine reference images, sounds, and videos into a requested audiovisual scene. A judge compares pairs of outputs against each reference's assigned role. | Everyday | Sep 2026 | | Not in cards |
| Qwen-Audio-Agent In-House Cockpit Benchmark Agents handle requests with cockpit tools. The evaluation measures whether they complete each task when they call tools directly, delegate, or choose between both routes. | Everyday | Sep 2026 | Not in cards | |
| ScreenASR-Bench Models transcribe speech using on-screen text, visible objects, or clues from the scene. The authors report word error rate and recall. | Everyday | Sep 2026 | | Not in cards |
| Simreal ML Benchmark ML research agents study a dataset, build a model and submit predictions on 60 tasks in tabular, forecasting, vision, language, audio and multimodal learning. Competition platforms score them. | Expert | Sep 2026 | Not in cards | |
| SMEBench Models edit recorded music according to instructions. The benchmark uses paired source and target recordings to check changes to pitch, instrumentation, articulation, and dynamics. | University | Sep 2026 | Not in cards | |
| Sonetra Norwegian SNOMED CT Term Recognition Models transcribe Norwegian sentences read by a synthetic voice. A term counts as correct when every word in the medical term matches. | University | Sep 2026 | 66.4% | Not in cards |
| SteerBench Speech models respond to spoken instructions about tone, persona, style, and speed. A judge checks their speech and words against binary rubrics. | Everyday | Sep 2026 | Not in cards | |
| TimelineBench Agents turn raw footage and a client brief into a finished video. The edit must pass delivery checks, meet the brief, and clear a quality threshold. | University | Sep 2026 | 32.1% | Not in cards |
| VAmoS Pro Bench Voice agents handle several utility billing requests in one call. A model checks the spoken amounts and recorded actions against the requirements for each request. | University | Sep 2026 | | Not in cards |
| ViNV-TTS Models read Vietnamese dialogue aloud and place sounds such as laughter or breathing where the text marks them. | Everyday | Sep 2026 | Not in cards | |
| Voice Arena Goal Dataset Judges compare two calls and choose which caller achieved the airline task more successfully. Human preferences provide the reference. | Everyday | Sep 2026 | | Not in cards |
| Voice Controllability Leaderboard Text to speech systems create requested voices and follow delivery, emotion and role instructions. Blind human ratings and objective prosody measures score controllability. | Expert | Sep 2026 | | Not in cards |
| Voice Replication Leaderboard Text to speech systems reproduce reference speakers from short recordings. Human ratings score speaker identity, quality and naturalness, with an objective speaker similarity check. | Expert | Sep 2026 | | Not in cards |
| VoxMem Audio language models answer questions that need memory of earlier spoken sessions. Questions cover what was said, who said it, how it was spoken and what was audible. | Everyday | Sep 2026 | | Not in cards |
| VoxParity Voice agents choose tool calls after listening to speech. Paired recordings keep the words fixed while changing an audible cue that affects the correct action. | Everyday | Sep 2026 | | Not in cards |
| Addis AI Amharic Text to Speech Benchmark Speech models read 100 Amharic prompts aloud. Native speakers rate naturalness, intelligibility and voice quality without seeing the system names. | Everyday | Aug 2026 | | Not in cards |
| ADVANCE audio and image retrieval (MTEB) Models match field recordings and aerial images from 5,075 locations. The correct recording and image come from the same place. | Everyday | Aug 2026 | Not in cards | |
| Artificial Analysis average word error rate Speech transcription systems are scored by average word error rate across the Artificial Analysis evaluation set. | Everyday | Aug 2026 | | In use |
| doteBench Models edit English and Chinese recordings using a transcript that marks where each change belongs. The evaluator checks edit success and preservation outside those regions. | University | Aug 2026 | Not in cards | |
| LongAudioSpan Models answer 3,240 questions about 360 English and Chinese recordings from 10 minutes to over 2 hours. Each question is scored as multiple choice and as open-ended. | Everyday | Aug 2026 | 80.2% | In use |
| MIVAS Bench Voice agents handle calls for a dermatology practice, a law firm, and a retailer. Each call must pass checks on tools, specialist handoffs, and database changes. | University | Aug 2026 | 56.5% | Not in cards |
| Pipecat Speech-to-Text Benchmark Speech-to-text services transcribe the same 1,000 realtime audio clips. Semantic word error rate measures accuracy, and time to final segment measures streaming latency. | Everyday | Aug 2026 | | Not in cards |
| Read-aloud Assessment Benchmark Speech models transcribe children reading aloud in Kiswahili. The evaluation measures transcription errors and disagreements with human assessors about which items the child misread. | Everyday | Aug 2026 | | Not in cards |
| Speech Agent Arena People hold live conversations with two hidden speech-to-speech agents on the same scenario, some with tool calls, then pick the one they preferred. Votes are fit to an Elo rating, and the arena also tracks task success. | Everyday | Aug 2026 | 1,070 | In use |
| TurnBench Turn-taking systems listen to two-person conversations and must detect when a speaker's turn ends and when the other speaker interrupts. The test set has 116 English conversations in six interaction styles, each labeled by three annotators. Systems rank by recall, and a system whose false-positive rate exceeds 0.15 ranks below every qualifying system. Latency after each event is also reported. | Everyday | Aug 2026 | | Not in cards |
| UA-Bench Models identify underwater sound sources and their subclasses from audio recordings. The authors repeat the test with silent audio to check reliance on sound. | University | Aug 2026 | | Not in cards |
| UniVerseBench Models answer questions about music across more than 38 cultural and linguistic groups. The authors report accuracy by language and skill. | University | Aug 2026 | | Not in cards |
| VoiceCodeBench Speech-to-text models transcribe 300 English workplace recordings. They are scored on whether every target entity, such as an address or email, survives exactly. | Everyday | Aug 2026 | 67.7% | Not in cards |
| WildArtifactBench Agents produce real deliverables in any format, from audio analysis and medical video reads to 3D meshes and kitchen layouts. Human and agent judges compare them with a baseline agent's work to give win rates and Elo. | Expert | Aug 2026 | Not in cards | |
| Cekura Agent Performance Scenario reliability across three repeated calls (Pass³). | Everyday | Jul 2026 | 81.7% | Not in cards |
| Cekura Bench Models complete separately scored speech recognition, speech generation, voice-agent, real-call, turn-taking and voice-quality benchmarks. Cekura publishes no combined score. | Everyday | Jul 2026 | Not in cards | |
| Cekura Production Voice Agents Published composite score for production voice agents. | Everyday | Jul 2026 | Not in cards | |
| Cekura Real Human Calls Real caller reliability across three repeated calls (Pass³). | Everyday | Jul 2026 | 42.9% | Not in cards |
| Cekura Speech to Text Overall word error rate across the published audio clips. | Everyday | Jul 2026 | 2.63% | Not in cards |
| Cekura Text to Speech Round-trip word error rate averaged across the two transcribers, as displayed by Cekura. | Everyday | Jul 2026 | 1.92% | Not in cards |
| Cekura Turn Bench Mean end-of-turn score across six line conditions, as published by Cekura. | Everyday | Jul 2026 | 84.7% | Not in cards |
| Context Aware Automatic Speech Recognition Speech transcription systems are scored for semantic accuracy with and without free-form context. | Everyday | Jul 2026 | 44.6% | In use |
| GigaAM Multilingual Internal In-the-Wild Speech Tests Models transcribe crowdsourced recordings in Russian, Kazakh, Kyrgyz and Uzbek. The evaluators report word error rate by language. | Everyday | Jul 2026 | Not in cards | |
| Indic DiarBench Models transcribe conversations in 22 Indian languages and identify who spoke each word. Human annotators supply speaker labels and timestamps, including for overlapping speech. | Everyday | Jul 2026 | 36.2% | Not in cards |
| MMGenre Models synthesize singing in different genres. The authors measure vocal quality, lyric errors, and how well the singing conveys the intended genre. | University | Jul 2026 | Not in cards | |
| Monsoon ASR Models transcribe Indian English and Hindi recordings. The Hindi evaluation accounts for alternative spellings in its word error rate. | Everyday | Jul 2026 | Not in cards | |
| MultiRef-Compass Models generate audio-video content from several visual references and text instructions across 350 samples. Fourteen metrics assess quality, reference fidelity, audio-visual consistency, and instruction following. The evaluator combines automatic measurements with multimodal model judgments. | Everyday | Jul 2026 | Not in cards | |
| MusICA-MetaBench Models answer the same music questions from audio, printed scores, and ABC notation. The authors publish fixed tests for two collections of choral music. | University | Jul 2026 | | Not in cards |
| PazaBench v2 Models transcribe speech in 61 African languages. The leaderboard measures character error rate, word error rate and transcription speed. | Everyday | Jul 2026 | 73.3% | Not in cards |
| Qwen-Music Cover Song Generation Models remake a reference melody using requested lyrics and style. The evaluation aligns the melodies and measures their average pitch difference in semitones. | University | Jul 2026 | Not in cards | |
| Qwen-Music Text-to-Music Generation Models turn English or Chinese lyrics and style instructions into complete songs. Music professionals compare anonymized recordings, and automatic judges score the same outputs. | University | Jul 2026 | Not in cards | |
| Real World VoiceEQ Voice systems recognize, produce and respond to acoustic information across speech recognition, speech to speech, speech understanding and text to speech tasks. | Expert | Jul 2026 | | Not in cards |
| Real-World Audio Recording Speech transcription systems are scored by word error rate on real-world recordings across multiple languages. | Everyday | Jul 2026 | 9.60% | In use |
| SoundingEarth audio and image retrieval (MTEB) Models match a field recording to an aerial image of the same location, or retrieve the recording from the image. Each direction has 1,000 queries and 2,048 candidates. | Everyday | Jul 2026 | Not in cards | |
| Speko Speech-to-Speech Benchmark (10-turn call) Voice agents answer a weather request with a tool, handle an interruption, and recall an earlier detail. A language model grades task success. | Everyday | Jul 2026 | | Not in cards |
| Speko Speech-to-Speech Benchmark (20-turn concierge call) Voice agents try to book a flight, handle a failed booking, and remember a changed date. Eight automated checks score their actions and replies. | Everyday | Jul 2026 | 1.00 | Not in cards |
| Uzbek Open ASR Leaderboard Models transcribe Uzbek recordings from four test sets. The main score averages the four word error rates with equal weight. | Everyday | Jul 2026 | Not in cards | |
| VAmoS Bench Voice agents answer simulated bank customers and use tools to manage credit cards. A model checks each call against its transcript and recorded actions. | University | Jul 2026 | | Not in cards |
| Ego-MC-Bench Models watch a cook follow spoken instructions and tell them when to correct a mistake. The evaluator checks intervention timing and recipe step completion. | Everyday | Jun 2026 | | Not in cards |
| Far-Field ASR Leaderboard Models transcribe speech under simulated room acoustics. The board compares word error rates. | Everyday | Jun 2026 | | Not in cards |
| HALAS Models transcribe earnings call segments with human annotations of hallucinations. NVIDIA measures word error and segments with inserted text. | Everyday | Jun 2026 | 23.8% | Not in cards |
| IndicContextEval Models transcribe speech in eight Indian languages using hints about the topic or names. One prompt deliberately supplies names from an unrelated field. | Everyday | Jun 2026 | | Not in cards |
| MMAE Audio models edit audio from instructions across a broad range of real-world editing tasks and sound types. | Everyday | Jun 2026 | Not in cards | |
| OmniCap-IF Models write captions for videos with audio while following detailed user constraints. Captions are scored on format and content compliance. | Everyday | Jun 2026 | | In use |
| Preference-ASR Models transcribe speech in a requested style. The evaluation checks preferences such as casing, number formatting, and removing disfluencies. | Everyday | Jun 2026 | | Not in cards |
| RealityTest Conversational models answer identity questions in text and speech across five languages. A judge classifies explicit, ambiguous and missing identity disclosure. | Everyday | Jun 2026 | Not in cards | |
| SLoMO Audio Description Competition 2026 Models write one sentence for each gap in a film’s dialogue. Judges score how well the descriptions convey visual information and match professional references. | University | Jun 2026 | Not in cards | |
| Speech-to-Speech Index Artificial Analysis's combined score for speech-to-speech models. | Research frontier | Jun 2026 | 86.9 | In use |
| Vaani Benchmark V1.0 Models transcribe speakers describing pictures. The scorer compares each prediction with three human transcripts and allows wording that matches any reference. | Everyday | Jun 2026 | Not in cards | |
| ACII DaiKon Challenge Models predict interpersonal influence, turn taking and rapport trajectories from multimodal dyadic conversations across five languages. | Expert | May 2026 | Not in cards | |
| AgenticVBench Agents complete 100 video post-production tasks in 4 task families built from real workflows by 20 industry experts. Programmatic verifiers and expert rubrics score the results. | Expert | May 2026 | 38.4% | In use |
| CueSpeak Models speak at the correct moment and give the expected semantic response during simultaneous speech. | Everyday | May 2026 | | Fading |
| EVA-Bench Voice agents hold simulated spoken conversations to complete enterprise tasks. Scores cover task success and voice-specific failure modes. | Everyday | May 2026 | 0.74 | In use |
| EvalVerse Video models generate cinematic clips from prompts and reference material derived from professional videos. Film experts compare outputs across visual design, acting, cinematography, aesthetics, emotion, shot sequences, and sound. A calibrated vision-language evaluator also estimates these judgments. | Expert | May 2026 | 0.73 | Not in cards |
| LongAV-Compass Models generate minute-long videos with sound from text, an image or a video. Scoring covers identity consistency, narrative coherence and audio-visual alignment over time. | Everyday | May 2026 | | Not in cards |
| MultiMedia-TerminalBench Agents inspect and edit media in persistent terminal workspaces. The 105 tasks require verifiable output files across five areas of professional multimedia work. | Expert | May 2026 | 16.2% | Not in cards |
| MusTBench Models answer five kinds of music questions that need the correct time region of the audio, such as instrument entries. Music experts validated the items. | Expert | May 2026 | | In use |
| OpenFARM Bioacoustics Models classify animal vocalizations across eight tasks, including pig call type and recording context. Each task uses labels from its source dataset. | Expert | May 2026 | Not in cards | |
| PitchBench Models listen to recordings and identify notes, intervals, chords, or melodies. The authors vary the instrument, timing, and background noise. | University | May 2026 | | Not in cards |
| PRISM-Bench (audio and video generation) Models generate speech, music, or environmental sounds with video. A judge compares outputs with reference clips across 35 criteria, including synchronization and prompt following. | Everyday | May 2026 | | Not in cards |
| ProactiveVideoQA Models answer spoken questions when a streamed video first reveals the answer. | Everyday | May 2026 | | Fading |
| TimeSpeak Models initiate speech at specified times while producing the requested content. | Everyday | May 2026 | | Fading |
| VideoFDB Multimodal agents are evaluated on two-person video conference recordings. They cover 11 conversational dynamics across verbal, nonverbal and mixed behavior, annotated by people in three passes. | Everyday | May 2026 | | Not in cards |
| AVGen-Bench Models generate videos with audio from prompts across eleven real-world categories. Specialist models and multimodal judges measure perceptual quality, synchronization, and adherence to detailed visual, speech, music, and physical requirements. | University | Apr 2026 | | Not in cards |
| Coval Speech-to-Text WER Speech models transcribe call-like audio. The score is mean word error percentage over the published pooled 24-hour window. | Everyday | Apr 2026 | | Not in cards |
| Coval Text-to-Speech TTFA Speech models synthesize customer-service prompts. The score is median time to first audible sample in milliseconds over the published 24-hour window. | Everyday | Apr 2026 | 267.3 | Not in cards |
| Coval Voice AI Benchmarks Speech-to-text, text-to-speech and speech-to-speech models from almost 40 providers are scored every 30 minutes on call-like audio with accents, noise and echo. | Everyday | Apr 2026 | Not in cards | |
| Deepgram Voice AI Benchmarks Speech models are compared on word error rate across production audio in 20 languages, while speech synthesis models are compared on read-aloud and hard-prompt sets. | Expert | Apr 2026 | Not in cards | |
| Full-Duplex-Bench v3 Voice agents hear real human speech with five kinds of disfluency and must make chained API calls across four task domains. Scored on Pass@1, response quality, latency and turn-taking. | Everyday | Apr 2026 | | Fading |
| LVSum Models summarize long videos with timestamped segments, relevance scores and descriptions using video frames, transcripts or both. | Expert | Apr 2026 | Not in cards | |
| MMEB-V3 Omni-modality embedding models are scored on 190 tasks across text, image, video, audio, visual documents and agent-centric retrieval. | Everyday | Apr 2026 | | Not in cards |
| MU-Bench Speech-to-text providers transcribe 4,270 caller utterances from real 8 kHz customer service phone calls in five locales, scored against human word-level transcripts. | Everyday | Apr 2026 | | Not in cards |
| NepTTS-Bench Speech models read Nepali sentences that test pronunciation contrasts. Native speakers rate the recordings, and the authors compare those ratings with an automated metric. | Everyday | Apr 2026 | | Not in cards |
| neuralCAD-Edit Agents edit existing CAD models from text or recorded instructions. CAD experts rate whether they would accept each edit. Software also compares the resulting geometry. | University | Apr 2026 | 0.25 | Not in cards |
| SpeechEditBench Models edit English and Chinese speech from written instructions. Joint success requires the requested change and preservation of the attributes checked for that task. | University | Apr 2026 | 96.7% | Not in cards |
| SpotSoundBench Models find the time span of short sound events hidden in long, unstructured YouTube recordings. It has 400 audio, query and timestamp triplets. | Everyday | Apr 2026 | | In use |
| Step-Caption Audio models describe speakers' vocal characteristics across language, age, rate, rhythm, pitch, timbre, emotion and related attributes. A judge compares the description with expert labels. | University | Apr 2026 | Not in cards | |
| Step-DU Audio models answer targeted questions about a speaker's vocal characteristics in dialogue. A judge scores answers against reference labels. | University | Apr 2026 | Not in cards | |
| Step-SPQA Audio models answer paralinguistic questions about age, emotion, events, gender, pitch, rhythm, scene, speed, voice sounds and style. | University | Apr 2026 | Not in cards | |
| TriScore Models identify composers and instruments from recordings, score images, and symbolic notation. They also retrieve excerpts from the same composition. | University | Apr 2026 | Not in cards | |
| VideoMME v2 (with subtitles) Models answer multiple choice questions about videos with subtitles or audio added. Related questions are scored as groups to penalize guessing. | Everyday | Apr 2026 | | In use |
| VoiceMOS Challenge 2026 Models predict listener ratings for enhanced speech, emotional speech, and synthesized accents. Organizers compare predictions with ratings for individual utterances. | Everyday | Apr 2026 | Not in cards | |
| BioVITA retrieval benchmark Models match animal images, audio recordings, and descriptions. The benchmark checks whether the retrieved items match the reference species, genus, or family. | University | Mar 2026 | Not in cards | |
| CodecMOS-Accent Speech systems reconstruct recordings or clone a voice across ten English accents. Listeners score naturalness, speaker similarity, and accent similarity separately. | Everyday | Mar 2026 | Not in cards | |
| Contextual Earnings-22 Models transcribe clips from earnings calls with a custom vocabulary. The benchmark measures keyword recognition and transcription errors. | Everyday | Mar 2026 | | Not in cards |
| EchoChain Voice agents reason through spoken exchanges while a speaker interrupts or changes instructions. The score measures whether the agent responds correctly. | Everyday | Mar 2026 | 44% | Not in cards |
| HaluEvalAudio 1000 Audio language models answer spoken questions against supplied knowledge and reference answers. | University | Mar 2026 | Not in cards | |
| HumMusQA Models answer 320 hand-written questions about music, curated and checked by people with musical training. | Expert | Mar 2026 | | In use |
| LVOmniBench Models answer questions about long videos with rich audio, often tens of minutes long. The questions need both the audio and the video track. | Everyday | Mar 2026 | | In use |
| MMOU Models answer 20,000 questions about 11,877 long web videos, using sound, images and text together. Questions cover 13 skill categories. | Everyday | Mar 2026 | | System cards only |
| OmniGUI Models use screenshots, sound, and video to predict the next action in 29 phone apps. The grader compares each action with a human demonstration. | Everyday | Mar 2026 | | Not in cards |
| Òwe-Voice Models transcribe spoken Yorùbá proverbs. The evaluation measures errors in words and tone marks. | Everyday | Mar 2026 | Not in cards | |
| RVMS-Bench Agents search open-domain videos and localize moments using global impression, key moment, temporal context and auditory memory cues. | Expert | Mar 2026 | Not in cards | |
| SongBench Models generate songs from lyrics and a style prompt. A trained music judge scores seven aspects, including the melody, arrangement, vocals, and mix. | University | Mar 2026 | Not in cards | |
| Teaching Monster Challenge Agents generate instructional videos for a specified topic and learner. A model screens submissions, viewers compare videos in pairs, and an expert panel sets the final ranking. | University | Mar 2026 | Not in cards | |
| τ-Voice Full-duplex voice agents handle real-time support calls in retail, airline and telecom with tools and a simulated caller. Scoring covers task completion and natural interaction. | Everyday | Mar 2026 | 81.7% | In use |
| Accessible Structured Subtitle Generation 2026 Systems produce timed Chinese subtitles for deaf and hard-of-hearing viewers. The structured track also scores speaker labels, key information, and readability. | University | Feb 2026 | Not in cards | |
| Audio Arena Speech-to-speech models hold one long spoken conversation per scenario and call tools when needed, such as booking a dental appointment or ordering groceries. Six scenarios add up to 221 spoken turns with corrections, false memory traps, prompt injection and state tracking. An LLM judge grades each turn against a golden answer. | Everyday | Feb 2026 | 85.7 | Not in cards |
| Audio-NTREX-4L Speech translation models translate long-form French, Spanish, Portuguese and German audio into English. | Expert | Feb 2026 | Not in cards | |
| CMI-RewardBench Models judge musical quality and how well recordings follow requests. Some requests combine text, lyrics, and reference audio. Scores measure agreement with human listeners. | Everyday | Feb 2026 | | Not in cards |
| DETOUR Agents ask a simulated user for clues and identify a partially remembered item. | Everyday | Feb 2026 | Not in cards | |
| IncompeBench Models retrieve music clips that match detailed natural language requests. | School | Feb 2026 | Not in cards | |
| MAEB Audio embedding models are scored on 30 tasks across speech, music, environmental sound and audio-text retrieval in over 100 languages. | Everyday | Feb 2026 | | System cards only |
| MSEB Audio representations from any multimodal system are scored on eight core tasks, including transcription, classification, retrieval, reasoning, segmentation, clustering, reranking and reconstruction. | Everyday | Feb 2026 | | Fading |
| NewsGuard false audio claims audit Voice assistants answer questions about 20 false news claims under three prompt styles. NewsGuard counts how often each assistant repeats a false claim aloud. | Everyday | Feb 2026 | Not in cards | |
| OmniGAIA Agents answer multi-hop questions that need reasoning and multi-turn tool use across video, audio and image inputs. Questions come from an omni-modal event graph built on real data. | University | Feb 2026 | | Fading |
| PazaBench Models transcribe speech in 39 African languages. The leaderboard measures character error rate, word error rate and transcription speed. | Everyday | Feb 2026 | Not in cards | |
| Synthetic Contextual ASR Benchmark (Indic) Models transcribe a synthetic caller’s response with or without the preceding conversation. A language model scores whether each transcript preserves the caller’s intent and named entities. | Everyday | Feb 2026 | Not in cards | |
| ADQA-Bench Models listen to recordings and answer 3,000 questions about speech, music, and environmental sounds. One question asks how a bird calls before it stops. | Everyday | Jan 2026 | | Not in cards |
| Agri STT Benchmarking Dataset Models transcribe farmers’ spoken questions in Hindi, Telugu and Odia. Researchers score errors against human transcripts and separately measure errors on agricultural terms. | Everyday | Jan 2026 | Not in cards | |
| AVMeme Exam Models answer questions about over one thousand iconic internet sounds and videos, from surface content to context, emotion, usage and world knowledge. | Everyday | Jan 2026 | | Fading |
| DISPLACE-M 2026 Models identify speakers, transcribe their words, identify medical topics and summarize conversations between community health workers and patients. Each task has its own score. | Everyday | Jan 2026 | Not in cards | |
| EntireSongs Models transcribe complete English and Chinese songs with background music. Qwen reports transcription errors separately for each language. | Everyday | Jan 2026 | | Not in cards |
| FutureOmni Models watch and hear 919 videos and answer 1,034 multiple-choice questions that ask what will happen next. Questions need audio and visual cues together. | Everyday | Jan 2026 | 66.7% | Fading |
| GAIK Ajokortti transcription test Models transcribe one three-minute Finnish dental webinar excerpt with two speakers. The project reports word error rate and separately counts errors that change the meaning. | Everyday | Jan 2026 | | Not in cards |
| Qwen3 forced alignment benchmarks Models place word or character boundaries in audio. Qwen measures timestamp shifts against automatic and human reference alignments. | School | Jan 2026 | Not in cards | |
| Qwen3-ASR internal transcription benchmarks Models transcribe accented conversations, noisy speech, tongue twisters, and multilingual news. Qwen reports error rates for each recording condition. | Everyday | Jan 2026 | Not in cards | |
| WAXAL Speech Recognition Test Speech models transcribe people describing pictures in African languages. Researchers compare the transcripts with human references using word error rate. | Everyday | Jan 2026 | Not in cards | |
| AMUSE Multimodal models reason over multi-speaker audio-visual interactions in zero-shot, guided and agentic modes across six task families. | Expert | Dec 2025 | Not in cards | |
| Audio MC Spoken multi-turn conversations test inference memory, instruction retention, self-coherence and voice editing. O4-mini judge applies per-conversation rubrics. | Everyday | Dec 2025 | | In use |
| AV-SpeakerBench Models answer 3,212 multiple-choice questions about who speaks, what is said and when in real-world videos. Questions need audio and video together. | Everyday | Dec 2025 | | Fading |
| JointAVBench Models answer questions about videos that cannot be answered from vision or audio alone. Questions cover speech and sound events over several scene spans. | Everyday | Dec 2025 | | Fading |
| KSAA 2026 Automatic Diacritization of Speech Dictation Models restore Arabic vowel marks from a recording and its transcript. Organizers rank systems by word error rate with case endings and unmarked characters included. | School | Dec 2025 | Not in cards | |
| LongShOTBench Models answer single-turn and multi-turn questions about long videos using visual, spoken, and ambient audio evidence. Weighted rubrics award partial credit. | Everyday | Dec 2025 | | Not in cards |
| PE-A-Frame internal audio event localization Models locate sound events within recordings. Meta compares model sizes using the area under the receiver operating characteristic curve. | Everyday | Dec 2025 | Not in cards | |
| PE-AV internal video-to-audio retrieval Models match videos to music tracks. Meta compares retrieval accuracy on internal clips that last from five to thirty seconds. | Everyday | Dec 2025 | Not in cards | |
| SAM Audio Judge evaluation Models rate the quality of separated speech, music, and environmental sounds. Meta measures correlation with listener ratings for completeness, precision, faithfulness, and overall quality. | Everyday | Dec 2025 | Not in cards | |
| SAM Audio-Bench Models isolate a requested sound from an audio mixture using text, visual, or temporal prompts. Listeners judge how well the output preserves the target. | Everyday | Dec 2025 | Not in cards | |
| Video Reality Test Vision-language models and humans judge whether ASMR videos with tight audio and visual coupling are real or AI-generated. It tests perceptual realism detection. | Everyday | Dec 2025 | | Not in cards |
| AssurAI Models respond to Korean prompts across 35 risk categories. Model judges rate the safety of text, image, audio, and video outputs from 1 to 5. | Everyday | Nov 2025 | Not in cards | |
| Audio Reasoning Challenge 2026 Models answer questions about speech, music, and environmental audio and explain their reasoning. Incorrect answers score zero. A model judge grades correct answers for reasoning quality, with the middle three of five evaluation runs averaged. | Everyday | Nov 2025 | Not in cards | |
| Clotho OVL-REL Test Set Models predict listener ratings for audio quality and caption relevance. Four generators produced environmental sounds from 100 captions drawn from Clotho. | Everyday | Nov 2025 | Not in cards | |
| CoSHE-Eval Models transcribe 1,985 conversational Hindi and English clips. The references use Devanagari for Hindi and Latin script for English. | Everyday | Nov 2025 | Not in cards | |
| DravidianLangTech 2026 Tamil Dialect Speech Recognition and Classification Models identify which of four regional Tamil dialects a speaker uses. A separate track tests how accurately models transcribe the speech. | Everyday | Nov 2025 | Not in cards | |
| Omnilingual ASR Corpus Models transcribe recordings from speakers of many languages. Meta compares character error rates on held-out speech, including languages with little existing transcription data. | School | Nov 2025 | Not in cards | |
| Swivuriso Models transcribe read and spontaneous speech in seven South African languages. The authors measure word error rates and keep speakers separate across splits. | Everyday | Nov 2025 | 15% | Not in cards |
| ADQA Models describe film scenes. An evaluator answers questions from the descriptions and dialogue, then checks whether the context supports each correct answer. | Everyday | Oct 2025 | Not in cards | |
| Full-Duplex-Bench v2 An automated examiner holds real-time multi-turn audio conversations with the model over WebRTC or WebSocket. A model judge scores task completion under conversational constraints. | Everyday | Oct 2025 | 4.05 | Not in cards |
| Omni-Cloze Models write detailed captions of audio, visual and audio-visual clips. A cloze test on the caption content scores the detail. | Everyday | Oct 2025 | | Fading |
| OmniVideoBench Models answer 1,000 questions about videos that need audio and visual evidence together. Each question has annotated reasoning steps. | University | Oct 2025 | | Fading |
| OutboundEval Agents conduct outbound calls with simulated users. The benchmark scores compliance with the professional workflow and the quality of the interaction. | Expert | Oct 2025 | Not in cards | |
| Real-time Audio-Visual Interaction Evaluation Trained conversationalists hold three minute live video calls with a model. Real users rate each session for naturalness and fluency on a 0 to 3 scale. | Everyday | Oct 2025 | | System cards only |
| UNO-Bench Models answer 1,250 human-written omni-modal questions and 2,480 uni-modal questions across 44 task types and 5 modality mixes. Multiple choice and multi-step open questions are scored, the open ones by a scoring model. | Everyday | Oct 2025 | | Dropped |
| VCB-Bench Audio language models answer spoken prompts across instruction following, empathy, safety, simulation and vocalization tasks. Judges score the responses. | Expert | Oct 2025 | Not in cards | |
| Ming-Freeform-Audio-Edit Models edit Chinese and English speech from written instructions. The evaluator checks edits to the spoken words and changes to voice characteristics. | University | Sep 2025 | Not in cards | |
| MultiChallenge Audio (Xiaomi) Models answer the final turn of MultiChallenge conversations converted to speech, with speech-to-text and speech-to-speech versions. Xiaomi built the conversion in house. | Everyday | Sep 2025 | 15.2% | System cards only |
| SLPHelm UltraSuitePlus Models analyze recordings of children’s speech for speech sound disorders. The suite also includes transcription and symptom classification tasks. | Expert | Sep 2025 | Not in cards | |
| SpeechMMLU Speech-language models answer MMLU-derived questions from text or synthesized speech in text-to-text, speech-to-text, text-to-speech and speech-to-speech settings. | University | Sep 2025 | Not in cards | |
| VOX-DUB Dubbing systems speak translated lines in the original speaker’s voice. Listeners compare pairs of recordings on five aspects of dubbing quality. | University | Sep 2025 | Not in cards | |
| WenetSpeech-Chuan-Eval Models transcribe Sichuanese speech. Qwen reports separate error rates on the easy and hard test sets. | Everyday | Sep 2025 | Not in cards | |
| WSYue-eval Models transcribe short and long Cantonese recordings. The suite also evaluates speech synthesis. | Everyday | Sep 2025 | Not in cards | |
| AHELM Audio-language models are scored across 10 aspects with standard prompts. The suite combines existing datasets with two new ones, PARADE for stereotypes and CoRe-Bench for conversational reasoning. | Everyday | Aug 2025 | | Not in cards |
| ComplexFuncBench Audio (OpenAI) Speech-to-speech models solve ComplexFuncBench function-calling tasks whose text prompts OpenAI converted into speech. Covers multi-step calls, constraints, implicit parameters and long inputs. | Everyday | Aug 2025 | 66.5% | Dropped |
| ICASSP 2026 Automatic Song Aesthetics Evaluation Challenge Models listen to generated songs and predict musicians’ ratings. One track scores overall musicality, and the other scores five aspects of each performance. | Everyday | Aug 2025 | Not in cards | |
| MMAU-Pro Models answer 5,305 expert-written questions about speech, sounds, music and their mixes, covering 49 listening skills. Some clips run up to 10 minutes. | University | Aug 2025 | | Fading |
| MultiChallenge Audio (OpenAI) Speech-to-speech models answer an audio-friendly subset of MultiChallenge multi-turn questions that OpenAI converted from text to speech. Scored on instruction following accuracy. | Everyday | Aug 2025 | 30.5% | Dropped |
| SpeechRole Speech role-playing agents talk in character across 98 roles. A model judge scores interaction ability, speech expressiveness and role fidelity from the generated speech. | Everyday | Aug 2025 | | Fading |
| ASFX-SED Models locate sound effects in audio recordings. Meta reports area under the receiver operating characteristic curve for event localization. | Everyday | Jul 2025 | Not in cards | |
| Audio Understanding Benchmark (Mistral) Models answer hard questions about 40 long-form audio recordings. Mistral built it in-house to test audio understanding beyond transcription. | Expert | Jul 2025 | | Dropped |
| ContextASR-Bench Models transcribe speech with optional domain labels and named entities. The benchmark measures transcription errors and recognition of named entities. | Everyday | Jul 2025 | 3.10% | Not in cards |
| CS-FLEURS Models transcribe speech that switches languages within sentences. Four test sets cover 113 language pairs. People read one set aloud, and synthesizers generate the others. | Everyday | Jul 2025 | Not in cards | |
| Full-Duplex-Bench v1.5 Full-duplex speech models are tested on overlap scenarios: listener backchannel, side conversation and ambient speech. Metrics cover how the model yields, continues or responds. | Everyday | Jul 2025 | | Fading |
| Music Arena Users write a music prompt, listen to tracks from two text-to-music systems, and vote. Preferences, listening data and feedback feed a leaderboard. | Everyday | Jul 2025 | | Not in cards |
| Music Arena Vocal Models generate music with vocals. Listeners compare anonymous tracks, and separate vocal preference votes determine the Arena score. | Everyday | Jul 2025 | | Not in cards |
| XACLE Challenge 2026 Models listen to generated audio and predict how closely it matches an English description. Organizers rank predictions by their correlation with listener scores. | Everyday | Jul 2025 | Not in cards | |
| Your Voice, Your Device, Your Language Challenge Models transcribe Kiswahili recordings for offline speech recognition. Zindi ranks submissions by word error rate before assessing the top ten for deployment. | Everyday | Jul 2025 | Not in cards | |
| CMI-Bench Models listen to music and answer requests such as naming an instrument, marking beats, or transcribing lyrics. Each task has its own scoring rule. | University | Jun 2025 | | Not in cards |
| COREBench (Conversational Reasoning) Models listen to a conversation with several speakers and answer a question that can require information from multiple turns. | Everyday | Jun 2025 | | Not in cards |
| InstructTTSEval Text-to-speech models follow natural-language style instructions in English and Chinese across acoustic parameter, descriptive style and role-play tasks. Gemini judges each clip. | Everyday | Jun 2025 | | Fading |
| MMSU Models answer questions about spoken language that need its content, its paralinguistic cues such as emotion and pitch, and its phonology. | University | Jun 2025 | | Fading |
| MultiFinBen Models complete 36 financial datasets across five languages and text, vision, and audio. The board reports a modality-balanced score. | University | Jun 2025 | | Not in cards |
| RELATE Models listen to an audio clip and predict how well it matches a caption. Human listeners also judge which sounds occur and in what order. | Everyday | Jun 2025 | Not in cards | |
| WildSpeech-Bench Models hold spoken conversations built from real chat data, with prosody, homophones and stuttering. A judge model scores the replies. | Everyday | Jun 2025 | | Fading |
| AGI-Eval Community Speech Synthesis Models synthesize speech. The board ranks models on how human the voices sound. | Everyday | May 2025 | 13.0 | Not in cards |
| AGI-Eval Community Voice Interaction Models converse by voice. The board scores naturalness, problem solving, learning assistance, leisure and empathy. | Everyday | May 2025 | 0.60 | Not in cards |
| AudioTrust Audio language models answer prompts that probe hallucination, robustness, speaker authentication, privacy, fairness, and safety. Automated evaluators, including language-model judges, compare responses with task-specific references and criteria. | Everyday | May 2025 | | Not in cards |
| CV3-Eval Text-to-speech models clone voices from in-the-wild reference audio across languages, dialects and emotions. Scored by ASR error rate, speaker similarity and DNSMOS audio quality. | Everyday | May 2025 | System cards only | |
| Daily-Omni Models answer 1,197 multiple choice questions about everyday videos that need audio and visual events aligned in time. Scored by accuracy. | Everyday | May 2025 | | Fading |
| EmotionTalk Models infer emotions and sentiment from Chinese conversations. | Everyday | May 2025 | 50.9 | Not in cards |
| General-Bench Models perform multimodal understanding and generation tasks. General-Level averages task scores and measures performance above specialist baselines at progressively broader levels. | University | May 2025 | | Not in cards |
| HiACC Models transcribe adults and children switching between Hindi and English. The recordings include story reading and spontaneous answers. | Everyday | May 2025 | | Not in cards |
| iNaturalist Sounds Models identify animal species from recordings submitted to iNaturalist. | Expert | May 2025 | Not in cards | |
| MiniMax TTS Multilingual Test Set Text-to-speech models clone two Common Voice speakers per language and read 100 test sentences in each of 24 languages. Scored by word error rate and speaker similarity. | Everyday | May 2025 | | In use |
| MMAR Models answer 1,000 questions about real-world audio of speech, sound, music and their mixes. Questions span four reasoning layers from signal to cultural knowledge. | University | May 2025 | | In use |
| MMAU v05.15.25 Models answer multiple-choice questions about speech, sound and music clips that need expert knowledge and reasoning. Scored by accuracy on test-mini or the hidden-answer test set. | University | May 2025 | | Not in cards |
| Polish Speech Emotion Recognition Challenge Models identify one of six emotions in Polish speech after training on recordings in other languages. | Everyday | May 2025 | Not in cards | |
| RESPIN-S1.0 Models transcribe dialect speakers reading sentences about farming and finance in nine Indian languages. The test holds out speakers and reports errors by dialect. | Everyday | May 2025 | Not in cards | |
| SongEval Models predict professional listeners' ratings for full-length songs in English and Mandarin. The test covers memorability, phrasing, structure, coherence, and musicality. | University | May 2025 | Not in cards | |
| FortisAVQA Models answer questions about sounds and visible events in music videos. Accuracy is reported separately for frequent and rare question distributions after the original questions are rephrased. | Everyday | Apr 2025 | 48.7% | Not in cards |
| Kimi-ASR Internal Testset Models transcribe speech from two internal Moonshot test subsets. Scored by word error rate, lower is better. | Everyday | Apr 2025 | System cards only | |
| Kimi-ASR Internal Testset Subset 1 Models transcribe speech from Moonshot internal subset 1. The score is the published word error rate. | Everyday | Apr 2025 | 1.42% | Not in cards |
| Kimi-ASR Internal Testset Subset 2 Models transcribe speech from Moonshot internal subset 2. The score is the published word error rate. | Everyday | Apr 2025 | 2.44% | Not in cards |
| Kimi-Audio speech conversation evaluation People rate spoken replies on speed, accent and emotion control, empathy and style control. Scored on a 1 to 5 scale and averaged. | Everyday | Apr 2025 | 3.90 | System cards only |
| MADASR 2.0 Models transcribe read speech and spontaneous conversations across eight Indian languages. The organizers rank each track by character error and also score language and dialect identification. | Everyday | Apr 2025 | Not in cards | |
| MUSDB-ALT Models transcribe 39 English songs from the MUSDB18 test set. Line timings and separate vocal recordings let researchers compare transcription with and without the accompaniment. | Everyday | Apr 2025 | Not in cards | |
| AudioMOS Challenge 2025 Models predict listener ratings for generated audio. Three tracks assess music quality and prompt alignment, general audio aesthetics, and speech quality across sampling rates. | Everyday | Mar 2025 | Not in cards | |
| AVUT Models answer questions about videos where the audio carries the key information. The benchmark removes questions that text alone can answer. | Everyday | Mar 2025 | | Fading |
| BLUR Models identify a remembered item from incomplete clues in several languages and media. | Everyday | Mar 2025 | 54% | Not in cards |
| Full-Duplex-Bench Full-duplex spoken dialogue models are scored on pause handling, backchanneling, smooth turn-taking and user interruption. Automatic metrics include takeover rate and latency. | Everyday | Mar 2025 | | In use |
| Golden3 speech summarization (Microsoft) Models summarize 108 recorded meetings of about 6 minutes each from 1071 instructions. GPT-4 rates overall quality from 1 to 7. | Everyday | Mar 2025 | | System cards only |
| Harmful content defect rate (Azure AI Evaluation SDK, Microsoft) GPT-4o simulates adversarial conversations with the model, and a judge flags violence, sexual, self-harm and hateful content. Scored as the share of harmful responses, lower is better. | Everyday | Mar 2025 | | System cards only |
| IVISTA Voice Interaction, revised 2023 protocol In-car assistants carry out spoken commands from different seats and answer follow-up requests. Testers check completed actions, including window adjustments and navigation commands. | Everyday | Mar 2025 | Not in cards | |
| MAVERIX Models answer 2,556 questions about 700 videos that need both the video and its audio. Questions come in multiple-choice and open-ended forms. | Everyday | Mar 2025 | 64.1% | Fading |
| OmniMMI Omni models answer 2,290 questions on 1,121 streaming videos, including proactive tasks where they must decide when to respond. | Everyday | Mar 2025 | | Fading |
| PARADE Audio Models answer questions about occupations and social status using audio clips. The evaluation tests whether voice cues bias their answers. | Everyday | Mar 2025 | | Not in cards |
| QIVD Models answer spoken questions about short phone videos in real time, as in a face to face talk. Each item has a short answer and a timestamp. | Everyday | Mar 2025 | | Fading |
| Qwen in-house voice-chat benchmark Models answer spoken versions of text benchmarks such as MMLU, CEval, IFEval, GSM8K, Math23K and Math401. Each part keeps the source benchmark's scoring. | School | Mar 2025 | System cards only | |
| SeniorTalk Models transcribe conversations with older Mandarin speakers and identify who speaks. | Everyday | Mar 2025 | 23.8% | Not in cards |
| EmoBench-M Multimodal models are tested on emotional intelligence across 13 scenarios in three dimensions: emotion recognition, conversational emotion understanding and socially complex emotion analysis. | Everyday | Feb 2025 | | Not in cards |
| KuralHub Models classify emotions in speech from 29 languages. | Everyday | Feb 2025 | Not in cards | |
| RivaBench Models answer questions about stand-up routines and academic talks using speech and visual cues. They also classify clips as real or synthetic. | University | Feb 2025 | Not in cards | |
| StepEval-Audio-360 Speech models answer multi-turn Chinese prompts about instruction following, voice understanding, reasoning, role play, creativity and speech control. | University | Feb 2025 | Not in cards | |
| URO-Bench Spoken dialogue models answer speech with speech across Understanding, Reasoning and Oral conversation. It has a basic track of 16 datasets and a pro track of 20. | Everyday | Feb 2025 | | Fading |
| WorldSense (omnimodal) Models answer multiple choice questions about real-world videos that need both the visual and the audio track. Scored by accuracy. | Everyday | Feb 2025 | | Fading |
| MER-UniBench Models describe emotions from audio, video, and text across nine datasets. Label normalization supports hit rate, weighted F1, and emotion-wheel F1. The reported mean averages the primary metric from each dataset. | Everyday | Jan 2025 | Not in cards | |
| MusicEval Models predict how experts rate generated music for musical quality and agreement with a text prompt. The public test contains 413 clips. | Everyday | Jan 2025 | Not in cards | |
| OpenAudioBench Models answer spoken questions in five sets: Reasoning QA, Llama Questions, Web Questions, TriviaQA and AlpacaEval. Knowledge sets use accuracy and the open sets use a model score. | Everyday | Jan 2025 | | Dropped |
| OpenAudioBench AlpacaEval Models respond to spoken AlpacaEval instructions. The score is the published judge quality score. | Everyday | Jan 2025 | | Not in cards |
| OpenAudioBench Llama Questions Models answer spoken Llama Questions. The score is the published accuracy. | Everyday | Jan 2025 | | Not in cards |
| OpenAudioBench Reasoning QA Models answer spoken reasoning questions. The score is the published accuracy. | Everyday | Jan 2025 | | Not in cards |
| OpenAudioBench TriviaQA Models answer spoken TriviaQA questions. The score is the published judge quality score. | Everyday | Jan 2025 | 6.19 | Not in cards |
| OpenAudioBench Web Questions Models answer spoken Web Questions. The score is the published judge quality score. | Everyday | Jan 2025 | 6.86 | Not in cards |
| Big Bench Audio Models answer 1,000 spoken questions from four Big Bench Hard tasks: formal fallacies, navigate, object counting and web of lies. Built to test reasoning over audio input. | School | Dec 2024 | 98.5% | In use |
| DCASE 2025 Audio Question Answering Models answer questions about marine mammal calls, the timing of sounds, and overlapping events. Each of the three sections counts equally toward the final score. | University | Dec 2024 | 52.1% | Not in cards |
| Labelbox Speech Generation v2 Models generate speech from text. Human raters compare the outputs for quality and naturalness. | Everyday | Dec 2024 | Not in cards | |
| Neptune Models answer multiple-choice questions about long videos that need reasoning over many segments. | Everyday | Dec 2024 | | System cards only |
| SuperCLUE-TTS Models read Chinese text aloud. The benchmark rates basic speech synthesis and delivery in situations such as public announcements. | Everyday | Dec 2024 | | Not in cards |
| UrduSER Models identify emotions in Urdu dialogue from Pakistani television. The corpus contains 3,500 excerpts from ten actors across seven emotion categories. | Everyday | Dec 2024 | Not in cards | |
| Audio PAIRS Models answer spoken questions adapted from PAIRS. The evaluation checks whether changing demographic cues changes their choices. | Everyday | Nov 2024 | Not in cards | |
| BEANS-Zero Models follow text prompts to classify, detect or caption animal sounds without task-specific training. | University | Nov 2024 | Not in cards | |
| Dynamic-SUPERB Phase-2 Spoken language models follow text instructions about audio across 180 tasks in speech, music and other sounds. Tasks include classification, regression and sequence generation, and researchers keep contributing new ones. Models run zero-shot, without task-specific training. | Everyday | Nov 2024 | Not in cards | |
| StreamingBench Models answer questions while a video streams in, as a human viewer would. It tests real-time visual understanding, omni-source understanding and context understanding. | Everyday | Nov 2024 | | Fading |
| LOKI Models distinguish synthetic content from real content across images, videos, audio, text, and 3D data. The benchmark checks real-or-synthetic labels and annotated abnormal details, with separate tasks for explaining anomalies. | Everyday | Oct 2024 | 60.8% | Not in cards |
| MMAU 10K audio clips with questions on speech, sound and music needing expert knowledge and reasoning. Multiple-choice accuracy. | University | Oct 2024 | | In use |
| Non-native Children's Automatic Speech Assessment Challenge Models rate children's Norwegian word pronunciations on a five-point scale. The challenge measures recall averaged across the five ratings. | University | Oct 2024 | Not in cards | |
| TaxaBench-8k Models identify species or match observations across photographs, sounds, and other field data. The test contains 8,813 observations paired across six types of data. | Expert | Oct 2024 | Not in cards | |
| VoiceBench Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. | Everyday | Oct 2024 | | In use |
| ChildMandarin Models transcribe Mandarin speech from children ages three to five or verify speakers. | Everyday | Sep 2024 | | Not in cards |
| Labelbox Speech Generation (original) Models generate speech from text. Human raters compare the outputs for quality and naturalness. | Everyday | Sep 2024 | Not in cards | |
| OmniBench Models answer multiple choice questions that need an image, an audio clip and text together. Scored by accuracy. | Everyday | Sep 2024 | | Dropped |
| Disparate performance on voice inputs Speech models answer capability and safety prompts spoken in system voices and 27 human voices. Compares accuracy across voices. | Everyday | Aug 2024 | System cards only | |
| GeoSound Models match environmental recordings to satellite images of their locations and retrieve recordings from images. The test holds out geographic areas and contains 9,931 examples. | Everyday | Aug 2024 | Not in cards | |
| LIMMITS 2025 Speech models synthesize text in four languages. Listeners rate naturalness and voice similarity. Separate tracks also measure streaming latency, codec bitrate, or both. | Everyday | Aug 2024 | Not in cards | |
| MuChoMusic Models answer multiple-choice questions about music recordings. It tests music knowledge and reasoning in audio-language models. | University | Aug 2024 | | Fading |
| Speaker identification Speech models get audio requests to identify a speaker by voice. Scored on refusing when they should and complying for famous quotes. | Everyday | Aug 2024 | System cards only | |
| Unauthorized voice generation Speech models produce audio that may deviate from the approved system voice. Scored on how many meaningful deviations the output classifier catches. | Everyday | Aug 2024 | 100% | System cards only |
| Ungrounded inference and sensitive trait attribution Speech models get requests to infer traits about a speaker from audio. Scored on refusing ungrounded inferences and hedging plausible ones. | Everyday | Aug 2024 | 84% | System cards only |
| Voice persuasion study on party preferences Participants hear model-voiced audio clips or talk with a voiced model. Scored on shifts in hypothetical party preference against human baselines. | Everyday | Aug 2024 | 5.74% | System cards only |
| AudioBench Audio language models follow instructions about speech, voice and environmental sounds. The paper covers 8 tasks and 26 datasets, including speech recognition, spoken question answering, voice traits and audio scenes. The repository has since grown to more than 50 datasets. | Everyday | Jun 2024 | Not in cards | |
| Seed-TTS-eval Models clone a voice from a short prompt and speak new text in English and Mandarin. Scored by word error rate from an ASR model and by speaker similarity. | Everyday | Jun 2024 | | Fading |
| SingMOS Models listen to Chinese and Japanese singing and predict how listeners rate its quality. The recordings include human voices and synthesized voices. | Everyday | Jun 2024 | Not in cards | |
| YorùLect Models transcribe Yorùbá speech and translate dialect text. The speech task covers three varieties. The text task also covers Ìjèbú. | Everyday | Jun 2024 | Not in cards | |
| IsiZulu Second Language Learner Speech Corpus Models detect sounds that learners mispronounce or insert when reading isiZulu. Teachers marked each sound against the intended pronunciation. | University | May 2024 | Not in cards | |
| Video-MME Models answer 2,700 multiple-choice questions about 900 videos from 11 seconds to 1 hour long, with and without subtitles. Scored by accuracy. | Everyday | May 2024 | | In use |
| Voice Jailbreak Attacks Against GPT-4o Models respond to spoken requests. Human reviewers measure how often a model complies with an unsafe request. | Everyday | May 2024 | Not in cards | |
| Ben-10 Regional ASR Models transcribe spontaneous speech in ten regional dialects of Bengali using spellings checked by linguists. The organizers report word and character error by dialect. | Everyday | Apr 2024 | Not in cards | |
| VoiceMOS Challenge 2024 Models predict listener ratings for high-quality synthetic speech, singing, and noisy or enhanced speech. The noisy speech track scores the voice and background noise separately. | Everyday | Apr 2024 | Not in cards | |
| BirdSet Models identify bird species from audio recordings across multiple evaluation datasets. | Expert | Mar 2024 | Not in cards | |
| DCASE 2024 Language-Queried Audio Source Separation Models isolate the sound named in a text query. Listeners score sound quality and agreement with the query on 100 real recordings with overlapping sounds. | University | Mar 2024 | Not in cards | |
| FeruzaSpeech Models transcribe one woman reading Uzbek. The reference text is available in Latin and Cyrillic scripts. | Everyday | Mar 2024 | Not in cards | |
| Hidden Voices Audio Difficulty Models rank piano works by Black women composers from recordings. The reference ordering comes from the Hidden Voices project’s seven difficulty grades. | University | Mar 2024 | Not in cards | |
| M³AV Speech Recognition Models transcribe academic lectures. The authors measure word errors across the transcript and separately for rare words. Slide text can provide context. | University | Mar 2024 | Not in cards | |
| Speech Robust Bench Models transcribe speech under different recording distortions. Word error rates show how each distortion affects recognition. | Everyday | Mar 2024 | Not in cards | |
| AIR-Bench (Audio Instruction Benchmark) Audio language models answer questions about speech, natural sounds and music. A foundation set has 19 tasks with about 19k single-choice questions, and a chat set has 2k open-ended questions scored by GPT-4. | Everyday | Feb 2024 | | System cards only |
| AIR-Bench Foundation Models answer foundational audio understanding questions. The score is the official average accuracy across nineteen tasks. | Everyday | Feb 2024 | | In use |
| PAM-Audio Models judge the quality of generated environmental sounds and how closely each recording matches its caption. The listening study used 100 AudioCaps captions. | Everyday | Feb 2024 | Not in cards | |
| TTS Arena Users listen to two anonymous text-to-speech models read the same text and vote for the more natural one. Votes become Elo ratings. | Everyday | Feb 2024 | | Not in cards |
| MuTox Models listen to short speech clips in 30 languages and classify whether they contain toxic speech. The harness compares their decisions with human labels. | Everyday | Jan 2024 | Not in cards | |
| NOTSOFAR-1 Models transcribe distant meetings and assign speech to speakers. NVIDIA also measures diarization error on the evaluation recordings. | Everyday | Jan 2024 | Not in cards | |
| DCASE 2024 Sound Scene Synthesis Models generate a four-second sound scene from a text description. Listeners rate how well the foreground and background match the prompt and assess audio quality. | University | Dec 2023 | Not in cards | |
| DISPLACE Challenge 2024 Models mark speakers and languages in multilingual conversations and transcribe individual speakers. The organizers score timing errors on room recordings and word errors on close microphone recordings. | Everyday | Dec 2023 | Not in cards | |
| SMILE Video Laugh Reasoning Models explain why people laugh during sitcoms and TED talks. The authors compare generated explanations with human references. | Everyday | Dec 2023 | Not in cards | |
| FAD Music Listening Test Models estimate acoustic and musical quality for ten-second music excerpts. The test compares predictions with human ratings from ten listeners. | Everyday | Nov 2023 | Not in cards | |
| GAIA Agents answer real-world questions that need web browsing, file handling, multimodal input, and tools, at three difficulty levels. Scored by exact answer match. | Everyday | Nov 2023 | 66.1% | Fading |
| Jam-ALT Models transcribe song lyrics, including punctuation and line breaks. The scorer reports word error rate and separate measures for formatting. | Everyday | Nov 2023 | Not in cards | |
| Alharthi LibriTTS Naturalness Listening Test Models estimate how natural synthesized speech sounds. Ten listeners rated each sample on a five-point scale for comparison. | Everyday | Oct 2023 | Not in cards | |
| ICNALE Global Rating Archives Models grade essays and spoken responses in English. The archive provides ratings from 80 people per sample across 140 essays and 140 speech samples. | University | Oct 2023 | Not in cards | |
| LIMMITS 2024 Speech models read text in a target voice across seven languages. Native listeners score naturalness and speaker similarity on the same prompts. | Everyday | Sep 2023 | Not in cards | |
| Open ASR Leaderboard Speech recognition models transcribe the test sets of several English and multilingual corpora. Scored by average word error rate, with speed reported next to it. | Everyday | Sep 2023 | | Fading |
| AudioSep Eval Benchmark Models isolate a sound from an audio mixture using a caption or a sound label. Scores compare the separated waveform with the original sound. | University | Aug 2023 | Not in cards | |
| VLSP 2023 Vietnamese Mispronunciation Detection Models compare spoken Vietnamese with the expected pronunciation and flag individual sounds that the speaker pronounced incorrectly. | Everyday | Aug 2023 | Not in cards | |
| JamendoLyrics MultiLang Models align written lyrics with singing across 80 songs in English, French, German, and Spanish. The researchers compare predicted and annotated word onsets. | Everyday | Jun 2023 | Not in cards | |
| MADASR 2023 Models transcribe dialectal Bengali and Bhojpuri read aloud by new speakers. Four tracks vary which speech recordings and written texts teams can use. | Everyday | May 2023 | Not in cards | |
| Massively Multilingual Speech labeled corpus Models transcribe recorded readings across languages. The evaluation compares predictions with reference transcripts using character error rate. | School | May 2023 | Not in cards | |
| OOD-Speech Models transcribe Bengali recordings from everyday media, including parliamentary sessions, online classes and poetry readings. The authors measure word and character error separately for each domain. | Everyday | May 2023 | Not in cards | |
| TMHINT-QI version II Models predict how clear Mandarin speech sounds and how many words listeners understand. The test adds street noise and two previously unseen enhancement systems. | Everyday | May 2023 | Not in cards | |
| Vistaar Models transcribe speech across 59 combinations of language and dataset in 12 Indian languages. Each test has its own word error rate. | Everyday | May 2023 | Not in cards | |
| Zoomed-in BVCC Models predict how listeners rate synthesized speech when they hear only the highest-rated systems. Listeners rated the same clips again in progressively smaller groups. | Everyday | May 2023 | Not in cards | |
| VALOR-32K Models match captions to video and audio clips. Meta reports separate retrieval results for text paired with audio and text paired with video. | Everyday | Apr 2023 | Not in cards | |
| VoiceMOS Challenge 2023 Models predict listener ratings for French speech synthesis, singing voice conversion, and noisy Mandarin speech. Teams predicted French and singing ratings before organizers received them. | Everyday | Apr 2023 | Not in cards | |
| Casual Conversations v2 Models classify attributes of speakers in recorded interviews. The HELM audio tasks compare age and gender predictions with participants’ own labels. | Everyday | Mar 2023 | Not in cards | |
| Te Hiku Media Māori speech recognition golden dataset Speech models transcribe te reo Māori recordings from native speakers and proficient second-language speakers. Te Hiku Media compares their word error rates. | Everyday | Jan 2023 | Not in cards | |
| DCASE 2023 Foley Sound Synthesis Models generate four-second sound effects for seven labels, including footsteps and rain. Listeners rate finalists on sound quality, fit to the label, and diversity. | University | Dec 2022 | Not in cards | |
| DISPLACE Challenge 2023 Models mark who speaks when and which language each person speaks in multilingual conversations. The scorer counts missed speech, false alarms and incorrect assignments, including overlaps. | Everyday | Dec 2022 | Not in cards | |
| CLIPSep Sound Separation Models isolate a requested sound from a mixture using a text or image query. The two tests mix music or everyday sounds with interfering audio. | University | Nov 2022 | Not in cards | |
| BanSpeech Models transcribe Bangladeshi Bangla broadcasts across 13 domains. Word error rate compares predictions with human transcripts. | Everyday | Oct 2022 | Not in cards | |
| Ravnursson Faroese Speech and Transcripts Models transcribe 3,002 Faroese recordings from 20 test speakers. The speakers read aloud for four and a half hours. | Everyday | Sep 2022 | Not in cards | |
| Kazakh Speech Corpus 2 Models transcribe Kazakh broadcasts, parliamentary speeches and podcasts alongside read speech. Some speakers switch between Kazakh and Russian. | Everyday | Aug 2022 | Not in cards | |
| FLEURS Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better. | Everyday | May 2022 | | In use |
| VocalSound Models classify human vocal sounds such as laughter, coughing, and sneezing. | Everyday | May 2022 | Not in cards | |
| Clotho-AQA Models answer crowdsourced yes or no and single word questions about short environmental audio clips. Scored by accuracy. | Everyday | Apr 2022 | | Dropped |
| BanglaSER Models identify the emotion in Bangla sentences recorded on phones and laptops. | Everyday | Mar 2022 | Not in cards | |
| Earnings-22 Models transcribe English earnings calls from speakers with different accents. NVIDIA reports long-form word error rate. | Everyday | Mar 2022 | Not in cards | |
| Hugging Face Speech Bench Speech recognition models transcribe recordings. The board collects error rates from model cards. | Everyday | Mar 2022 | Not in cards | |
| Lahjoita puhetta ASR test Models transcribe spontaneous Finnish speech on the official ten-hour test. Human transcripts supply the reference words. | Everyday | Mar 2022 | Not in cards | |
| TAU Urban Acoustic Scenes 2022 Mobile Models identify an urban acoustic scene from a one-second recording made on a mobile device. | Everyday | Mar 2022 | Not in cards | |
| AliMeeting Models identify speakers in Mandarin meetings. NVIDIA compares diarization error using headset mixes and distant microphones. | Everyday | Feb 2022 | Not in cards | |
| Opencpop Models transcribe lyrics from Mandarin singing. Qwen uses the test recordings to compare speech recognition systems. | Everyday | Jan 2022 | Not in cards | |
| ASCEND Models transcribe conversations that switch between Mandarin and English. The evaluation accounts for Chinese characters and English words. | Everyday | Dec 2021 | Not in cards | |
| Amharic Speech Emotion Dataset Models identify the emotion expressed in a short Amharic recording. The speakers use four regional dialects. | Everyday | Nov 2021 | Not in cards | |
| TMHINT-QI Models listen to clean, noisy, and enhanced Mandarin sentences. They predict listener quality ratings and how many words listeners understood. | Everyday | Nov 2021 | Not in cards | |
| WenetSpeech Models transcribe Mandarin speech from many domains. Scored by character error rate on the test-net and test-meeting sets. | Everyday | Oct 2021 | Fading | |
| WenetSpeech Test_Meeting Models transcribe spontaneous Mandarin speech from real meetings in the Test_Meeting set. The baseline score is mixture error rate, counting Mandarin characters and English words. | Everyday | Oct 2021 | Not in cards | |
| WenetSpeech Test_Net Models transcribe Mandarin internet speech from the Test_Net set. The baseline score is mixture error rate, counting Mandarin characters and English words. | Everyday | Oct 2021 | Not in cards | |
| Samrómur Children Models transcribe Icelandic speech from children reading prompts. The test covers girls and boys aged six through sixteen. | Everyday | Sep 2021 | Not in cards | |
| SD-QA Models answer spoken questions recorded in different English dialects. The benchmark compares answer accuracy across regional varieties of English. | Everyday | Sep 2021 | Not in cards | |
| Uzbek Speech Corpus Models transcribe held-out recordings of people reading Uzbek. Scorers compare the transcripts with the reference words. | Everyday | Jul 2021 | Not in cards | |
| GigaSpeech Models transcribe English speech from audiobooks, podcasts, and online videos. The test compares transcripts using word error rate. | Everyday | Jun 2021 | Not in cards | |
| IIT Madras Hindi-Tamil-English ASR Challenge Speech models transcribe Hindi, Tamil and Indian English recordings. The organizers report word error rates separately for each language and training condition. | Everyday | May 2021 | Not in cards | |
| PopCS Models transcribe lyrics from Chinese pop singing. Qwen compares recognition errors across speech models. | Everyday | May 2021 | Not in cards | |
| Samrómur 21.05 Models transcribe Icelandic sentences read aloud by volunteers. The training, development, and test sets use different speakers. | Everyday | May 2021 | Not in cards | |
| SUPERB Self-supervised speech models are tested as frozen encoders on a range of speech tasks, including speech recognition, keyword spotting, speaker identification, intent classification and emotion recognition. Each task trains only a light prediction head on top of the shared model. | Everyday | May 2021 | Not in cards | |
| AISHELL-4 Models transcribe Mandarin meetings and identify speakers from conference recordings. NVIDIA reports character error after matching speakers. | Everyday | Apr 2021 | Not in cards | |
| Earnings-21 Models transcribe long earnings calls. The benchmark tests recognition of financial terminology in real conversations. | Everyday | Apr 2021 | Not in cards | |
| MInDS-14 Models identify the banking service a caller wants from a spoken request. | Everyday | Apr 2021 | Not in cards | |
| MUCS 2021 Models transcribe speech in Indian languages and mixed-language recordings. NVIDIA reports results on Bengali-English and Hindi-English speech. | Everyday | Apr 2021 | Not in cards | |
| SPGISpeech Models transcribe English financial recordings. The references include formatting such as capitalization and punctuation. | Everyday | Apr 2021 | | Not in cards |
| SUST Bangla Emotional Speech Corpus Models identify which of seven emotions a speaker acts out in a Bangla sentence. | Everyday | Feb 2021 | Not in cards | |
| VoxPopuli Models transcribe European Parliament speeches. The corpus also supports speech translation across languages. | Everyday | Jan 2021 | Not in cards | |
| Multilingual LibriSpeech Models transcribe read audiobook speech in eight languages from LibriVox. Scored by word error rate. | Everyday | Dec 2020 | | Fading |
| SLURP Models identify the intent and named entities in spoken requests. NVIDIA evaluates both tasks within AIR-Bench. | Everyday | Nov 2020 | Not in cards | |
| VoxLingua107 Models identify the language in a speech recording. The corpus includes 107 languages. | Everyday | Nov 2020 | Not in cards | |
| Aligned Scores and Performances for Piano Transcription Models turn piano performances into written scores. The reference scores let researchers assess pitches, voices, and the positions of notes within bars. | Expert | Oct 2020 | Not in cards | |
| FSD50K Models identify the sound events present in a Freesound recording. | Everyday | Oct 2020 | Not in cards | |
| Nigerian Pidgin ASR Models transcribe Nigerian Pidgin sentences read by native speakers. The published test split contains 892 recordings. | Everyday | Oct 2020 | Not in cards | |
| Norwegian Parliamentary Speech Corpus Models transcribe speeches from the Norwegian parliament. The corpus includes manual transcripts in Bokmål and Nynorsk. | Everyday | Oct 2020 | Not in cards | |
| PiRhDy Melody Completion Models rank 50 possible continuations of a MIDI melody. One phrase follows the prompt in the original piece. The authors report mean average precision. | School | Oct 2020 | Not in cards | |
| DIHARD III Models identify who speaks when across eleven recording domains. The benchmark includes overlapping speech. | Everyday | Sep 2020 | Not in cards | |
| Kazakh Speech Corpus Models transcribe Kazakh speech from speakers held out of training. The scorer counts word errors against human transcripts. | Everyday | Sep 2020 | Not in cards | |
| CoVoST2 Models translate speech from 21 languages into English and from English into 15 languages. Scored by BLEU. | University | Jul 2020 | | Fading |
| IIT Madras Hindi ASR Challenge Speech models transcribe Hindi newspaper passages read aloud. The organizers score word errors and rank open and restricted training separately. | Everyday | Jun 2020 | Not in cards | |
| Taiwan Mandarin Speech with Video: Speech Enhancement Models use lip video to remove noise from Mandarin sentences. Researchers compare the enhanced signal with clean speech for quality and intelligibility. | University | May 2020 | Not in cards | |
| VGGSound Models identify sound events in video clips where the sound source is visible. | Everyday | Apr 2020 | Not in cards | |
| CHiME-6 Track 2: Unsegmented Speech Recognition Models identify who spoke when and transcribe dinner party recordings without utterance boundaries. The organizers rank systems by word error rate after matching speakers. | Everyday | Dec 2019 | Not in cards | |
| Common Voice Crowd-sourced multilingual read-speech corpus. Used here as an ASR test set scored by word error rate. | Everyday | Dec 2019 | | Fading |
| Clotho Models write a caption describing what can be heard in an audio recording. | Everyday | Oct 2019 | Not in cards | |
| Quartets Audio-to-Score Transcription Models transcribe synthesized string quartets by Haydn, Mozart, and Beethoven into four-part scores. The released test split contains 7,611 excerpts. | Expert | Oct 2019 | Not in cards | |
| WHAMR! Models recover individual voices from noisy speech mixtures with simulated room reverberation. The test contains 3,000 mixtures. | University | Oct 2019 | Not in cards | |
| WHAM! Models separate overlapping speakers or suppress background noise in speech mixtures. The authors compare output signals with the reference speech using scale-invariant signal-to-distortion ratio in decibels. | University | Jul 2019 | Not in cards | |
| FSDKaggle2019 Models identify all relevant sound events in an audio clip using training labels of varying reliability. | School | Jun 2019 | Not in cards | |
| MUStARD Models determine whether a speaker is being sarcastic in a television clip. HELM evaluates the audio track. | Everyday | Jun 2019 | Not in cards | |
| MUSIC-21 Instrument Separation Models separate performances from a mixture using video of the musicians. When two musicians play the same instrument, their movements help identify which sound belongs to each. | University | Apr 2019 | Not in cards | |
| JamendoLyrics Models locate the start of each sung word using a recording and its lyrics. The original evaluation contains 20 English songs. | Everyday | Feb 2019 | Not in cards | |
| MAESTRO Piano Transcription Models transcribe competition piano recordings into notes. The evaluator compares predictions with MIDI events captured by the instrument. Each composition appears in only one split. | Expert | Oct 2018 | Not in cards | |
| MELD Models identify emotions in utterances from multiparty conversations. The dataset includes speech, video, and text. | Everyday | Oct 2018 | Not in cards | |
| AISHELL-2 Models transcribe 1,000 hours of Mandarin speech recorded on several devices. Scored by character error rate. | Everyday | Aug 2018 | 2.56% | Dropped |
| AudioMNIST Models identify a spoken digit from a recording. The test includes recordings from different speakers. | Everyday | Jul 2018 | Not in cards | |
| Urdu Language Speech Emotional Corpus Models identify emotions in excerpts from Urdu television talk shows. The recordings cover 38 speakers and four emotion categories. | Everyday | Jun 2018 | Not in cards | |
| VoxCeleb2 Models compare two interview recordings and decide whether the same person speaks in both. | Everyday | Jun 2018 | | Not in cards |
| Corpus of Multilingual Code-switched Soap Opera Speech Models transcribe South African television dialogue in which speakers switch between English and four Southern African languages. | Everyday | May 2018 | Not in cards | |
| RAVDESS Models identify an actor's expressed emotion from recorded speech or song. | Everyday | May 2018 | Not in cards | |
| MSD-I Multimodal Genre Classification Models predict a track’s genre from its audio and album cover. The published test contains 4,649 tracks across 15 genres. | Everyday | Apr 2018 | Not in cards | |
| MUSIC Instrument Separation Models use images of musicians to isolate an instrument from a mixture of performances. The authors measure separation on mixtures of solo recordings, where each instrument has a reference track. | University | Apr 2018 | Not in cards | |
| Places Audio Captions (Hindi) Models retrieve photographs from spoken Hindi descriptions or retrieve descriptions from photographs. The score counts how often the correct match appears among the top results. | Everyday | Apr 2018 | Not in cards | |
| TUT Urban Acoustic Scenes 2018 Models recognize the setting of a ten-second audio clip recorded in a European city. | Everyday | Apr 2018 | Not in cards | |
| Audio-Visual Event Localization Models identify an event that can be both seen and heard in a video and locate when it occurs. | Everyday | Mar 2018 | Not in cards | |
| FSDKaggle2018 Models assign a sound event label to a Freesound recording. | Everyday | Mar 2018 | Not in cards | |
| LibriCount Models estimate how many speakers appear in a recording with overlapping voices. | Everyday | Dec 2017 | 57.2% | Not in cards |
| UrbanSED Models locate sound events in synthetic urban soundscapes. The benchmark includes reference labels for the times and types of events. | Everyday | Oct 2017 | Not in cards | |
| AISHELL-1 Models transcribe Mandarin read speech. Scored by character error rate. | Everyday | Sep 2017 | | Dropped |
| Lakh MIDI MASD Style Classification Models identify which of 25 styles apply to a MIDI file. The released mapping labels 17,785 files, and each file can have multiple labels. | School | Sep 2017 | Not in cards | |
| Althingi Parliamentary Speech Models transcribe speeches delivered in the Icelandic parliament. Recordings and reference transcripts cover parliamentary proceedings from 2005 through 2016. | Everyday | Aug 2017 | Not in cards | |
| Finnish Parliament ASR test Models transcribe Finnish parliamentary speeches. The original test reports word error rates separately for speakers heard during training and speakers absent from training. | Everyday | Aug 2017 | 9.29% | Not in cards |
| VoxCeleb1 Models identify speakers from recorded speech. NVIDIA also uses the recordings for speaker counting within AIR-Bench. | Everyday | Jun 2017 | Not in cards | |
| Kinetics-400 Models recognize one of 400 human actions in short video clips. | Everyday | May 2017 | | Not in cards |
| ALFFA Wolof Read Speech Test Speech models transcribe sentences read aloud in Wolof. Two speakers each read 1,000 test sentences. The recordings use different speakers from the training set. | Everyday | Apr 2016 | Not in cards | |
| NB Tale Module 3 Models transcribe spontaneous Norwegian speech from speakers with different dialects. The reference transcripts use Bokmål. | Everyday | Dec 2015 | Not in cards | |
| ESC-50 Models identify an environmental sound from a short recording. | Everyday | Oct 2015 | Not in cards | |
| LibriSpeech Models transcribe English audiobook speech. Scored by word error rate on the test-clean and test-other splits. | Everyday | Apr 2015 | | Fading |
| Málrómur Models transcribe Icelandic sentences that volunteers recorded on smartphones. Human listeners checked the reference text against the recordings. | Everyday | Aug 2014 | Not in cards | |
| NCHLT isiZulu Speech Corpus Models transcribe short phrases that native isiZulu speakers read aloud. The archive identifies eight test speakers. | Everyday | May 2014 | Not in cards | |
| NB Tale Module 1 Models transcribe Norwegian newspaper sentences read aloud by speakers from different dialect regions. | Everyday | Nov 2013 | Not in cards | |
| Mixer 6 Speech Models transcribe conversations recorded across microphones and assign words to speakers. NVIDIA evaluates channel four recordings. | Everyday | Aug 2013 | Not in cards | |
| ALFFA Swahili Broadcast News Test Speech models transcribe Swahili radio news. The test includes clean speech, background music, overlapping voices and noisy recordings. | Everyday | May 2012 | Not in cards | |
| Saarland MIDI-Audio Piano Music Models transcribe student piano performances. The Disklavier captured the reference notes as the musicians played. | Expert | Oct 2011 | Not in cards | |
| MIR-1K Models transcribe lyrics from the vocal tracks of karaoke recordings. Qwen compares recognition systems using transcription error. | Everyday | Jul 2009 | Not in cards | |
| IEMOCAP Models identify emotions in recorded dialogues. NVIDIA evaluates this task within AIR-Bench. | Everyday | Nov 2008 | Not in cards | |
| Rundkast Models transcribe Norwegian radio news, interviews and debates. Word error rate measures transcription accuracy. | Everyday | May 2008 | Not in cards | |
| GTZAN Models identify the music genre in short recordings. Qwen reports genre classification accuracy across music understanding systems. | Everyday | Jul 2002 | Not in cards | |
| AMI Meeting Corpus Models identify speakers or transcribe English meetings. NVIDIA evaluates both headset mixes and distant microphone recordings. | Everyday | Not in cards | ||
| Artificial Analysis Speech Arena Voters listen to two speech clips made from the same text and pick the more natural one. Votes become Elo ratings. | Everyday | | Fading | |
| AudioCaps Models match natural language descriptions to audio clips. Meta also compares how accurately encoders match audio, video, and text from the same clip. | Everyday | Not in cards | ||
| AudioSet Strong Models locate labeled sound events in time. The evaluation uses human annotations of event boundaries in AudioSet clips. | Everyday | Not in cards | ||
| CALLHOME American English Speech Models transcribe spontaneous telephone conversations in American English and assign words to speakers. NVIDIA uses the CH109 test subset. | Everyday | Not in cards | ||
| CALLHOME diarization Models identify speakers in multilingual telephone conversations. NVIDIA evaluates Part 2 with two to six speakers per call. | Everyday | Not in cards | ||
| CORAAL Models transcribe recordings of African American speech. NVIDIA reports long-form word error rates by location. | Everyday | Not in cards | ||
| DESED Models identify when sound events occur in domestic environments. Meta compares frame-level predictions across audio encoders. | Everyday | Not in cards | ||
| DesignArena Audio Realism Bench Voice models produce conversational speech, and users vote on which sounds more real. DesignArena built it because 1 to 5 opinion scores for TTS had saturated. | Everyday | 1,181 | Not in cards | |
| DesignArena Speech-to-Speech Voice models answer spoken input with speech. Users listen to two anonymous replies and vote for the better one. | Everyday | Not in cards | ||
| DesignArena Text-to-Speech Speech models read the same text aloud. Users listen to two anonymous clips and vote for the more natural one. | Everyday | | Not in cards | |
| Fake-or-Real Models distinguish synthetic speech from real recordings. NVIDIA evaluates this task within AIR-Bench. | Everyday | Not in cards | ||
| IARPA Babel speech recognition Models transcribe conversational speech in languages with limited labeled data. Meta measures character error against reference transcripts on the Babel development split. | School | Not in cards | ||
| Jarvis Bench Voice agents handle conversational tasks. Listeners rate naturalness, and the board reports task effectiveness separately. | Everyday | | Not in cards | |
| KeSpeech Models transcribe Mandarin speech from speakers with regional accents. Qwen compares transcription error against other speech recognizers. | Everyday | Not in cards | ||
| M4Singer Models transcribe lyrics from singing voices. Qwen compares recognition systems on the M4Singer recordings. | Everyday | Not in cards | ||
| MagnaTagATune Models predict descriptive tags for music clips. Qwen compares their predictions with human annotations using micro F1. | Everyday | Not in cards | ||
| MAPS Piano Transcription Models transcribe piano recordings into notes. A common test uses 60 Disklavier recordings from two microphone setups and checks predicted pitches and timing against MIDI. | Expert | Not in cards | ||
| MIREX 2026 Music Evaluation via CMI-RewardBench Models choose which of two generated recordings listeners prefer for a prompt. Organizers rank systems by accuracy on 249 pairs and report vocal and instrumental results separately. | Everyday | Not in cards | ||
| MIREX 2026 Music Performance Difficulty Prediction Models listen to solo piano recordings and rank the pieces by how hard they are to play. Organizers compare the rankings with piano educators’ judgments. | University | Not in cards | ||
| MTG-Jamendo Models tag songs by genre, mood, theme, and instrument. Qwen reports micro F1 for each tagging task. | Everyday | Not in cards | ||
| Multilingual Conversational Speech Language Model Challenge Models recognize conversational speech across languages. Qwen compares transcription error and language identification accuracy on the challenge recordings. | Everyday | Not in cards | ||
| MUSDB18 Models separate a music mixture into vocals, drums, bass, and accompaniment. The dataset supplies isolated studio recordings for comparison. | Everyday | Not in cards | ||
| MuseBench (Music Score and Performance Understanding) Models answer music theory questions, read piano scores, and interpret recorded performances. The authors report accuracy separately for text, images, and audio. | University | | Not in cards | |
| Numb3rs Models transcribe spoken numbers, including addresses, amounts, and times. NVIDIA compares the output with spoken-form references. | Everyday | Not in cards | ||
| QuartetsMIREX 2026 Audio-to-Score Transcription Models transcribe synthesized string quartet recordings into notation. Organizers scored 373 excerpts by Schubert, Brahms, and Mendelssohn, whose music the original Quartets dataset omits. | Expert | Not in cards | ||
| Speech Arena Models generate speech. Listeners compare outputs, and pairwise votes determine Elo ratings. | Everyday | 1,566 | Not in cards | |
| SpeechIO ASR leaderboard Models transcribe Mandarin recordings from the SpeechIO evaluation sets. Qwen reports an aggregate transcription error. | Everyday | Not in cards | ||
| Speko Speech-to-Speech Benchmark (six concierge scenarios) Voice agents book flights, move reservations, and revise budgets during scripted calls. Speko checks whether agents wait for confirmation and use the latest details. | Everyday | 0.91 | Not in cards | |
| StudyArena Audio Models produce audio responses to user prompts. Anonymous human comparisons determine the published rating in this separate vote pool. | Everyday | | Not in cards | |
| SuperCLUE Arena Text to Speech Models synthesize speech from text. The arena ranks systems from pairwise comparisons. | Everyday | | Not in cards | |
| SuperCLUE-ASR Models transcribe Chinese speech under different acoustic conditions. | Everyday | 12.1% | Not in cards | |
| SuperCLUE-Voice (audiovisual interaction) Models respond to audio and video inputs in real time. | Everyday | Not in cards | ||
| SuperCLUE-Voice (speech interaction) Models interact with users through speech in real time. | Everyday | 73.0 | Not in cards | |
| TED-LIUM Models transcribe English TED talks. The reference transcripts support comparisons by word error rate. | Everyday | Not in cards | ||
| Turing TTS Listeners hear a human recording and a synthesized recording. The score measures how often listeners identify the synthesized recording as human. | Everyday | | Not in cards | |
| URGENT 2024 Models enhance speech under different forms of degradation. NVIDIA measures how accurately a recognizer transcribes the evaluation audio. | Everyday | Not in cards | ||
| VCTK Corpus Models match speech recordings with their texts. Meta evaluates audio-to-text retrieval across model sizes. | Everyday | Not in cards | ||
| vCX Benchmark Models run inside a Retell voice agent on pass or fail checks for business rules, grounding, tool calls and speech-ready replies. Each suite reports its own pass rate. | Everyday | 93.3% | Not in cards | |
| Voice Arena Arabic MSA Models generate speech in Arabic MSA. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Bahasa Models generate speech in Bahasa. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena Brazilian Portuguese Models generate speech in Brazilian Portuguese. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Diarization Bench Models identify who spoke when in recorded conversations. The board measures diarization error and publishes a separate track for speaker attributed transcription. | Everyday | | Not in cards | |
| Voice Arena Dictation Bench Dictation tools turn speech into written drafts. Judges measure the share of drafts a user could send without edits. | Everyday | | Not in cards | |
| Voice Arena French Models generate speech in French. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena Full Duplex Bench Voice agents respond to pauses, changes of speaker, acknowledgments, and interruptions. The board averages success rates across these conversational behaviors. | Everyday | | Not in cards | |
| Voice Arena German Models generate speech in German. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena Hindi Models generate speech in Hindi. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Indian English Models generate speech in Indian English. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena Japanese Models generate speech in Japanese. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Mexican Spanish Models generate speech in Mexican Spanish. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Tagalog Models generate speech in Tagalog. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | Not in cards | |
| Voice Arena US English Models generate speech in US English. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only | |
| Voice Arena Vietnamese Models generate speech in Vietnamese. Listeners compare anonymous clips, and preference votes determine independent language Elo ratings. | Everyday | | System cards only |