MER-UniBench
Models describe emotions from audio, video, and text across nine datasets. Label normalization supports hit rate, weighted F1, and emotion-wheel F1. The reported mean averages the primary metric from each dataset.
The catalog's paper link points to OV-MER, not this benchmark. MER-Caption and MER2025 are separate training resources. Modality combinations must match for comparisons.
- Released
- January 2025
- Kind
- Audio understanding and voice
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Open-ended
- Tasks
- All public
- Measures
- Capability
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Vision, Audio and speech
In the authors’ words
The emergence of multimodal large language models (MLLMs) advances multimodal emotion recognition (MER) to the next level, from naive discriminative tasks to complex emotion understanding with advanced video understanding abilities and natural language description. However, the current community suffers from a lack of large-scale datasets with intensive, descriptive emotion annotations, as well as a multimodal-centric framework to maximize the potential of MLLMs for emotion understanding. To address this, we establish a new benchmark for MLLM-based emotion understanding with a novel dataset (MER-Caption) and a new model (AffectGPT). Utilizing our model-based crowd-sourcing data collection strategy, we construct the largest descriptive emotion dataset to date (by far), featuring over 2K fine-grained emotion categories across 115K samples. We also introduce the AffectGPT model, designed with pre-fusion operations to enhance multimodal integration. Finally, we present MER-UniBench, a unified benchmark with evaluation metrics tailored for typical MER tasks and the free-form, natural language output style of MLLMs. Extensive experimental results show AffectGPT's robust performance across various MER tasks. We have released both the code and the dataset to advance research and development in emotion understanding: https://github.com/zeroQiaoba/AffectGPT.
AffectGPT tops the leaderboard with 74.8
Best score per entry from the benchmark's paper, read on November 18th, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.
- 1 AffectGPT audio+video+text 74.8
- 2 Emotion-LLaMA audio+video+text 64.2
- 3 mPLUG-Owl video+text 62.5
- 4 Chat-UniVi video+text 57.9
- 5 SALMONN audio+text 57.9
- 6 VideoChat video+text 56.7
- 7 LLaMA-VID video+text 56.1
- 8 PandaGPT audio+video+text 52.9
- 9 VideoChat2 video+text 52.7
- 10 Video-ChatGPT video+text 52.6
- 11 Qwen-Audio audio+text 49.3
- 12 SECap audio+text 46.6
- 13 Video-LLaVA video+text 44.4
- 14 OneLLM audio+text 41.1
- 15 Otter video+text 34.8
Source: The MER-UniBench paper
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | AffectGPT | 74.8 | audio+video+text | Official leaderboard | |
| 2 | Emotion-LLaMA | 64.2 | audio+video+text | Official leaderboard | |
| 3 | mPLUG-Owl | 62.5 | video+text | Official leaderboard | |
| 4 | Chat-UniVi | 57.9 | video+text | Official leaderboard | |
| 5 | SALMONN | 57.9 | audio+text | Official leaderboard | |
| 6 | VideoChat | 56.7 | video+text | Official leaderboard | |
| 7 | LLaMA-VID | 56.1 | video+text | Official leaderboard | |
| 8 | PandaGPT | 52.9 | audio+video+text | Official leaderboard | |
| 9 | VideoChat2 | 52.7 | video+text | Official leaderboard | |
| 10 | Video-ChatGPT | 52.6 | video+text | Official leaderboard | |
| 11 | Qwen-Audio | 49.3 | audio+text | Official leaderboard | |
| 12 | SECap | 46.6 | audio+text | Official leaderboard | |
| 13 | Video-LLaVA | 44.4 | video+text | Official leaderboard | |
| 14 | OneLLM | 41.1 | audio+text | Official leaderboard | |
| 15 | Otter | 34.8 | video+text | Official leaderboard |
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Audio and video
- Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy.
Qwen 3.8 Max 88.7%
- LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization.
Gemini 4 Argon 91.7%
- MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy.
Qwen 3.8 Max 82.4%
- MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy.
Qwen 3.8 Max 90.8%
- FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better.
Phi-4-multimodal 4.00%
- VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules.
Qwen3.5-Omni-Plus 93.1%
Data audit
All 8 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
- Pass Independent runs Scores from the benchmark’s paper (15 entries).
- Pass Models scored 15 entries on its leaderboard, none a model we track.
- Note Newest score No scores yet.
- Note Frontier models No models scored yet.
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on arxiv.org: 15 entries at the last read, none a model we track.
- Pass Last read November 18th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.