OmniEchoBench
Models answer 2,972 spatial questions from recorded audio and video or navigate toward sound sources in 900 episodes. Rules score extracted spatial answers and multiple-choice letters. Navigation metrics measure success within one meter and path efficiency.
The scorer uses a model to extract answers before deterministic checks. Public annotations and code exist, but the README says media download links remain pending. Navigation stopping rules and annotation paths need explicit versioning.
- Released
- September 2026
- Kind
- Embodied reasoning
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Agent task
- Measures
- Capability
- Language
- English
- Interaction
- Agentic
- Environment
- A simulator
- Items from
- Real work and use
- Refresh
- One fixed set
- Run by
- A host
- In views
- Agents, Vision, Audio and speech, Robotics
- Strongest model tested
-
Gemini 3.8 Flash #24 on the capability index
- Last model added
- September 2026
- Scores refreshed
- November 18th, 2026
In the authors’ words
Humans can effortlessly localize the direction of a sound source and integrate it with visual cues for reasoning, yet this remains challenging for embodied agents. In particular, it is still unclear how to effectively evaluate and model spatial audio understanding in embodied settings. To address this gap, we introduce OmniEchoBench, a unified benchmark for spatial audio-visual perception and audio-vision-language navigation. OmniEchoBench comprises six tasks over 197 real-world spatial audio-visual scenes, 2,972 question-answer pairs, and 900 navigation samples with first-order ambisonics (FOA) audio collected from 30 real-world environments. To enable scalable training supervision, we develop a controllable rendering pipeline for spatial audio. It preserves geometric consistency among sound sources, visual observations, and agent trajectories. Building on this, we propose OmniEcho, a spatially aware omni-modal model. It introduces an FOA spatial encoder alongside a pretrained semantic audio pathway. Extensive experiments show that OmniEcho achieves state-of-the-art performance on spatial audio-visual perception. For our sound-guided navigation, OmniEcho reaches a performance level close to that of traditional vision-language navigation. These results demonstrate that spatial audio can serve as a valuable signal for embodied scene reasoning and navigation, while also highlighting fine-grained spatial localization and distance estimation as important open challenges. Our code and data will be available in https://github.com/PKU-VaLuE-Lab/OmniEcho/tree/main
Qwen3.8-Omni-Flash leads with 43.9% among models released by September 2026 Qwen3.8-Omni-Flash leads with 43.9% among models released by September 2026
Best score per entry from the benchmark's own leaderboard, read on November 18th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- Claude Opus 5.5
- and 16 more
- 1
Qwen3.8-Omni-Flash 43.9%
- 2
Gemini 3.8 Flash 32%
- 3
Gemini 3.7 Flash 31.4%
- 4
Muse Spark 1.2 31.3%
- 5 OmniEcho (Ours) 28.5%
- 6 doubao-seed-2-0-lite-260428 28.3%
- 7
Qwen3.5-Omni-Plus 23.3%
- 8
Qwen3-Omni-30B-A3B 18.5%
- 9
Qwen2.5-Omni-7B 12.6%
- 10 SO-7B 11.4%
Source: Official OmniEchoBench leaderboard
- 1
Qwen3.8-Omni-Flash 43.9%
- 2
Gemini 3.8 Flash 32%
- 3
Gemini 3.7 Flash 31.4%
- 4
Muse Spark 1.2 31.3%
- 5
Qwen3.5-Omni-Plus 23.3%
- 6
Qwen3-Omni-30B-A3B 18.5%
- 7
Qwen2.5-Omni-7B 12.6%
Source: Official OmniEchoBench leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 43.9% | Sep 2026 | Official leaderboard | |
| 2 | | 32% | Sep 2026 | Official leaderboard | |
| 3 | | 31.4% | Aug 2026 | Official leaderboard | |
| 4 | | 31.3% | Aug 2026 | Official leaderboard | |
| 5 | OmniEcho (Ours) | 28.5% | Official leaderboard | ||
| 6 | doubao-seed-2-0-lite-260428 | 28.3% | Official leaderboard | ||
| 7 | | 23.3% | Mar 2026 | Official leaderboard | |
| 8 | | 18.5% | Sep 2025 | Official leaderboard | |
| 9 | | 12.6% | Mar 2025 | Official leaderboard | |
| 10 | SO-7B | 11.4% | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 43.9% | Sep 2026 | Official leaderboard | |
| 2 | | 32% | Sep 2026 | Official leaderboard | |
| 3 | | 31.4% | Aug 2026 | Official leaderboard | |
| 4 | | 31.3% | Aug 2026 | Official leaderboard | |
| 5 | | 23.3% | Mar 2026 | Official leaderboard | |
| 6 | | 18.5% | Sep 2025 | Official leaderboard | |
| 7 | | 12.6% | Mar 2025 | Official leaderboard |
Rankings by part
We read 5 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as Camera rotation, Source direction and 3D localization. Qwen3.8-Omni-Flash leads every one of them. Arrows show how far a model moved from its overall rank among the models we track.
Camera rotation
7 models we track have a score here.
- 1
Qwen3.8-Omni-Flash 74.1%
- 2
Muse Spark 1.2 ↑2 55%
- 3
Gemini 3.8 Flash ↓1 53.7%
- 4
Qwen3.5-Omni-Plus ↑1 52.4%
- 5
Gemini 3.7 Flash ↓2 45.3%
- 6
Qwen3-Omni-30B-A3B 32.7%
- 7
Qwen2.5-Omni-7B 24.9%
Source direction
7 models we track have a score here.
- 1
Qwen3.8-Omni-Flash 47.9%
- 2
Muse Spark 1.2 ↑2 34.2%
- 3
Gemini 3.8 Flash ↓1 34.1%
- 4
Gemini 3.7 Flash ↓1 32.8%
- 5
Qwen3-Omni-30B-A3B ↑1 16.3%
- 6
Qwen3.5-Omni-Plus ↓1 16.1%
- 7
Qwen2.5-Omni-7B 8.00%
3D localization
7 models we track have a score here.
- 1
Qwen3.8-Omni-Flash 28.6%
- 2
Gemini 3.7 Flash ↑1 24.4%
- 3
Gemini 3.8 Flash ↓1 22.1%
- 4
Qwen3.5-Omni-Plus ↑1 19.6%
- 5
Muse Spark 1.2 ↓1 19.6%
- 6
Qwen3-Omni-30B-A3B 7.70%
- 7
Qwen2.5-Omni-7B 5.40%
Source motion
7 models we track have a score here.
- 1
Qwen3.8-Omni-Flash 37.4%
- 2
Gemini 3.7 Flash ↑1 33.4%
- 3
Gemini 3.8 Flash ↓1 31.3%
- 4
Muse Spark 1.2 29.6%
- 5
Qwen3.5-Omni-Plus 22.2%
- 6
Qwen3-Omni-30B-A3B 20.9%
- 7
Qwen2.5-Omni-7B 14%
Cognitive map
7 models we track have a score here.
- 1
Qwen3.8-Omni-Flash 39.5%
- 2
Gemini 3.7 Flash ↑1 26.2%
- 3
Gemini 3.8 Flash ↓1 26%
- 4
Qwen3.5-Omni-Plus ↑1 25.2%
- 5
Muse Spark 1.2 ↓1 25.2%
- 6
Qwen3-Omni-30B-A3B 22.3%
- 7
Qwen2.5-Omni-7B 20%
3 models have held number one on the official leaderboard since its release in September 2026
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official OmniEchoBench leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official OmniEchoBench leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Robotics
- ERQA In use 400 multiple-choice visual questions from robot and egocentric footage on spatial, trajectory, action and state reasoning. Accuracy.
Gemini Robotics ER 2 78.5%
- SimplerEnv In use Real-to-sim replicas of Google Robot and WidowX/Bridge setups. Policies scored by success rate on pick, move, drawer and Bridge tasks. Praxis-1 80%
- RoboLab In use Zero-shot DROID Franka policies in Isaac Lab on 120 tabletop tasks at three instruction levels, 10 episodes each. Success rate. Praxis-1 46.2%
- RoboDojo In use Generalist manipulation policies run 42 Isaac Sim tasks and 18 remote real-robot tasks. Scored by task score and success rate. GPT-6 Astra 27.6%
- Progress classification (Google DeepMind) In use Models assign each frame of a robot video to one of five task-progress bands from 0 to 100 percent. Scored by accuracy.
Gemini Robotics ER 2 57.4%
- Human proximity (Google DeepMind robotics) In use Models judge from robot camera views whether a human is within 1 meter, a spatial awareness check for safe robot behavior. Scored by accuracy.
Gemini Robotics ER 2 93%
Data audit
9 of 10 checks pass, 1 warning. Look first at the Task access check. We don’t know whether the tasks are public. Every benchmark
Catalog record 1 warning 1 pass
- Warning Task access We don’t know whether the tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (10 entries).
- Pass Models scored 7 models we track, plus 3 other entries on its leaderboard.
- Pass Newest score The newest is from September 2026, for Qwen3.8-Omni-Flash.
- Note Frontier models None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24).
- Pass Headroom The best is 43.9%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on pku-value-lab.github.io: 7 rows for models we track and 3 other entries at the last read.
- Pass Last read November 18th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.