DeepSearchQA
The official leaderboard hasn’t scored a model released after GPT-5.4 (March 2026). Labs report their own numbers for 20 newer models, such as Kimi K3 at 95%.
Questions that need multi-step search and synthesis. Final answers graded against references.
Official grading needs the gemini-2.5-flash autorater and prompt. Other judges shift results. Only Kaggle starter code, no GitHub repo.
- Made by
- Google DeepMind
- Released
- January 2026
- Kind
- Deep research
- Level
- Expert
- Scored
- Checked automatically
- Answers
- Agent task
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Agentic
- Environment
- Browser
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Agents
- Strongest model tested
- Claude Opus 5 #11 on the capability index
- Last in a launch post
- November 2026
- Last model added
- November 2026
- Scores refreshed
- November 18th, 2026
In the authors’ words
We introduce DeepSearchQA, a 900-prompt benchmark for evaluating agents on difficult multi-step information-seeking tasks across 17 different fields. Unlike traditional benchmarks that target single answer retrieval or broad-spectrum factuality, DeepSearchQA features a dataset of challenging, handcrafted tasks designed to evaluate an agent's ability to execute complex search plans to generate exhaustive answer lists. This shift in design explicitly tests three critical, yet under-evaluated capabilities: 1) systematic collation of fragmented information from disparate sources, 2) de-duplication and entity resolution to ensure precision, and 3) the ability to reason about stopping criteria within an open-ended search space. Each task is structured as a causal chain, where discovering information for one step is dependent on the successful completion of the previous one, stressing long-horizon planning and context retention. All tasks are grounded in the open web with objectively verifiable answer sets. Our comprehensive evaluation of state-of-the-art agent architectures reveals significant performance limitations: even the most advanced models struggle to balance high recall with precision. We observe distinct failure modes ranging from premature stopping (under-retrieval) to hedging behaviors, where agents cast an overly wide net of low-confidence answers to artificially boost recall. These findings highlight critical headroom in current agent designs and position DeepSearchQA as an essential diagnostic tool for driving future research toward more robust, deep-research capabilities.
2 models share the top score of 95%
Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.
GPT-5 Pro leads with 65.2% across all independent runs
Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.
Kimi K3 leads with 95%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 95.0 Kimi K3
- 95.0 Claude Opus 5
- 93.1 Claude Opus 4.8
- 92.8
Step 3.7 Flash
- 92.5 Kimi K2.6
- 91.3 Claude Opus 4.6
- 91.0
Tencent Hy3
- 90.9
Inkling 2 Small
- 90.3
Muse Spark 1.3
- 89.2 Claude Sonnet 4.6
- 89.1 Claude Opus 4.7
- 86.7 MiMo-V2-Pro
- 85.3
Inkling 2 Mini
- 84.9
Muse Spark 1.1
- 81.6 Grok 4.6
- 80.1 Beam 1.0
- 77.4
Seed2.0 Pro
- 77.3
ERNIE 5.1
- 77.1 Kimi K2.5
- 74.8
Muse Spark
- 74.6
Muse Glimmer
- 71.3 Mercury 3
- 38.4 Grok 4.5
- 34.0 Mercury 2.5
- 1 Kimi K3 95%
- 2 Claude Opus 5 max effort 95%
- 3 Claude Opus 4.8 max effort 93.1%
- 4
Step 3.7 Flash 92.8%
- 5 Kimi K2.6 92.5%
- 6 Claude Opus 4.6 max effort 91.3%
- 7
Tencent Hy3 91%
- 8
Inkling 2 Small 90.9%
- 9
Muse Spark 1.3 90.3%
- 10 Claude Sonnet 4.6 max effort 89.2%
- 11 Claude Opus 4.7 max effort 89.1%
- 12 MiMo-V2-Pro 86.7%
- 13
Inkling 2 Mini 85.3%
- 14
Muse Spark 1.1 84.9%
- 15 Grok 4.6 81.6%
- 16 Beam 1.0 80.1%
- 17
Seed2.0 Pro 77.4%
- 18
ERNIE 5.1 77.3%
- 19 Kimi K2.5 77.1%
- 20
Muse Spark 74.8%
- 21
Muse Glimmer 74.6%
- 22 Mercury 3 71.3%
- 23 Grok 4.5 38.4%
- 24 Mercury 2.5 34%
Source: Labs’ model cards
- 1 GPT-5 Pro 65.2%
- 2 GPT-5.4 63.7%
- 3
Gemini 3.1 Pro 60.2%
- 4 GPT-5 59.4%
- 5 OpenAI deep research 44.2%
- 6 o4-mini-deep-research 40.4%
- 7
Gemini 2.5 Flash Jun 2025 25.9%
- 8
Gemini 2.5 Pro Jun 2025 24.6%
- 9 Claude Opus 4.5 thinking 24%
- 10 Claude Sonnet 4.5 thinking 16%
- 11 Claude Haiku 4.5 thinking 12.8%
Source: Official DeepSearchQA leaderboard
- 95.0 Kimi K3
- 95.0 Claude Opus 5
- 93.1 Claude Opus 4.8
- 92.8
Step 3.7 Flash
- 92.5 Kimi K2.6
- 91.3 Claude Opus 4.6
- 91.0
Tencent Hy3
- 90.9
Inkling 2 Small
- 90.3
Muse Spark 1.3
- 89.2 Claude Sonnet 4.6
- 89.1 Claude Opus 4.7
- 86.7 MiMo-V2-Pro
- 85.3
Inkling 2 Mini
- 84.9
Muse Spark 1.1
- 81.6 Grok 4.6
- 80.1 Beam 1.0
- 77.4
Seed2.0 Pro
- 77.3
ERNIE 5.1
- 77.1 Kimi K2.5
- 74.8
Muse Spark
- 74.6
Muse Glimmer
- 71.3 Mercury 3
- 65.2 GPT-5 Pro
- 63.7 GPT-5.4
- 60.2
Gemini 3.1 Pro
- 59.4 GPT-5
- 44.2 OpenAI deep research
- 40.4 o4-mini-deep-research
- 38.4 Grok 4.5
- 34.0 Mercury 2.5
- 25.9
Gemini 2.5 Flash
- 24.6
Gemini 2.5 Pro
- 24.0 Claude Opus 4.5
- 16.0 Claude Sonnet 4.5
- 12.8 Claude Haiku 4.5
- 1 Kimi K3 95%
- 2 Claude Opus 5 max effort 95%
- 3 Claude Opus 4.8 max effort 93.1%
- 4
Step 3.7 Flash 92.8%
- 5 Kimi K2.6 92.5%
- 6 Claude Opus 4.6 max effort 91.3%
- 7
Tencent Hy3 91%
- 8
Inkling 2 Small 90.9%
- 9
Muse Spark 1.3 90.3%
- 10 Claude Sonnet 4.6 max effort 89.2%
- 11 Claude Opus 4.7 max effort 89.1%
- 12 MiMo-V2-Pro 86.7%
- 13
Inkling 2 Mini 85.3%
- 14
Muse Spark 1.1 84.9%
- 15 Grok 4.6 81.6%
- 16 Beam 1.0 80.1%
- 17
Seed2.0 Pro 77.4%
- 18
ERNIE 5.1 77.3%
- 19 Kimi K2.5 77.1%
- 20
Muse Spark 74.8%
- 21
Muse Glimmer 74.6%
- 22 Mercury 3 71.3%
- 23 GPT-5 Pro 65.2%
- 24 GPT-5.4 63.7%
- 25
Gemini 3.1 Pro 60.2%
- 26 GPT-5 59.4%
- 27 OpenAI deep research 44.2%
- 28 o4-mini-deep-research 40.4%
- 29 Grok 4.5 38.4%
- 30 Mercury 2.5 34%
- 31
Gemini 2.5 Flash Jun 2025 25.9%
- 32
Gemini 2.5 Pro Jun 2025 24.6%
- 33 Claude Opus 4.5 thinking 24%
- 34 Claude Sonnet 4.5 thinking 16%
- 35 Claude Haiku 4.5 thinking 12.8%
Sources: Labs’ model cards and official DeepSearchQA leaderboard
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Kimi K3 | 95% | Jul 2026 | Lab model card | |
| 2 | Claude Opus 5 | 95% | max effort | Jul 2026 | Lab model card |
| 3 | Claude Opus 4.8 | 93.1% | max effort | May 2026 | Lab model card |
| 4 | | 92.8% | May 2026 | Lab model card | |
| 5 | Kimi K2.6 | 92.5% | Apr 2026 | Lab model card | |
| 6 | Claude Opus 4.6 | 91.3% | max effort | Feb 2026 | Lab model card |
| 7 | | 91% | Jul 2026 | Lab model card | |
| 8 | | 90.9% | Nov 2026 | Lab model card | |
| 9 | | 90.3% | Sep 2026 | Lab model card | |
| 10 | Claude Sonnet 4.6 | 89.2% | max effort | Feb 2026 | Lab model card |
| 11 | Claude Opus 4.7 | 89.1% | max effort | Apr 2026 | Lab model card |
| 12 | MiMo-V2-Pro | 86.7% | Mar 2026 | Lab model card | |
| 13 | | 85.3% | Nov 2026 | Lab model card | |
| 14 | | 84.9% | Jul 2026 | Lab model card | |
| 15 | Grok 4.6 | 81.6% | Aug 2026 | Lab model card | |
| 16 | Beam 1.0 | 80.1% | Oct 2026 | Lab model card | |
| 17 | | 77.4% | Feb 2026 | Lab model card | |
| 18 | | 77.3% | May 2026 | Lab model card | |
| 19 | Kimi K2.5 | 77.1% | Jan 2026 | Lab model card | |
| 20 | | 74.8% | Apr 2026 | Lab model card | |
| 21 | | 74.6% | Aug 2026 | Lab model card | |
| 22 | Mercury 3 | 71.3% | Oct 2026 | Lab model card | |
| 23 | Grok 4.5 | 38.4% | Jul 2026 | Lab model card | |
| 24 | Mercury 2.5 | 34% | Sep 2026 | Lab model card |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-5 Pro | 65.2% | Oct 2025 | Official leaderboard | |
| 2 | GPT-5.4 | 63.7% | Mar 2026 | Official leaderboard | |
| 3 | 60.2% | Feb 2026 | Official leaderboard | ||
| 4 | GPT-5 | 59.4% | Aug 2025 | Official leaderboard | |
| 5 | OpenAI deep research | 44.2% | Feb 2025 | Official leaderboard | |
| 6 | o4-mini-deep-research | 40.4% | Jun 2025 | Official leaderboard | |
| 7 | 25.9% | Jun 2025 | Official leaderboard | ||
| 8 | 24.6% | Jun 2025 | Official leaderboard | ||
| 9 | Claude Opus 4.5 | 24% | thinking | Nov 2025 | Official leaderboard |
| 10 | Claude Sonnet 4.5 | 16% | thinking | Sep 2025 | Official leaderboard |
| 11 | Claude Haiku 4.5 | 12.8% | thinking | Oct 2025 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Kimi K3 | 95% | Jul 2026 | Lab model card | |
| 2 | Claude Opus 5 | 95% | max effort | Jul 2026 | Lab model card |
| 3 | Claude Opus 4.8 | 93.1% | max effort | May 2026 | Lab model card |
| 4 | 92.8% | May 2026 | Lab model card | ||
| 5 | Kimi K2.6 | 92.5% | Apr 2026 | Lab model card | |
| 6 | Claude Opus 4.6 | 91.3% | max effort | Feb 2026 | Lab model card |
| 7 | 91% | Jul 2026 | Lab model card | ||
| 8 | 90.9% | Nov 2026 | Lab model card | ||
| 9 | 90.3% | Sep 2026 | Lab model card | ||
| 10 | Claude Sonnet 4.6 | 89.2% | max effort | Feb 2026 | Lab model card |
| 11 | Claude Opus 4.7 | 89.1% | max effort | Apr 2026 | Lab model card |
| 12 | MiMo-V2-Pro | 86.7% | Mar 2026 | Lab model card | |
| 13 | 85.3% | Nov 2026 | Lab model card | ||
| 14 | 84.9% | Jul 2026 | Lab model card | ||
| 15 | Grok 4.6 | 81.6% | Aug 2026 | Lab model card | |
| 16 | Beam 1.0 | 80.1% | Oct 2026 | Lab model card | |
| 17 | 77.4% | Feb 2026 | Lab model card | ||
| 18 | 77.3% | May 2026 | Lab model card | ||
| 19 | Kimi K2.5 | 77.1% | Jan 2026 | Lab model card | |
| 20 | 74.8% | Apr 2026 | Lab model card | ||
| 21 | 74.6% | Aug 2026 | Lab model card | ||
| 22 | Mercury 3 | 71.3% | Oct 2026 | Lab model card | |
| 23 | GPT-5 Pro | 65.2% | Oct 2025 | Official leaderboard | |
| 24 | GPT-5.4 | 63.7% | Mar 2026 | Official leaderboard | |
| 25 | 60.2% | Feb 2026 | Official leaderboard | ||
| 26 | GPT-5 | 59.4% | Aug 2025 | Official leaderboard | |
| 27 | OpenAI deep research | 44.2% | Feb 2025 | Official leaderboard | |
| 28 | o4-mini-deep-research | 40.4% | Jun 2025 | Official leaderboard | |
| 29 | Grok 4.5 | 38.4% | Jul 2026 | Lab model card | |
| 30 | Mercury 2.5 | 34% | Sep 2026 | Lab model card | |
| 31 | 25.9% | Jun 2025 | Official leaderboard | ||
| 32 | 24.6% | Jun 2025 | Official leaderboard | ||
| 33 | Claude Opus 4.5 | 24% | thinking | Nov 2025 | Official leaderboard |
| 34 | Claude Sonnet 4.5 | 16% | thinking | Sep 2025 | Official leaderboard |
| 35 | Claude Haiku 4.5 | 12.8% | thinking | Oct 2025 | Official leaderboard |
The benchmark's own leaderboard has 12 entries, 11 of them models we track. Switch the scores above to rank them together.
5 models have held number one in labs’ own numbers since its release in January 2026 Number one over time 6 models have held number one with claims included since its release in January 2026
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Anthropic’s own number for Claude Opus 4.6 passed 85% in February 2026
That was 1 month after its release. By the same age, 20% of the benchmarks from 2026 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official DeepSearchQA leaderboard and labs’ model cards
6 labs put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Nov 20, 2026 | 85.3 | DeepSearchQA with compaction | |
| Nov 5, 2026 | 90.9 | DeepSearchQA, 90.9% | |
| Oct 19, 2026 | Mercury 3 | 71.3 | DeepSearchQA |
| Oct 5, 2026 | Beam 1.0 | 80.1 | DeepSearchQA w/context management |
| Sep 8, 2026 | Mercury 2.5 | 34 | DSQA (@10 tool calls) |
| Sep 2, 2026 | 90.3 | DeepSearchQA (Agentic browsing) | |
| Aug 10, 2026 | 74.6 | DeepSearch QA | |
| Jul 16, 2026 | Kimi K3 | 95 | DeepSearchQA (F1) |
| Jul 2, 2026 | 91 | DeepSearchQA | |
| May 29, 2026 | 92.8 | deepsearchQA (F1), acc 81.7 | |
| May 9, 2026 | 77.3 | DeepSearchQA (F1) | |
| Apr 20, 2026 | Kimi K2.6 | 92.5 | DeepSearchQA (f1-score) |
| Apr 8, 2026 | 74.8 | DeepSearchQA (Agentic Search) | |
| Mar 18, 2026 | MiMo-V2-Pro | 86.7 | DeepSearch QA-F1 |
| Feb 14, 2026 | 77.4 | DeepSearchQA | |
| Jan 27, 2026 | Kimi K2.5 | 77.1 | DeepSearchQA |
13 system cards report it, from 6 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Nov 20, 2026 | | 85.3 | DeepSearchQA with compaction | Capabilities |
| Oct 19, 2026 | | 74.6 | DeepSearchQA, high, 75.3 xhigh (Tables 2 and 5) | Capabilities |
| Aug 12, 2026 | Grok 4.6 model card | 81.6 | DeepSearchQA, internal implementation, accuracy, high (6.2) | Capabilities |
| Jul 27, 2026 | Kimi K3 technical report | 95 | DeepSearchQA (F1) (Table 2) | Capabilities |
| Jul 24, 2026 | Claude Opus 5 system card | 95 | DeepSearchQA mean F1 at max effort, 980k-token budget, printed on chart (8.10.3, Figure 8.10.3.A) | Capabilities |
| Jul 14, 2026 | Grok 4.5 model card | 38.4 | DeepSearchQA, internal implementation, accuracy, high (6.2) | Capabilities |
| Jul 9, 2026 | | 84.9 | DeepSearchQA (Figure 44) | Capabilities |
| May 28, 2026 | Claude Opus 4.8 system card | 93.1 | DeepSearchQA F1, max effort, 1M-token budget, no compaction (fully correct 84.8%) (Table 8.10.3.B) | Capabilities |
| Apr 16, 2026 | Claude Opus 4.7 system card | 89.1 | DeepSearchQA F1, adaptive thinking at max effort, compaction up to 10M tokens. Fully correct 80.7 (Table 8.8.3.B) | Capabilities |
| Feb 17, 2026 | Claude Sonnet 4.6 system card | 89.2 | DeepSearchQA F1, single agent, adaptive thinking, max effort, up to 10M tokens; multi-agent 91.1 (Figure 2.20.3.A, 2.20.3.1) | Capabilities |
| Feb 14, 2026 | | 77.4 | DeepSearchQA (Table 11) | Capabilities |
| Feb 5, 2026 | Claude Opus 4.6 system card | 91.3 | DeepSearchQA F1, best single-agent configuration, adaptive thinking, max effort, compaction (2.21.3.2, Figure 2.21.3.1.A); multi-agent 92.5 | Capabilities |
| Jan 30, 2026 | Kimi K2.5 technical report | 77.1 | DeepSearchQA (Table 4) | Capabilities |
More in Tool use
- AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state.
Gemini 4 Neon 56.3%
- MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks.
Gemini 4 Argon 90.7%
- BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes.
Qwen3.7-Max 75%
- τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state.
Qwen 3.8 Max 55.1%
- Toolathlon-Verified In use 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. GLM-5.3-Flash 78.4%
- Claw-Eval In use 300 human-verified agent tasks (service orchestration, multimodal, multi-turn dialogue) audited via traces and snapshots. Rubric scores with Pass^3 consistency. Claude Sonnet 4.6 81.4%
Data audit
All 9 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (12 entries), launch posts (16 scores) and system cards (13 scores).
- Pass Models scored 35 models we track, plus 1 other entry on its leaderboard.
- Pass Newest score The newest is from November 2026, for Inkling 2 Mini.
- Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11).
- Note Headroom Saturated. The best is 95%, past 85% since February 2026.
Leaderboard import 3 pass
- Pass Importer We read the board on kaggle.com: 11 rows for models we track and 1 other entry at the last read.
- Pass Last read November 18th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.