ATLAS (Exa)
Agents search the web to build a full table of every entity that meets a set of conditions, with 2 to 10 attributes each. A row counts only if every cell is correct.
547 research tasks, published October 8th, 2026. Fireworks runs 200 of them in Exa's Scout agent for 9 models, scored by row F1.
- Made by
- Exa
- Released
- October 2026
- Kind
- Deep research
- Level
- Expert
- Scored
- Graded by a model
- Answers
- Agent task
No scores here yet
No lab, leaderboard or Epoch AI run has reported a score for ATLAS (Exa) yet.
No lab model card reports it yet
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.
More in Tool use
- AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state.
Gemini 4 Carbon 53.9% - MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. Gemini 4 Argon 90.7%
- BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes.
Qwen3.7-Max 75% - τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. Qwen 3.8 Max 55.1%
- Toolathlon-Verified In use 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. GLM-5.3-Flash 78.4%
- DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%