Benchmark Almanac

APEX-Agents 1.1

Gemini 4 Carbon 90.4% leads since October 2026
Not in model cards

Agents do long banking, consulting and law tasks in worlds with docs, sheets, email and chat. A new judge scores hedged multiple answers as zero.

Mercor audited the tasks three times, changed the judge model and the system prompt to stop scattergunning. Rankings shifted against 1.0.

Made by
Mercor
Released
September 2026
Kind
Simulated companies
Level
Expert
Scored
Graded by a model
Answers
Agent task
Tasks
All public
Strongest model tested
Claude Fable 5.2 #1 on the capability index
Last model added
November 2026
Scores refreshed
November 13th, 2026

In the authors’ words

We introduce the AI Productivity Index for Agents (APEX-Agents), a benchmark for assessing whether AI agents can execute long-horizon, cross-application tasks created by investment banking analysts, management consultants, and corporate lawyers. APEX-Agents requires agents to navigate realistic work environments with files and tools. We test eight agents for the leaderboard using Pass@1. Gemini 3 Flash (Thinking=High) achieves the highest score of 24.0%, followed by GPT-5.2 (Thinking=High), Claude Opus 4.5 (Thinking=High), and Gemini 3 Pro (Thinking=High). We open source the APEX-Agents benchmark (n=480) with all prompts, rubrics, gold outputs, files, and metadata. We also open source Archipelago, our infrastructure for agent execution and evaluation.

Abstract of APEX-Agents (arXiv:2601.14242, CC0)
69
Entries on its own leaderboard, 65 models we track

Gemini 4 Carbon leads with 90.4%

Best score per entry from the benchmark's own leaderboard, read on November 13th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

69 entries. Scroll sideways for the rest.

  1. 90.4 Gemini 4 Carbon
  2. 87.6 Claude Fable 5.2
  3. 87.4 Gemini 4 Argon
  4. 85.1 Tencent Hy4
  5. 84.0 Gemini 4 Hydrogen
  6. 83.7 Claude Sonnet 5.5
  7. 81.3 Claude Opus 5.5
  8. 79.4 Gemini 3.7 Flash
  9. 79.4 DeepSeek V4.1 Flash 1017
  10. 79.3 GPT-6.1 Terra
  11. 77.3 Grok 4.6
  12. 77.1 Claude Fable 5.1
  13. 77.0 Claude Opus 5
  14. 77.0 Muse Wonder
  15. 75.9 Kimi K3.1 Pro
  16. 75.7 Muse Ember
  17. 75.1 GPT-6 Astra
  18. 74.5 Claude Fable 5
  19. 74.3 Qwen 3.8 Max
  20. 73.2 GPT-6.1 Sol
  21. 72.9 Gemini 3.8 Flash
  22. 71.8 GPT-5.6 Terra
  23. 70.5 Grok 4.5
  24. 70.4 MiMo V2.6 Flash RL
  25. 70.4 Muse Spark 1.3
  26. 70.3 GPT-5.5
  27. 68.7 Grok 4.7
  28. 68.7 GPT-6 Sol
  29. 68.3 Claude Haiku 5.5
  30. 68.2 GPT-5.4
  31. 68.1 MiniMax-M3.1-Flash
  32. 67.5 DeepSeek-V4-Flash
  33. 67.4 Claude Sonnet 5
  34. 66.8 GLM-5.3-Flash
  35. 66.2 GLM-5.3
  36. 65.3 MiMo-V2.6 Pro
  37. 65.2 GPT-5.6 Sol
  38. 64.4 Claude Opus 4.8
  39. 64.3 Mistral Large 4
  40. 63.4 Gemini 3.6 Flash
  41. 63.1 Claude Opus 4.7
  42. 62.9 Mercury 3
  43. 62.5 Beam
  44. 62.3 DeepSeek V4 Pro 0813
  45. 62.2 Kimi K3
  46. 62.2 Qwen 3.8 27B
  47. 61.8 Claude Opus 4.6
  48. 61.1 GLM-5.2
  49. 60.9 Inkling 2 Small
  50. 59.8 Muse Spark 1.2
  51. 59.0 GPT-6 Luna
  52. 57.8 GPT-5.6 Luna
  53. 57.6 Claude Sonnet 4.6
  54. 56.8 GLM-5.1
  55. 53.5 MiniMax-M3
  56. 53.4 Kimi K2.7 Code
  57. 52.2 Gemini 3.1 Pro
  58. 50.5 DeepSeek V4.1 Flash
  59. 48.8 Inkling
  60. 48.7 Muse Spark Open
  61. 46.2 Muse Spark 1.1
  62. 46.1 Gemini 3.5 Flash-Lite
  63. 45.9 Nemotron 3.5 Super VL
  64. 44.3 Gemini 3.5 Flash
  65. 43.9 Nemotron 3.5 Bolt
  66. 40.3 Qwen3.5
  67. 38.3 Nemotron 3 Ultra
  68. 35.1 DeepSeek-V3.2
  69. 8.30 gpt-oss-120b
  1. 1 Gemini 4 Carbon high 90.4%
  2. 2 Claude Fable 5.2 max 87.6%
  3. 3 Gemini 4 Argon high 87.4%
  4. 4 Tencent Hy4 high 85.1%
  5. 5 Gemini 4 Hydrogen xhigh 84%
  6. 6 Claude Sonnet 5.5 max 83.7%
  7. 7 Claude Opus 5.5 max 81.3%
  8. 8 Gemini 3.7 Flash high 79.4%
  9. 9 DeepSeek V4.1 Flash 1017 max 79.4%
  10. 10 GPT-6.1 Terra max 79.3%
  11. 11 Grok 4.6 xhigh 77.3%
  12. 12 Claude Fable 5.1 max 77.1%
  13. 13 Claude Opus 5 max 77%
  14. 14 Muse Wonder max 77%
  15. 15 Kimi K3.1 Pro max 75.9%
  16. 16 Muse Ember max 75.7%
  17. 17 GPT-6 Astra max 75.1%
  18. 18 Claude Fable 5 max 74.5%
  19. 19 Qwen 3.8 Max xhigh 74.3%
  20. 20 GPT-6.1 Sol max 73.2%
  21. 21 Gemini 3.8 Flash high 72.9%
  22. 22 GPT-5.6 Terra max 71.8%
  23. 23 Grok 4.5 high 70.5%
  24. 24 MiMo V2.6 Flash RL auto 70.4%
  25. 25 Muse Spark 1.3 xhigh 70.4%
  26. 26 GPT-5.5 xhigh 70.3%
  27. 27 Grok 4.7 xhigh 68.7%
  28. 28 GPT-6 Sol max 68.7%
  29. 29 Claude Haiku 5.5 max 68.3%
  30. 30 GPT-5.4 xhigh 68.2%
  31. 31 MiniMax-M3.1-Flash max 68.1%
  32. 32 DeepSeek-V4-Flash max 67.5%
  33. 33 Claude Sonnet 5 max 67.4%
  34. 34 GLM-5.3-Flash max 66.8%
  35. 35 GLM-5.3 max 66.2%
  36. 36 MiMo-V2.6 Pro auto 65.3%
  37. 37 GPT-5.6 Sol max 65.2%
  38. 38 Claude Opus 4.8 max 64.4%
  39. 39 Mistral Large 4 high 64.3%
  40. 40 Gemini 3.6 Flash high 63.4%
  41. 41 Claude Opus 4.7 max 63.1%
  42. 42 Mercury 3 high 62.9%
  43. 43 Beam max 62.5%
  44. 44 DeepSeek V4 Pro 0813 max 62.3%
  45. 45 Kimi K3 max 62.2%
  46. 46 Qwen 3.8 27B xhigh 62.2%
  47. 47 Claude Opus 4.6 max 61.8%
  48. 48 GLM-5.2 max 61.1%
  49. 49 Inkling 2 Small 60.9%
  50. 50 Muse Spark 1.2 xhigh 59.8%
  51. 51 GPT-6 Luna max 59%
  52. 52 GPT-5.6 Luna max 57.8%
  53. 53 Claude Sonnet 4.6 high 57.6%
  54. 54 GLM-5.1 56.8%
  55. 55 MiniMax-M3 high 53.5%
  56. 56 Kimi K2.7 Code high 53.4%
  57. 57 Gemini 3.1 Pro high 52.2%
  58. 58 DeepSeek V4.1 Flash max 50.5%
  59. 59 Inkling high 48.8%
  60. 60 Muse Spark Open xhigh 48.7%
  61. 61 Muse Spark 1.1 xhigh 46.2%
  62. 62 Gemini 3.5 Flash-Lite high 46.1%
  63. 63 Nemotron 3.5 Super VL high 45.9%
  64. 64 Gemini 3.5 Flash high 44.3%
  65. 65 Nemotron 3.5 Bolt high 43.9%
  66. 66 Qwen3.5 40.3%
  67. 67 Nemotron 3 Ultra high 38.3%
  68. 68 DeepSeek-V3.2 35.1%
  69. 69 gpt-oss-120b high 8.30%

Source: Official APEX-Agents 1.1 leaderboard

#EntryScoreSettingDateSource
1 Gemini 4 Carbon 90.4% high, Loop Oct 2026 Official leaderboard
2 Claude Fable 5.2 87.6% max, Loop Nov 2026 Official leaderboard
3 Gemini 4 Argon 87.4% high, Loop Sep 2026 Official leaderboard
4 Tencent Hy4 85.1% high, Loop Nov 2026 Official leaderboard
5 Gemini 4 Hydrogen 84% xhigh, Loop Nov 2026 Official leaderboard
6 Claude Sonnet 5.5 83.7% max, Loop Sep 2026 Official leaderboard
7 Claude Opus 5.5 81.3% max, Loop Sep 2026 Official leaderboard
8 Gemini 3.7 Flash 79.4% high, Loop Aug 2026 Official leaderboard
9 DeepSeek V4.1 Flash 1017 79.4% max, Loop Oct 2026 Official leaderboard
10 GPT-6.1 Terra 79.3% max, Loop Oct 2026 Official leaderboard
11 Grok 4.6 77.3% xhigh, Loop Aug 2026 Official leaderboard
12 Claude Fable 5.1 77.1% max, Loop Sep 2026 Official leaderboard
13 Claude Opus 5 77% max, Loop Jul 2026 Official leaderboard
14 Muse Wonder 77% max, Loop Oct 2026 Official leaderboard
15 Kimi K3.1 Pro 75.9% max, Loop Oct 2026 Official leaderboard
16 Muse Ember 75.7% max, Loop Nov 2026 Official leaderboard
17 GPT-6 Astra 75.1% max, Loop Sep 2026 Official leaderboard
18 Claude Fable 5 74.5% max, Loop Jun 2026 Official leaderboard
19 Qwen 3.8 Max 74.3% xhigh, Loop Aug 2026 Official leaderboard
20 GPT-6.1 Sol 73.2% max, Loop Sep 2026 Official leaderboard
21 Gemini 3.8 Flash 72.9% high, Loop Sep 2026 Official leaderboard
22 GPT-5.6 Terra 71.8% max, Loop Jul 2026 Official leaderboard
23 Grok 4.5 70.5% high, Loop Jul 2026 Official leaderboard
24 MiMo V2.6 Flash RL 70.4% auto, Loop Official leaderboard
25 Muse Spark 1.3 70.4% xhigh, Loop Sep 2026 Official leaderboard
26 GPT-5.5 70.3% xhigh, Loop Apr 2026 Official leaderboard
27 Grok 4.7 68.7% xhigh, Loop Sep 2026 Official leaderboard
28 GPT-6 Sol 68.7% max, Loop Sep 2026 Official leaderboard
29 Claude Haiku 5.5 68.3% max, Loop Oct 2026 Official leaderboard
30 GPT-5.4 68.2% xhigh, Loop Mar 2026 Official leaderboard
31 MiniMax-M3.1-Flash 68.1% max, Loop Oct 2026 Official leaderboard
32 DeepSeek-V4-Flash 67.5% max, Loop Apr 2026 Official leaderboard
33 Claude Sonnet 5 67.4% max, Loop Jun 2026 Official leaderboard
34 GLM-5.3-Flash 66.8% max, Loop Aug 2026 Official leaderboard
35 GLM-5.3 66.2% max, Loop Aug 2026 Official leaderboard
36 MiMo-V2.6 Pro 65.3% auto, Loop Sep 2026 Official leaderboard
37 GPT-5.6 Sol 65.2% max, Loop Jul 2026 Official leaderboard
38 Claude Opus 4.8 64.4% max, Loop May 2026 Official leaderboard
39 Mistral Large 4 64.3% high, Loop Oct 2026 Official leaderboard
40 Gemini 3.6 Flash 63.4% high, Loop Jul 2026 Official leaderboard
41 Claude Opus 4.7 63.1% max, Loop Apr 2026 Official leaderboard
42 Mercury 3 62.9% high, Loop Oct 2026 Official leaderboard
43 Beam 62.5% max, Loop Official leaderboard
44 DeepSeek V4 Pro 0813 62.3% max, Loop Aug 2026 Official leaderboard
45 Kimi K3 62.2% max, Loop Jul 2026 Official leaderboard
46 Qwen 3.8 27B 62.2% xhigh, Loop Aug 2026 Official leaderboard
47 Claude Opus 4.6 61.8% max, Loop Feb 2026 Official leaderboard
48 GLM-5.2 61.1% max, Loop Jun 2026 Official leaderboard
49 Inkling 2 Small 60.9% Loop Nov 2026 Official leaderboard
50 Muse Spark 1.2 59.8% xhigh, Loop Aug 2026 Official leaderboard
51 GPT-6 Luna 59% max, Loop Sep 2026 Official leaderboard
52 GPT-5.6 Luna 57.8% max, Loop Jul 2026 Official leaderboard
53 Claude Sonnet 4.6 57.6% high, Loop Feb 2026 Official leaderboard
54 GLM-5.1 56.8% Loop Apr 2026 Official leaderboard
55 MiniMax-M3 53.5% high, Loop Jun 2026 Official leaderboard
56 Kimi K2.7 Code 53.4% high, Loop Jun 2026 Official leaderboard
57 Gemini 3.1 Pro 52.2% high, Loop Feb 2026 Official leaderboard
58 DeepSeek V4.1 Flash 50.5% max, Loop Sep 2026 Official leaderboard
59 Inkling 48.8% high, Loop Jul 2026 Official leaderboard
60 Muse Spark Open 48.7% xhigh, Loop Oct 2026 Official leaderboard
61 Muse Spark 1.1 46.2% xhigh, Loop Jul 2026 Official leaderboard
62 Gemini 3.5 Flash-Lite 46.1% high, Loop Jul 2026 Official leaderboard
63 Nemotron 3.5 Super VL 45.9% high, Loop Oct 2026 Official leaderboard
64 Gemini 3.5 Flash 44.3% high, Loop May 2026 Official leaderboard
65 Nemotron 3.5 Bolt 43.9% high, Loop Official leaderboard
66 Qwen3.5 40.3% Loop Official leaderboard
67 Nemotron 3 Ultra 38.3% high, Loop Jun 2026 Official leaderboard
68 DeepSeek-V3.2 35.1% Loop Dec 2025 Official leaderboard
69 gpt-oss-120b 8.30% high, Loop Aug 2025 Official leaderboard

More effort adds a median 16 points

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Sonnet 5.5 Loop
medium max
83.7%
Claude Opus 5.5 Loop
medium max
81.3%
Claude Fable 5.1 Loop
high max
77.1%
Muse Spark 1.3 Loop
xhigh max
70.4%

Source: Official APEX-Agents 1.1 leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Gemini 4 Carbon high Loop 90.4%
Claude Fable 5.2 max Loop 87.6%
Gemini 4 Argon high Loop 87.4%
Tencent Hy4 high Loop 85.1%
Gemini 4 Hydrogen xhigh Loop 84%
Claude Sonnet 5.5 max Loop 83.7%
Claude Opus 5.5 max Loop 81.3%
Gemini 3.7 Flash high Loop 79.4%
DeepSeek V4.1 Flash 1017 max Loop 79.4%
GPT-6.1 Terra max Loop 79.3%
Grok 4.6 xhigh Loop 77.3%
Claude Fable 5.1 max Loop 77.1%
Claude Opus 5 max Loop 77%
Muse Wonder max Loop 77%
Kimi K3.1 Pro max Loop 75.9%
Muse Ember max Loop 75.7%
GPT-6 Astra max Loop 75.1%
Claude Fable 5 max Loop 74.5%
Qwen 3.8 Max xhigh Loop 74.3%
GPT-6.1 Sol max Loop 73.2%
Gemini 3.8 Flash high Loop 72.9%
GPT-5.6 Terra max Loop 71.8%
Claude Fable 5.1 high Loop 71.8%
Grok 4.5 high Loop 70.5%
Muse Spark 1.3 xhigh Loop 70.4%
GPT-5.5 xhigh Loop 70.3%
Grok 4.7 xhigh Loop 68.7%
GPT-6 Sol max Loop 68.7%
Claude Haiku 5.5 max Loop 68.3%
GPT-5.4 xhigh Loop 68.2%
MiniMax-M3.1-Flash max Loop 68.1%
DeepSeek-V4-Flash max Loop 67.5%
Claude Sonnet 5 max Loop 67.4%
GLM-5.3-Flash max Loop 66.8%
GLM-5.3 max Loop 66.2%
Claude Opus 5.5 medium Loop 65.7%
MiMo-V2.6 Pro auto Loop 65.3%
GPT-5.6 Sol max Loop 65.2%
Claude Opus 4.8 max Loop 64.4%
Mistral Large 4 high Loop 64.3%
Gemini 3.6 Flash high Loop 63.4%
Claude Opus 4.7 max Loop 63.1%
Mercury 3 high Loop 62.9%
DeepSeek V4 Pro 0813 max Loop 62.3%
Kimi K3 max Loop 62.2%
Qwen 3.8 27B xhigh Loop 62.2%
Claude Opus 4.6 max Loop 61.8%
Muse Spark 1.3 max Loop 61.3%
GLM-5.2 max Loop 61.1%
Inkling 2 Small Loop 60.9%
Muse Spark 1.2 xhigh Loop 59.8%
Claude Sonnet 5.5 medium Loop 59.4%
GPT-6 Luna max Loop 59%
GPT-5.6 Luna max Loop 57.8%
Claude Sonnet 4.6 high Loop 57.6%
GLM-5.1 Loop 56.8%
MiniMax-M3 high Loop 53.5%
Kimi K2.7 Code high Loop 53.4%
Gemini 3.1 Pro high Loop 52.2%
DeepSeek V4.1 Flash max Loop 50.5%
Inkling high Loop 48.8%
Muse Spark Open xhigh Loop 48.7%
Muse Spark 1.1 xhigh Loop 46.2%
Gemini 3.5 Flash-Lite high Loop 46.1%
Nemotron 3.5 Super VL high Loop 45.9%
Gemini 3.5 Flash high Loop 44.3%
Nemotron 3 Ultra high Loop 38.3%
DeepSeek-V3.2 Loop 35.1%
gpt-oss-120b high Loop 8.30%

5 models have held number one on the official leaderboard since its release in September 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Anthropic
Own leaderboard Passed 85% in Sep 2026, by Gemini 4 Argon
Gemini 4 Carbon 90.4%

Source: Official APEX-Agents 1.1 leaderboard

Gemini 4 Argon on the official leaderboard passed 85% in September 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Gemini 4 Carbon Claude Sonnet 5.5 Gemini 3.7 Flash Claude Fable 5 GPT-5.5 Claude Opus 4.6 DeepSeek-V3.2 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Gemini 4 Carbon Claude Sonnet 5.5

Source: Official APEX-Agents 1.1 leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Google DeepMind ByteDance Seed Moonshot AI xAI Anthropic
APEX-Agents 1.1 Passed 85% in Sep 2026, by Gemini 4 Argon
Gemini 4 Carbon 90.4%
APEX-Agents 5 leaders
Kimi K3.1 Pro 67.2%
APEX-Agents 1.1 Passed 85% in Sep 2026, by Gemini 4 Argon
Gemini 4 Carbon 90.4%

Sources: Official leaderboards and labs’ model cards

Replaces APEX-Agents

More in Economic work

All 97 in Economic work
  • JobBench In use 130 tasks from 35 occupations workers most want delegated. Agent works through messy reference files, graded by fact-anchored binary rubrics. GLM-5.3 61.4%
  • GDPval In use Real work tasks from 44 occupations with reference files. Experts blindly compare model deliverables with expert work. Metric is win-or-tie rate. Seed2.1 Pro 87.9%
  • GDPval-AA v2.1 In use Agents with shell and web access complete 220 GDPval gold tasks from 44 occupations. Blind pairwise comparisons give Elo ratings. Claude Opus 5.5 1,846
  • AA-Briefcase v1.1 In use Version 1.1 of Artificial Analysis's multi-hour office work evaluation, with Elo ratings fitted by Crowd-BT. Claude Opus 5.5 1,822
  • CoWorkBench In use Agent completes long-horizon productivity tasks across computer science, finance, law and medicine. Qwen in-house scoring (method unpublished). Claude Fable 5 75.9%
  • OfficeQA In use Grounded reasoning over a large corpus of U.S. Treasury Bulletin documents: locate tables and compute precise numeric answers. Claude Opus 4.7 86.3%