Benchmark Almanac

APEX-1

Gemini 4 Argon 81.1% leads since September 2026
Not in model cards

Models complete expert-written work cases with source documents for four jobs: investment banking, consulting, big law and primary care. A judge model grades each response against rubrics.

The leaderboard uses a hidden set of 400 cases, 100 per job, with 8 samples per model. Mercor open-sourced 100 in-distribution cases.

Made by
Mercor
Released
September 2025
Kind
Tasks from real jobs
Level
Expert
Scored
Graded by a model
Answers
Open-ended
Tasks
Public, plus a private test set
Strongest model tested
Gemini 4 Carbon #2 on the capability index
Last model added
November 2026
Scores refreshed
November 13th, 2026

In the authors’ words

We present an extended version of the AI Productivity Index (APEX-v1-extended), a benchmark for assessing whether frontier models are capable of performing economically valuable tasks in four jobs: investment banking associate, management consultant, big law associate, and primary care physician (MD). This technical report details the extensions to APEX-v1, including an increase in the held-out evaluation set from n = 50 to n = 100 cases per job (n = 400 total) and updates to the grading methodology. We present a new leaderboard, where GPT5 (Thinking = High) remains the top performing model with a score of 67.0%. APEX-v1-extended shows that frontier models still have substantial limitations when performing typical professional tasks. To support further research, we are open sourcing n = 25 non-benchmark example cases per role (n = 100 total) along with our evaluation harness.

Abstract of The AI Productivity Index (APEX) (arXiv:2509.25721, CC0)
75
Entries on its own leaderboard, 71 models we track

Gemini 4 Argon leads with 81.1%

Best score per entry from the benchmark's own leaderboard, read on November 13th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

75 entries. Scroll sideways for the rest.

  1. 81.1 Gemini 4 Argon
  2. 80.0 Gemini 4 Carbon
  3. 76.9 Gemini 4 Hydrogen
  4. 75.6 Muse Spark 1.2
  5. 75.1 Muse Wonder 1.0
  6. 73.9 Muse Spark 1.3
  7. 73.9 Muse Spark Open
  8. 73.8 Muse Spark 1.1
  9. 73.8 Claude Fable 5.1
  10. 73.1 GPT-5.6 Sol
  11. 70.4 Muse Ember
  12. 69.5 GPT-5.6 Terra
  13. 69.3 GPT-6 Astra
  14. 69.3 GPT-6.1 Sol
  15. 69.3 Kimi K3.1 Pro
  16. 68.8 Gemini 3.8 Flash
  17. 68.6 Claude Opus 5
  18. 68.4 GPT-5.5
  19. 68.4 Kimi K3
  20. 68.3 DeepSeek V4.1 Flash 1017
  21. 67.7 Grok 4.6
  22. 67.0 Gemini 3.7 Flash
  23. 66.6 GPT-5.6 Luna
  24. 66.0 Claude Fable 5
  25. 66.0 Gemini 3.6 Flash
  26. 66.0 DeepSeek V4 Pro 0813
  27. 65.9 GPT-5.3 Codex
  28. 65.8 Claude Opus 4.6
  29. 65.8 Claude Opus 5.5
  30. 65.6 GPT-5.4
  31. 65.5 DeepSeek V4.1 Flash
  32. 65.4 Grok 4.7
  33. 65.3 Qwen 3.8 Max
  34. 65.0 MiniMax-M3.1-Flash
  35. 64.7 GPT-5.2
  36. 64.7 Claude Haiku 5.5
  37. 64.2 GPT-6.1 Terra
  38. 63.8 DeepSeek-V4-Flash
  39. 63.6 GPT-6 Sol
  40. 63.0 GLM-5.3
  41. 62.7 Claude Opus 4.8
  42. 62.6 Qwen 3.8 27B
  43. 62.2 Claude Opus 4.7
  44. 62.2 Gemini 3.5 Flash
  45. 62.0 GPT-6 Luna
  46. 61.8 Gemini 3.1 Pro
  47. 61.8 Grok 4.5
  48. 61.6 Inkling
  49. 61.5 GLM-5.2
  50. 60.9 Gemini 3 Pro
  51. 60.9 Gemini 3 Flash
  52. 60.6 Beam
  53. 60.5 o3
  54. 60.1 Kimi K2.7 Code
  55. 59.8 MiniMax-M3
  56. 59.7 DeepSeek-V4-Pro
  57. 59.6 Mistral Large 4
  58. 58.7 GLM-5.3-Flash
  59. 57.6 Nemotron 3.5 Super VL
  60. 57.5 Grok 4.1
  61. 57.3 Claude Opus 4.5
  62. 57.1 Grok 4
  63. 56.1 Nemotron 3 Ultra
  64. 55.1 Claude Sonnet 4.6
  65. 54.7 Gemini 3.5 Flash-Lite
  66. 53.9 Claude Sonnet 5.5
  67. 52.7 Nemotron 3.5 Bolt
  68. 52.4 Kimi K2
  69. 52.4 GLM-5.1
  70. 51.5 DeepSeek-V3.2
  71. 50.3 Qwen3.5
  72. 49.6 Claude Sonnet 5
  73. 48.1 Gemma 4 31B IT
  74. 46.4 gpt-oss-120b
  75. 45.8 MiniMax-M2.7
  1. 1 Gemini 4 Argon high 81.1%
  2. 2 Gemini 4 Carbon high 80%
  3. 3 Gemini 4 Hydrogen xhigh 76.9%
  4. 4 Muse Spark 1.2 xhigh 75.6%
  5. 5 Muse Wonder 1.0 max 75.1%
  6. 6 Muse Spark 1.3 xhigh 73.9%
  7. 7 Muse Spark Open xhigh 73.9%
  8. 8 Muse Spark 1.1 xhigh 73.8%
  9. 9 Claude Fable 5.1 max 73.8%
  10. 10 GPT-5.6 Sol max 73.1%
  11. 11 Muse Ember max 70.4%
  12. 12 GPT-5.6 Terra max 69.5%
  13. 13 GPT-6 Astra xhigh 69.3%
  14. 14 GPT-6.1 Sol max 69.3%
  15. 15 Kimi K3.1 Pro max 69.3%
  16. 16 Gemini 3.8 Flash high 68.8%
  17. 17 Claude Opus 5 xhigh 68.6%
  18. 18 GPT-5.5 xhigh 68.4%
  19. 19 Kimi K3 max 68.4%
  20. 20 DeepSeek V4.1 Flash 1017 max 68.3%
  21. 21 Grok 4.6 xhigh 67.7%
  22. 22 Gemini 3.7 Flash high 67%
  23. 23 GPT-5.6 Luna max 66.6%
  24. 24 Claude Fable 5 max 66%
  25. 25 Gemini 3.6 Flash high 66%
  26. 26 DeepSeek V4 Pro 0813 max 66%
  27. 27 GPT-5.3 Codex high 65.9%
  28. 28 Claude Opus 4.6 max 65.8%
  29. 29 Claude Opus 5.5 max 65.8%
  30. 30 GPT-5.4 xhigh 65.6%
  31. 31 DeepSeek V4.1 Flash max 65.5%
  32. 32 Grok 4.7 xhigh 65.4%
  33. 33 Qwen 3.8 Max xhigh 65.3%
  34. 34 MiniMax-M3.1-Flash max 65%
  35. 35 GPT-5.2 high 64.7%
  36. 36 Claude Haiku 5.5 max 64.7%
  37. 37 GPT-6.1 Terra max 64.2%
  38. 38 DeepSeek-V4-Flash max 63.8%
  39. 39 GPT-6 Sol max 63.6%
  40. 40 GLM-5.3 max 63%
  41. 41 Claude Opus 4.8 max 62.7%
  42. 42 Qwen 3.8 27B xhigh 62.6%
  43. 43 Claude Opus 4.7 max 62.2%
  44. 44 Gemini 3.5 Flash high 62.2%
  45. 45 GPT-6 Luna max 62%
  46. 46 Gemini 3.1 Pro high 61.8%
  47. 47 Grok 4.5 high 61.8%
  48. 48 Inkling high 61.6%
  49. 49 GLM-5.2 max 61.5%
  50. 50 Gemini 3 Pro high 60.9%
  51. 51 Gemini 3 Flash high 60.9%
  52. 52 Beam max 60.6%
  53. 53 o3 high 60.5%
  54. 54 Kimi K2.7 Code high 60.1%
  55. 55 MiniMax-M3 high 59.8%
  56. 56 DeepSeek-V4-Pro max 59.7%
  57. 57 Mistral Large 4 high 59.6%
  58. 58 GLM-5.3-Flash max 58.7%
  59. 59 Nemotron 3.5 Super VL high 57.6%
  60. 60 Grok 4.1 fast 57.5%
  61. 61 Claude Opus 4.5 high 57.3%
  62. 62 Grok 4 57.1%
  63. 63 Nemotron 3 Ultra high 56.1%
  64. 64 Claude Sonnet 4.6 high 55.1%
  65. 65 Gemini 3.5 Flash-Lite high 54.7%
  66. 66 Claude Sonnet 5.5 max 53.9%
  67. 67 Nemotron 3.5 Bolt high 52.7%
  68. 68 Kimi K2 Sep 2025, high 52.4%
  69. 69 GLM-5.1 52.4%
  70. 70 DeepSeek-V3.2 51.5%
  71. 71 Qwen3.5 50.3%
  72. 72 Claude Sonnet 5 max 49.6%
  73. 73 Gemma 4 31B IT 48.1%
  74. 74 gpt-oss-120b high 46.4%
  75. 75 MiniMax-M2.7 high 45.8%

Source: Official APEX-1 leaderboard

#EntryScoreSettingDateSource
1 Gemini 4 Argon 81.1% high, singleshot_agent Sep 2026 Official leaderboard
2 Gemini 4 Carbon 80% high, singleshot_agent Oct 2026 Official leaderboard
3 Gemini 4 Hydrogen 76.9% xhigh, singleshot_agent Nov 2026 Official leaderboard
4 Muse Spark 1.2 75.6% xhigh, singleshot_agent Aug 2026 Official leaderboard
5 Muse Wonder 1.0 75.1% max, singleshot_agent Official leaderboard
6 Muse Spark 1.3 73.9% xhigh, singleshot_agent Sep 2026 Official leaderboard
7 Muse Spark Open 73.9% xhigh, singleshot_agent Oct 2026 Official leaderboard
8 Muse Spark 1.1 73.8% xhigh, singleshot_agent Jul 2026 Official leaderboard
9 Claude Fable 5.1 73.8% max, singleshot_agent Sep 2026 Official leaderboard
10 GPT-5.6 Sol 73.1% max, singleshot_agent Jul 2026 Official leaderboard
11 Muse Ember 70.4% max, singleshot_agent Nov 2026 Official leaderboard
12 GPT-5.6 Terra 69.5% max, singleshot_agent Jul 2026 Official leaderboard
13 GPT-6 Astra 69.3% xhigh, singleshot_agent Sep 2026 Official leaderboard
14 GPT-6.1 Sol 69.3% max, singleshot_agent Sep 2026 Official leaderboard
15 Kimi K3.1 Pro 69.3% max, singleshot_agent Oct 2026 Official leaderboard
16 Gemini 3.8 Flash 68.8% high, singleshot_agent Sep 2026 Official leaderboard
17 Claude Opus 5 68.6% xhigh, singleshot_agent Jul 2026 Official leaderboard
18 GPT-5.5 68.4% xhigh, singleshot_agent Apr 2026 Official leaderboard
19 Kimi K3 68.4% max, singleshot_agent Jul 2026 Official leaderboard
20 DeepSeek V4.1 Flash 1017 68.3% max, singleshot_agent Oct 2026 Official leaderboard
21 Grok 4.6 67.7% xhigh, singleshot_agent Aug 2026 Official leaderboard
22 Gemini 3.7 Flash 67% high, singleshot_agent Aug 2026 Official leaderboard
23 GPT-5.6 Luna 66.6% max, singleshot_agent Jul 2026 Official leaderboard
24 Claude Fable 5 66% max, singleshot_agent Jun 2026 Official leaderboard
25 Gemini 3.6 Flash 66% high, singleshot_agent Jul 2026 Official leaderboard
26 DeepSeek V4 Pro 0813 66% max, singleshot_agent Aug 2026 Official leaderboard
27 GPT-5.3 Codex 65.9% high, singleshot_agent Feb 2026 Official leaderboard
28 Claude Opus 4.6 65.8% max, singleshot_agent Feb 2026 Official leaderboard
29 Claude Opus 5.5 65.8% max, singleshot_agent Sep 2026 Official leaderboard
30 GPT-5.4 65.6% xhigh, singleshot_agent Mar 2026 Official leaderboard
31 DeepSeek V4.1 Flash 65.5% max, singleshot_agent Sep 2026 Official leaderboard
32 Grok 4.7 65.4% xhigh, singleshot_agent Sep 2026 Official leaderboard
33 Qwen 3.8 Max 65.3% xhigh, singleshot_agent Aug 2026 Official leaderboard
34 MiniMax-M3.1-Flash 65% max, singleshot_agent Oct 2026 Official leaderboard
35 GPT-5.2 64.7% high, singleshot_agent Dec 2025 Official leaderboard
36 Claude Haiku 5.5 64.7% max, singleshot_agent Oct 2026 Official leaderboard
37 GPT-6.1 Terra 64.2% max, singleshot_agent Oct 2026 Official leaderboard
38 DeepSeek-V4-Flash 63.8% max, singleshot_agent Apr 2026 Official leaderboard
39 GPT-6 Sol 63.6% max, singleshot_agent Sep 2026 Official leaderboard
40 GLM-5.3 63% max, singleshot_agent Aug 2026 Official leaderboard
41 Claude Opus 4.8 62.7% max, singleshot_agent May 2026 Official leaderboard
42 Qwen 3.8 27B 62.6% xhigh, singleshot_agent Aug 2026 Official leaderboard
43 Claude Opus 4.7 62.2% max, singleshot_agent Apr 2026 Official leaderboard
44 Gemini 3.5 Flash 62.2% high, singleshot_agent May 2026 Official leaderboard
45 GPT-6 Luna 62% max, singleshot_agent Sep 2026 Official leaderboard
46 Gemini 3.1 Pro 61.8% high, singleshot_agent Feb 2026 Official leaderboard
47 Grok 4.5 61.8% high, singleshot_agent Jul 2026 Official leaderboard
48 Inkling 61.6% high, singleshot_agent Jul 2026 Official leaderboard
49 GLM-5.2 61.5% max, singleshot_agent Jun 2026 Official leaderboard
50 Gemini 3 Pro 60.9% high, singleshot_agent Nov 2025 Official leaderboard
51 Gemini 3 Flash 60.9% high, singleshot_agent Dec 2025 Official leaderboard
52 Beam 60.6% max, singleshot_agent Official leaderboard
53 o3 60.5% high, singleshot_agent Apr 2025 Official leaderboard
54 Kimi K2.7 Code 60.1% high, singleshot_agent Jun 2026 Official leaderboard
55 MiniMax-M3 59.8% high, singleshot_agent Jun 2026 Official leaderboard
56 DeepSeek-V4-Pro 59.7% max, singleshot_agent Apr 2026 Official leaderboard
57 Mistral Large 4 59.6% high, singleshot_agent Oct 2026 Official leaderboard
58 GLM-5.3-Flash 58.7% max, singleshot_agent Aug 2026 Official leaderboard
59 Nemotron 3.5 Super VL 57.6% high, singleshot_agent Oct 2026 Official leaderboard
60 Grok 4.1 57.5% fast, singleshot_agent Nov 2025 Official leaderboard
61 Claude Opus 4.5 57.3% high, singleshot_agent Nov 2025 Official leaderboard
62 Grok 4 57.1% singleshot_agent Jul 2025 Official leaderboard
63 Nemotron 3 Ultra 56.1% high, singleshot_agent Jun 2026 Official leaderboard
64 Claude Sonnet 4.6 55.1% high, singleshot_agent Feb 2026 Official leaderboard
65 Gemini 3.5 Flash-Lite 54.7% high, singleshot_agent Jul 2026 Official leaderboard
66 Claude Sonnet 5.5 53.9% max, singleshot_agent Sep 2026 Official leaderboard
67 Nemotron 3.5 Bolt 52.7% high, singleshot_agent Official leaderboard
68 Kimi K2 Sep 2025 52.4% high, singleshot_agent Sep 2025 Official leaderboard
69 GLM-5.1 52.4% singleshot_agent Apr 2026 Official leaderboard
70 DeepSeek-V3.2 51.5% singleshot_agent Dec 2025 Official leaderboard
71 Qwen3.5 50.3% singleshot_agent Official leaderboard
72 Claude Sonnet 5 49.6% max, singleshot_agent Jun 2026 Official leaderboard
73 Gemma 4 31B IT 48.1% singleshot_agent Apr 2026 Official leaderboard
74 gpt-oss-120b 46.4% high, singleshot_agent Aug 2025 Official leaderboard
75 MiniMax-M2.7 45.8% high, singleshot_agent Mar 2026 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Muse Spark 1.3 singleshot_agent
xhigh max
73.9%
Claude Opus 4.6 singleshot_agent
high
65.8%
GPT-5.4 singleshot_agent
high xhigh
65.6%

Source: Official APEX-1 leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Gemini 4 Argon high singleshot_agent 81.1%
Gemini 4 Carbon high singleshot_agent 80%
Gemini 4 Hydrogen xhigh singleshot_agent 76.9%
Muse Spark 1.2 xhigh singleshot_agent 75.6%
Muse Spark 1.3 xhigh singleshot_agent 73.9%
Muse Spark Open xhigh singleshot_agent 73.9%
Muse Spark 1.1 xhigh singleshot_agent 73.8%
Claude Fable 5.1 max singleshot_agent 73.8%
GPT-5.6 Sol max singleshot_agent 73.1%
Muse Spark 1.3 max singleshot_agent 73%
Muse Ember max singleshot_agent 70.4%
GPT-5.6 Terra max singleshot_agent 69.5%
GPT-6 Astra xhigh singleshot_agent 69.3%
GPT-6.1 Sol max singleshot_agent 69.3%
Kimi K3.1 Pro max singleshot_agent 69.3%
Gemini 3.8 Flash high singleshot_agent 68.8%
Claude Opus 5 xhigh singleshot_agent 68.6%
GPT-5.5 xhigh singleshot_agent 68.4%
Kimi K3 max singleshot_agent 68.4%
DeepSeek V4.1 Flash 1017 max singleshot_agent 68.3%
Grok 4.6 xhigh singleshot_agent 67.7%
Gemini 3.7 Flash high singleshot_agent 67%
GPT-5.6 Luna max singleshot_agent 66.6%
Claude Fable 5 max singleshot_agent 66%
Gemini 3.6 Flash high singleshot_agent 66%
DeepSeek V4 Pro 0813 max singleshot_agent 66%
GPT-5.3 Codex high singleshot_agent 65.9%
Claude Opus 4.6 max singleshot_agent 65.8%
Claude Opus 5.5 max singleshot_agent 65.8%
GPT-5.4 xhigh singleshot_agent 65.6%
DeepSeek V4.1 Flash max singleshot_agent 65.5%
Grok 4.7 xhigh singleshot_agent 65.4%
Claude Opus 4.6 high singleshot_agent 65.3%
Qwen 3.8 Max xhigh singleshot_agent 65.3%
MiniMax-M3.1-Flash max singleshot_agent 65%
GPT-5.2 high singleshot_agent 64.7%
Claude Haiku 5.5 max singleshot_agent 64.7%
GPT-6.1 Terra max singleshot_agent 64.2%
DeepSeek-V4-Flash max singleshot_agent 63.8%
GPT-6 Sol max singleshot_agent 63.6%
GPT-5.4 high singleshot_agent 63.2%
GLM-5.3 max singleshot_agent 63%
Claude Opus 4.8 max singleshot_agent 62.7%
Qwen 3.8 27B xhigh singleshot_agent 62.6%
Claude Opus 4.7 max singleshot_agent 62.2%
Gemini 3.5 Flash high singleshot_agent 62.2%
GPT-6 Luna max singleshot_agent 62%
Gemini 3.1 Pro high singleshot_agent 61.8%
Grok 4.5 high singleshot_agent 61.8%
Inkling high singleshot_agent 61.6%
GLM-5.2 max singleshot_agent 61.5%
Gemini 3 Pro high singleshot_agent 60.9%
Gemini 3 Flash high singleshot_agent 60.9%
o3 high singleshot_agent 60.5%
Kimi K2.7 Code high singleshot_agent 60.1%
MiniMax-M3 high singleshot_agent 59.8%
DeepSeek-V4-Pro max singleshot_agent 59.7%
Mistral Large 4 high singleshot_agent 59.6%
GLM-5.3-Flash max singleshot_agent 58.7%
Nemotron 3.5 Super VL high singleshot_agent 57.6%
Grok 4.1 fast singleshot_agent 57.5%
Claude Opus 4.5 high singleshot_agent 57.3%
Grok 4 singleshot_agent 57.1%
Nemotron 3 Ultra high singleshot_agent 56.1%
Claude Sonnet 4.6 high singleshot_agent 55.1%
Gemini 3.5 Flash-Lite high singleshot_agent 54.7%
Claude Sonnet 5.5 max singleshot_agent 53.9%
Kimi K2 Sep 2025 high singleshot_agent 52.4%
GLM-5.1 singleshot_agent 52.4%
DeepSeek-V3.2 singleshot_agent 51.5%
Claude Sonnet 5 max singleshot_agent 49.6%
Gemma 4 31B IT singleshot_agent 48.1%
gpt-oss-120b high singleshot_agent 46.4%
MiniMax-M2.7 high singleshot_agent 45.8%

8 models have held number one on the official leaderboard since its release in September 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Google DeepMind Meta
Own leaderboard 8 leaders
Gemini 4 Argon 81.1%

Source: Official APEX-1 leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Gemini 4 Argon Muse Spark 1.2 GPT-5.5 GPT-5.2 o3 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Gemini 4 Argon Muse Spark 1.1

Source: Official APEX-1 leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Economic work

All 97 in Economic work
  • JobBench In use 130 tasks from 35 occupations workers most want delegated. Agent works through messy reference files, graded by fact-anchored binary rubrics. GLM-5.3 61.4%
  • GDPval In use Real work tasks from 44 occupations with reference files. Experts blindly compare model deliverables with expert work. Metric is win-or-tie rate. Seed2.1 Pro 87.9%
  • GDPval-AA v2.1 In use Agents with shell and web access complete 220 GDPval gold tasks from 44 occupations. Blind pairwise comparisons give Elo ratings. Claude Opus 5.5 1,846
  • AA-Briefcase v1.1 In use Version 1.1 of Artificial Analysis's multi-hour office work evaluation, with Elo ratings fitted by Crowd-BT. Claude Opus 5.5 1,822
  • CoWorkBench In use Agent completes long-horizon productivity tasks across computer science, finance, law and medicine. Qwen in-house scoring (method unpublished). Claude Fable 5 75.9%
  • OfficeQA In use Grounded reasoning over a large corpus of U.S. Treasury Bulletin documents: locate tables and compute precise numeric answers. Claude Opus 4.7 86.3%