Benchmark Almanac

SAGE (Vals)

Claude Opus 4.7 56.1% leads since April 2026
Not in model cards

Models grade handwritten undergraduate work in PDEs, linear algebra and computer science against rubrics. Scores compare their deductions with expert graders.

v1 scored the share of submissions where every rubric check was right, which rewarded graders that never deduct. Board frozen October 8th, 2026 with 56 models.

Made by
Vals AI
Released
October 2025
Kind
Other
Level
University
Scored
Graded by people
Answers
Open-ended
Tasks
Kept private
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
81
Models scored on its own leaderboard

Claude Opus 4.7 leads with 56.1%

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

81 models. Scroll sideways for the rest.

  1. 56.1 Claude Opus 4.7
  2. 55.0 Gemma 4 31B IT
  3. 54.8 Claude Opus 4.8
  4. 53.6 Gemini 4 Argon
  5. 52.8 Kimi K3
  6. 52.6 GPT-5.6 Sol
  7. 52.4 Qwen 3.8 27B
  8. 52.1 Claude Opus 4.5
  9. 51.9 Claude Fable 5
  10. 51.9 Gemini 3 Flash
  11. 51.8 Claude Sonnet 5.5
  12. 51.6 Claude Opus 4.6
  13. 51.5 GPT-5.5
  14. 51.3 Qwen 3.8 Max
  15. 50.8 GPT-5.4 Mini
  16. 50.6 MiniMax-M3
  17. 50.2 Kimi K2.6
  18. 49.9 Gemini 3.5 Flash
  19. 49.9 Kimi K2.5
  20. 49.5 Gemini 3.1 Flash-Lite
  21. 49.4 Claude Opus 5
  22. 49.3 GPT-5.2
  23. 49.2 Gemini 3.7 Flash
  24. 48.9 Claude Sonnet 5
  25. 48.7 Gemini 3.6 Flash
  26. 48.7 Gemini 3.1 Pro
  27. 48.5 Claude Fable 5.1
  28. 48.1 GPT-6 Luna
  29. 47.9 DeepSeek V4.1 Flash
  30. 47.7 Muse Spark 1.2
  31. 47.6 Gemini 3 Pro
  32. 47.3 Gemini 3.5 Flash-Lite
  33. 47.0 GPT-5.6 Terra
  34. 46.6 Claude Sonnet 4.6
  35. 46.5 GPT-6.1 Sol
  36. 46.4 GPT-6 Astra
  37. 45.8 Claude Opus 5.5
  38. 45.8 Claude Haiku 5.5
  39. 45.6 Qwen3.6 27B
  40. 45.6 Muse Spark 1.1
  41. 45.0 MiMo-V2.6 Pro
  42. 44.9 Qwen 3.6 Plus
  43. 44.8 GPT-6 Sol
  44. 44.8 Gemini 2.5 Flash
  45. 44.2 GPT-5.6 Luna
  46. 44.1 Gemini 2.5 Flash
  47. 43.7 GPT-5
  48. 43.5 MiMo-V2.6 Flash
  49. 43.3 GPT-5.4
  50. 43.3 MiMo-V2.5
  51. 43.2 GPT-5.1
  52. 43.0 GPT-5 mini
  53. 42.5 Qwen 3.5 Flash (hosted 35B-A3B)
  54. 42.1 Qwen3-VL-Plus
  55. 41.9 Gemini 2.5 Pro
  56. 41.8 o3
  57. 41.1 o4-mini
  58. 40.8 Grok 4.7
  59. 39.3 Qwen3.7-Plus
  60. 38.2 Grok 4.20
  61. 38.1 GPT-5.4 Nano
  62. 37.6 Mistral Medium 3.5
  63. 36.5 Inkling
  64. 36.1 Claude Sonnet 4.5
  65. 35.7 Llama 4 Maverick
  66. 35.1 Gemini 3.8 Flash
  67. 35.0 Claude Sonnet 4
  68. 35.0 Grok 4.5
  69. 34.8 Llama 4 Scout
  70. 32.7 Inkling-Small
  71. 31.8 Claude Haiku 4.5
  72. 31.4 Claude Opus 4.1
  73. 31.3 Grok 4.1 Fast
  74. 30.8 Gemini 2.5 Flash-Lite
  75. 30.4 Qwen 3.5 Plus (hosted 397B-A17B)
  76. 30.4 GPT-5 nano
  77. 29.8 Grok 4 Fast
  78. 28.9 Grok 4.6
  79. 25.1 Grok 4
  80. 24.6 Mistral Large 3
  81. 19.7 Grok 4.3
  1. 1 Claude Opus 4.7 56.1%
  2. 2 Gemma 4 31B IT 55%
  3. 3 Claude Opus 4.8 54.8%
  4. 4 Gemini 4 Argon 53.6%
  5. 5 Kimi K3 52.8%
  6. 6 GPT-5.6 Sol 52.6%
  7. 7 Qwen 3.8 27B 52.4%
  8. 8 Claude Opus 4.5 thinking 52.1%
  9. 9 Claude Fable 5 51.9%
  10. 10 Gemini 3 Flash 51.9%
  11. 11 Claude Sonnet 5.5 51.8%
  12. 12 Claude Opus 4.6 thinking 51.6%
  13. 13 GPT-5.5 51.5%
  14. 14 Qwen 3.8 Max 51.3%
  15. 15 GPT-5.4 Mini 50.8%
  16. 16 MiniMax-M3 50.6%
  17. 17 Kimi K2.6 50.2%
  18. 18 Gemini 3.5 Flash 49.9%
  19. 19 Kimi K2.5 thinking 49.9%
  20. 20 Gemini 3.1 Flash-Lite 49.5%
  21. 21 Claude Opus 5 49.4%
  22. 22 GPT-5.2 49.3%
  23. 23 Gemini 3.7 Flash 49.2%
  24. 24 Claude Sonnet 5 48.9%
  25. 25 Gemini 3.6 Flash 48.7%
  26. 26 Gemini 3.1 Pro 48.7%
  27. 27 Claude Fable 5.1 48.5%
  28. 28 GPT-6 Luna 48.1%
  29. 29 DeepSeek V4.1 Flash 47.9%
  30. 30 Muse Spark 1.2 47.7%
  31. 31 Gemini 3 Pro 47.6%
  32. 32 Gemini 3.5 Flash-Lite 47.3%
  33. 33 GPT-5.6 Terra 47%
  34. 34 Claude Sonnet 4.6 46.6%
  35. 35 GPT-6.1 Sol 46.5%
  36. 36 GPT-6 Astra 46.4%
  37. 37 Claude Opus 5.5 45.8%
  38. 38 Claude Haiku 5.5 45.8%
  39. 39 Qwen3.6 27B 45.6%
  40. 40 Muse Spark 1.1 45.6%
  41. 41 MiMo-V2.6 Pro 45%
  42. 42 Qwen 3.6 Plus 44.9%
  43. 43 GPT-6 Sol 44.8%
  44. 44 Gemini 2.5 Flash Jun 2025, non-thinking 44.8%
  45. 45 GPT-5.6 Luna 44.2%
  46. 46 Gemini 2.5 Flash Sep 2025, non-thinking 44.1%
  47. 47 GPT-5 43.7%
  48. 48 MiMo-V2.6 Flash 43.5%
  49. 49 GPT-5.4 xhigh 43.3%
  50. 50 MiMo-V2.5 43.3%
  51. 51 GPT-5.1 43.2%
  52. 52 GPT-5 mini 43%
  53. 53 Qwen 3.5 Flash (hosted 35B-A3B) 42.5%
  54. 54 Qwen3-VL-Plus 42.1%
  55. 55 Gemini 2.5 Pro Jun 2025 41.9%
  56. 56 o3 41.8%
  57. 57 o4-mini 41.1%
  58. 58 Grok 4.7 40.8%
  59. 59 Qwen3.7-Plus 39.3%
  60. 60 Grok 4.20 reasoning 38.2%
  61. 61 GPT-5.4 Nano 38.1%
  62. 62 Mistral Medium 3.5 37.6%
  63. 63 Inkling 36.5%
  64. 64 Claude Sonnet 4.5 thinking 36.1%
  65. 65 Llama 4 Maverick 35.7%
  66. 66 Gemini 3.8 Flash 35.1%
  67. 67 Claude Sonnet 4 non-thinking 35%
  68. 68 Grok 4.5 35%
  69. 69 Llama 4 Scout 34.8%
  70. 70 Inkling-Small 32.7%
  71. 71 Claude Haiku 4.5 thinking 31.8%
  72. 72 Claude Opus 4.1 non-thinking 31.4%
  73. 73 Grok 4.1 Fast reasoning 31.3%
  74. 74 Gemini 2.5 Flash-Lite Sep 2025, thinking 30.8%
  75. 75 Qwen 3.5 Plus (hosted 397B-A17B) thinking 30.4%
  76. 76 GPT-5 nano 30.4%
  77. 77 Grok 4 Fast reasoning 29.8%
  78. 78 Grok 4.6 28.9%
  79. 79 Grok 4 25.1%
  80. 80 Mistral Large 3 24.6%
  81. 81 Grok 4.3 19.7%

Source: Official SAGE (Vals) leaderboard

#ModelScoreSettingDateSource
1 Claude Opus 4.7 56.1% Apr 2026 Official leaderboard
2 Gemma 4 31B IT 55% Apr 2026 Official leaderboard
3 Claude Opus 4.8 54.8% May 2026 Official leaderboard
4 Gemini 4 Argon 53.6% Sep 2026 Official leaderboard
5 Kimi K3 52.8% Jul 2026 Official leaderboard
6 GPT-5.6 Sol 52.6% Jul 2026 Official leaderboard
7 Qwen 3.8 27B 52.4% Aug 2026 Official leaderboard
8 Claude Opus 4.5 52.1% thinking Nov 2025 Official leaderboard
9 Claude Fable 5 51.9% Jun 2026 Official leaderboard
10 Gemini 3 Flash 51.9% Dec 2025 Official leaderboard
11 Claude Sonnet 5.5 51.8% Sep 2026 Official leaderboard
12 Claude Opus 4.6 51.6% thinking Feb 2026 Official leaderboard
13 GPT-5.5 51.5% Apr 2026 Official leaderboard
14 Qwen 3.8 Max 51.3% Aug 2026 Official leaderboard
15 GPT-5.4 Mini 50.8% Mar 2026 Official leaderboard
16 MiniMax-M3 50.6% Jun 2026 Official leaderboard
17 Kimi K2.6 50.2% Apr 2026 Official leaderboard
18 Gemini 3.5 Flash 49.9% May 2026 Official leaderboard
19 Kimi K2.5 49.9% thinking Jan 2026 Official leaderboard
20 Gemini 3.1 Flash-Lite 49.5% Mar 2026 Official leaderboard
21 Claude Opus 5 49.4% Jul 2026 Official leaderboard
22 GPT-5.2 49.3% Dec 2025 Official leaderboard
23 Gemini 3.7 Flash 49.2% Aug 2026 Official leaderboard
24 Claude Sonnet 5 48.9% Jun 2026 Official leaderboard
25 Gemini 3.6 Flash 48.7% Jul 2026 Official leaderboard
26 Gemini 3.1 Pro 48.7% Feb 2026 Official leaderboard
27 Claude Fable 5.1 48.5% Sep 2026 Official leaderboard
28 GPT-6 Luna 48.1% Sep 2026 Official leaderboard
29 DeepSeek V4.1 Flash 47.9% Sep 2026 Official leaderboard
30 Muse Spark 1.2 47.7% Aug 2026 Official leaderboard
31 Gemini 3 Pro 47.6% Nov 2025 Official leaderboard
32 Gemini 3.5 Flash-Lite 47.3% Jul 2026 Official leaderboard
33 GPT-5.6 Terra 47% Jul 2026 Official leaderboard
34 Claude Sonnet 4.6 46.6% Feb 2026 Official leaderboard
35 GPT-6.1 Sol 46.5% Sep 2026 Official leaderboard
36 GPT-6 Astra 46.4% Sep 2026 Official leaderboard
37 Claude Opus 5.5 45.8% Sep 2026 Official leaderboard
38 Claude Haiku 5.5 45.8% Oct 2026 Official leaderboard
39 Qwen3.6 27B 45.6% Apr 2026 Official leaderboard
40 Muse Spark 1.1 45.6% Jul 2026 Official leaderboard
41 MiMo-V2.6 Pro 45% Sep 2026 Official leaderboard
42 Qwen 3.6 Plus 44.9% Mar 2026 Official leaderboard
43 GPT-6 Sol 44.8% Sep 2026 Official leaderboard
44 Gemini 2.5 Flash Jun 2025 44.8% non-thinking Jun 2025 Official leaderboard
45 GPT-5.6 Luna 44.2% Jul 2026 Official leaderboard
46 Gemini 2.5 Flash Sep 2025 44.1% non-thinking Sep 2025 Official leaderboard
47 GPT-5 43.7% Aug 2025 Official leaderboard
48 MiMo-V2.6 Flash 43.5% Sep 2026 Official leaderboard
49 GPT-5.4 43.3% xhigh Mar 2026 Official leaderboard
50 MiMo-V2.5 43.3% Apr 2026 Official leaderboard
51 GPT-5.1 43.2% Nov 2025 Official leaderboard
52 GPT-5 mini 43% Aug 2025 Official leaderboard
53 Qwen 3.5 Flash (hosted 35B-A3B) 42.5% Feb 2026 Official leaderboard
54 Qwen3-VL-Plus 42.1% Sep 2025 Official leaderboard
55 Gemini 2.5 Pro Jun 2025 41.9% Jun 2025 Official leaderboard
56 o3 41.8% Apr 2025 Official leaderboard
57 o4-mini 41.1% Apr 2025 Official leaderboard
58 Grok 4.7 40.8% Sep 2026 Official leaderboard
59 Qwen3.7-Plus 39.3% Jun 2026 Official leaderboard
60 Grok 4.20 38.2% reasoning Feb 2026 Official leaderboard
61 GPT-5.4 Nano 38.1% Mar 2026 Official leaderboard
62 Mistral Medium 3.5 37.6% Apr 2026 Official leaderboard
63 Inkling 36.5% Jul 2026 Official leaderboard
64 Claude Sonnet 4.5 36.1% thinking Sep 2025 Official leaderboard
65 Llama 4 Maverick 35.7% Apr 2025 Official leaderboard
66 Gemini 3.8 Flash 35.1% Sep 2026 Official leaderboard
67 Claude Sonnet 4 35% non-thinking May 2025 Official leaderboard
68 Grok 4.5 35% Jul 2026 Official leaderboard
69 Llama 4 Scout 34.8% Apr 2025 Official leaderboard
70 Inkling-Small 32.7% Jul 2026 Official leaderboard
71 Claude Haiku 4.5 31.8% thinking Oct 2025 Official leaderboard
72 Claude Opus 4.1 31.4% non-thinking Aug 2025 Official leaderboard
73 Grok 4.1 Fast 31.3% reasoning Nov 2025 Official leaderboard
74 Gemini 2.5 Flash-Lite Sep 2025 30.8% thinking Sep 2025 Official leaderboard
75 Qwen 3.5 Plus (hosted 397B-A17B) 30.4% thinking Feb 2026 Official leaderboard
76 GPT-5 nano 30.4% Aug 2025 Official leaderboard
77 Grok 4 Fast 29.8% reasoning Sep 2025 Official leaderboard
78 Grok 4.6 28.9% Aug 2026 Official leaderboard
79 Grok 4 25.1% Jul 2025 Official leaderboard
80 Mistral Large 3 24.6% Dec 2025 Official leaderboard
81 Grok 4.3 19.7% Apr 2026 Official leaderboard

The best score, 56.1%, costs $0.33 a task

Runs on the cost frontier score higher than any run that's cheaper. Gemini 3 Flash comes within 4.3 points of the best for $0.02 a task, 18 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Google DeepMind xAI Anthropic Qwen OpenAI Cost frontier One model at each effort level
0% 20% 40% 60% $0.0001 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Opus 4.7 Gemini 4 Argon Qwen 3.8 27B Gemini 3.1 Flash-Lite Gemini 2.5 Flash, non-thinking Grok 4.1 Fast, reasoning Gemini 2.5 Flash-Lite, non-thinking 0% 20% 40% 60% $0.0001 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Opus 4.7 Qwen 3.8 27B

Source: Official SAGE (Vals) leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.7 56.1% $0.33
Gemma 4 31B IT 55%
Claude Opus 4.8 54.8% $0.51
Gemini 4 Argon 53.6% $0.10
Kimi K3 52.8% $0.16
GPT-5.6 Sol 52.6% $0.15
Qwen 3.8 27B 52.4% $0.04
Claude Opus 4.5 thinking 52.1% $0.15
Claude Fable 5 51.9% $0.65
Gemini 3 Flash 51.9% $0.02
Claude Sonnet 5.5 51.8% $0.18
Claude Opus 4.6 thinking 51.6% $0.35
GPT-5.5 51.5% $0.14
Qwen 3.8 Max 51.3% $0.07
GPT-5.4 Mini 50.8% $0.08
MiniMax-M3 50.6% $0.03
Kimi K2.6 50.2% $0.07
Gemini 3.5 Flash 49.9% $0.05
Kimi K2.5 thinking 49.9% $0.03
Gemini 3.1 Flash-Lite 49.5% $0.007
Claude Opus 5 49.4% $0.19
GPT-5.2 49.3% $0.12
Gemini 3.7 Flash 49.2% $0.03
Claude Sonnet 5 48.9% $0.59
Gemini 3.6 Flash 48.7% $0.05
Gemini 3.1 Pro 48.7% $0.07
Claude Fable 5.1 48.5% $0.61
GPT-6 Luna 48.1% $0.007
DeepSeek V4.1 Flash 47.9% $0.01
Muse Spark 1.2 47.7% $0.03
Gemini 3 Pro 47.6% $0.01
Gemini 3.5 Flash-Lite 47.3% $0.01
GPT-5.6 Terra 47% $0.04
Claude Sonnet 4.6 46.6% $0.23
GPT-6.1 Sol 46.5% $0.06
GPT-6 Astra 46.4% $0.28
Claude Opus 5.5 45.8% $0.33
Claude Haiku 5.5 45.8% $0.02
Qwen3.6 27B 45.6% $0.05
Muse Spark 1.1 45.6%
MiMo-V2.6 Pro 45% $0.02
Claude Opus 4.5 non-thinking 45% $0.02
Qwen 3.6 Plus 44.9% $0.04
GPT-6 Sol 44.8% $0.06
Gemini 2.5 Flash Jun 2025 non-thinking 44.8% $0.003
GPT-5.6 Luna 44.2% $0.01
Gemini 2.5 Flash Sep 2025 non-thinking 44.1% $0.002
GPT-5 43.7% $0.03
MiMo-V2.6 Flash 43.5% $0.005
GPT-5.4 xhigh 43.3% $0.10
MiMo-V2.5 43.3% $0.004
GPT-5.1 43.2% $0.005
GPT-5 mini 43% $0.005
Qwen 3.5 Flash (hosted 35B-A3B) 42.5% $0.006
Qwen3-VL-Plus 42.1% $0.005
Gemini 2.5 Pro Jun 2025 41.9% $0.009
o3 41.8% $0.03
o4-mini 41.1% $0.03
Grok 4.7 40.8% $0.32
Gemini 2.5 Flash Jun 2025 thinking 39.6% $0.003
Gemini 2.5 Flash Sep 2025 thinking 39.4% $0.002
Qwen3.7-Plus 39.3% $0.02
Grok 4.20 reasoning 38.2% $0.06
GPT-5.4 Nano 38.1% $0.006
Mistral Medium 3.5 37.6% $0.07
Inkling 36.5% $0.16
Claude Sonnet 4.5 thinking 36.1% $0.14
Llama 4 Maverick 35.7% $0.005
Gemini 3.8 Flash 35.1% $0.009
Claude Sonnet 4 non-thinking 35% $0.05
Grok 4.5 35% $0.12
Llama 4 Scout 34.8% $0.004
Claude Sonnet 4.5 non-thinking 32.9% $0.05
Inkling-Small 32.7% $0.02
Claude Sonnet 4 thinking 32% $0.11
Claude Haiku 4.5 thinking 31.8% $0.05
Claude Opus 4.1 non-thinking 31.4% $0.22
Grok 4.1 Fast reasoning 31.3% $0.001
Gemini 2.5 Flash-Lite Sep 2025 thinking 30.8% $0.0009
Qwen 3.5 Plus (hosted 397B-A17B) thinking 30.4% $0.04
Claude Opus 4.1 thinking 30.4% $0.32
GPT-5 nano 30.4% $0.002
Grok 4 Fast reasoning 29.8% $0.01
Grok 4.6 28.9% $0.11
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 28% $0.001
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 27.5% $0.0009
Grok 4 25.1% $0.41
Mistral Large 3 24.6% $0.008
Grok 4.3 19.7% $0.04
Grok 4 Fast non-reasoning 16% $0.01
Grok 4.1 Fast non-reasoning 11.9% $0.001

5 models have held number one on the official leaderboard since its release in October 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Anthropic
Own leaderboard 5 leaders
Claude Opus 4.7 56.1%

Source: Official SAGE (Vals) leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 4.7 Claude Opus 4.5 Gemini 3 Pro Gemini 2.5 Flash Llama 4 Maverick 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 4.7 Gemini 3 Pro

Source: Official SAGE (Vals) leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Google DeepMind Anthropic
SAGE v1.1 (Vals) Passed 85% in Oct 2026, by Claude Opus 5.5
Claude Opus 5.5 91.3%
SAGE (Vals) 5 leaders
Claude Opus 4.7 56.1%
SAGE v1.1 (Vals) Passed 85% in Oct 2026, by Claude Opus 5.5
Claude Opus 5.5 91.3%

Source: Official leaderboards

Replaced by SAGE v1.1 (Vals)

More in Other

All 43 in Other
  • Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
  • Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness. Gemini 4 Carbon 59.0
  • MTEB Fading Massive Text Embedding Benchmark: retrieval, clustering, classification across languages and modalities. sonar 42%
  • Anthropic ECI (AECI) Only in system cards Index that combines a basket of capability benchmarks into one capability score to track trends across Anthropic models. Claude Opus 5.5 169.4
  • SPEED-Bench Only in system cards Measures speculative decoding across varied prompts and domains. Reports the average acceptance length of draft tokens, so higher means faster generation at the same output. Nemotron 3 Ultra 5.15
  • Qwen in-house English automatic evaluation Only in system cards Models answer Qwen in-house English datasets on knowledge, comprehension, coding, math and related skills. Skills and weights change between reports. Qwen2-72B 72.9%