OlymMATH
Models solve Olympiad-level mathematics problems in English or Chinese. The evaluator checks final answers and reports pass@1 and consistency across repeated samples.
Easy and hard subsets exist in both languages. The released model-output collection is separate from the benchmark questions.
- Made by
- Renmin University of China
- Released
- March 2025
- Kind
- Competition problems with answers
- Level
- Competition
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Measures
- Capability
- Subject
- Mathematics
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
-
Gemini 2.5 Pro Mar 2025 #104 on the capability index
- Last model added
- April 2025
- Scores refreshed
- November 20th, 2026
In the authors’ words
The rapid advancement of large reasoning models has saturated existing math benchmarks, underscoring the urgent need for more challenging evaluation frameworks. To address this, we introduce OlymMATH, a rigorously curated, Olympiad-level math benchmark comprising 350 problems, each with parallel English and Chinese versions. OlymMATH is the first benchmark to unify dual evaluation paradigms within a single suite: (1) natural language evaluation through OlymMATH-EASY and OlymMATH-HARD, comprising 200 computational problems with numerical answers for objective rule-based assessment, and (2) formal verification through OlymMATH-LEAN, offering 150 problems formalized in Lean 4 for rigorous process-level evaluation. All problems are manually sourced from printed publications to minimize data contamination, verified by experts, and span four core domains. Extensive experiments reveal the benchmark's significant challenge, and our analysis also uncovers consistent performance gaps between languages and identifies cases where models employ heuristic "guessing" rather than rigorous reasoning. To further support community research, we release 582k+ reasoning trajectories, a visualization tool, and expert solutions at https://github.com/RUCAIBox/OlymMATH.
Gemini 2.5 Pro leads with 74.2% among models released by April 2025 Gemini 2.5 Pro leads with 74.2% among models released by April 2025
Best score per entry from the benchmark's paper, read on November 20th, 2026. A paper’s table doesn’t add newer models. The headline is the plain mean of English easy, English hard, Chinese easy and Chinese hard pass@1. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in April 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 74.2
Gemini 2.5 Pro
- 61.5 o3-mini
- 60.6
Qwen3-235B-A22B
- 53.3
Qwen3-30B-A3B
- 48.9
QwQ-32B
- 48.6 OpenMath-Nemo. (14B)
- 47.5 Skywork-OR1-Pre. (32B)
- 46.4
DeepSeek-R1
- 46.2 OpenMath-Nemo. (7B)
- 43.6 GLM-Z1-32B-0414
- 43.3 Light-R1-DS (32B)
- 40.3 OpenThinker2 (32B)
- 40.1 OpenMath-Nemo. (32B)
- 38.7
Qwen3-4B
- 33.8 Light-R1-DS (14B)
- 32.6
DeepSeek-R1-Distill-Qwen-32B
- 32.3 AceMath-RL (7B)
- 31.4 OpenMath-Nemo. (1.5B)
- 30.7 Skywork-OR1-Math (7B)
- 29.6
DeepSeek-R1-Distill-Qwen-14B
- 29.4 OpenThinker2 (7B)
- 27.6 Light-R1-DS (7B)
- 26.5 Skywork-OR1-Pre. (7B)
- 23.9
DeepSeek-R1-Distill-Qwen-7B
- 10.8 DeepScaleR-Pre. (1.5B)
- 9.00 STILL-3-Pre. (1.5B)
- 6.88
DeepSeek-R1-Distill-Qwen-1.5B
- 5.13
Qwen3-0.6B
- 1
Gemini 2.5 Pro Mar 2025 74.2%
- 2 o3-mini High reasoning effort 61.5%
- 3
Qwen3-235B-A22B Thinking 60.6%
- 4
Qwen3-30B-A3B Thinking 53.3%
- 5
QwQ-32B 48.9%
- 6 OpenMath-Nemo. (14B) 48.6%
- 7 Skywork-OR1-Pre. (32B) 47.5%
- 8
DeepSeek-R1 46.4%
- 9 OpenMath-Nemo. (7B) 46.2%
- 10 GLM-Z1-32B-0414 43.6%
- 11 Light-R1-DS (32B) 43.3%
- 12 OpenThinker2 (32B) 40.3%
- 13 OpenMath-Nemo. (32B) 40.1%
- 14
Qwen3-4B Thinking 38.7%
- 15 Light-R1-DS (14B) 33.8%
- 16
DeepSeek-R1-Distill-Qwen-32B 32.6%
- 17 AceMath-RL (7B) 32.3%
- 18 OpenMath-Nemo. (1.5B) 31.4%
- 19 Skywork-OR1-Math (7B) 30.7%
- 20
DeepSeek-R1-Distill-Qwen-14B 29.6%
- 21 OpenThinker2 (7B) 29.4%
- 22 Light-R1-DS (7B) 27.6%
- 23 Skywork-OR1-Pre. (7B) 26.5%
- 24
DeepSeek-R1-Distill-Qwen-7B 23.9%
- 25 DeepScaleR-Pre. (1.5B) 10.8%
- 26 STILL-3-Pre. (1.5B) 9.00%
- 27
DeepSeek-R1-Distill-Qwen-1.5B 6.88%
- 28
Qwen3-0.6B Thinking 5.13%
Source: The OlymMATH paper
- 1
Gemini 2.5 Pro Mar 2025 74.2%
- 2 o3-mini High reasoning effort 61.5%
- 3
Qwen3-235B-A22B Thinking 60.6%
- 4
Qwen3-30B-A3B Thinking 53.3%
- 5
QwQ-32B 48.9%
- 6
DeepSeek-R1 46.4%
- 7 GLM-Z1-32B-0414 43.6%
- 8
Qwen3-4B Thinking 38.7%
- 9
DeepSeek-R1-Distill-Qwen-32B 32.6%
- 10
DeepSeek-R1-Distill-Qwen-14B 29.6%
- 11
DeepSeek-R1-Distill-Qwen-7B 23.9%
- 12
DeepSeek-R1-Distill-Qwen-1.5B 6.88%
- 13
Qwen3-0.6B Thinking 5.13%
Source: The OlymMATH paper
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 74.2% | Mar 2025 | Official leaderboard | |
| 2 | o3-mini | 61.5% | High reasoning effort | Jan 2025 | Official leaderboard |
| 3 | | 60.6% | Thinking | Apr 2025 | Official leaderboard |
| 4 | | 53.3% | Thinking | Apr 2025 | Official leaderboard |
| 5 | | 48.9% | Mar 2025 | Official leaderboard | |
| 6 | OpenMath-Nemo. (14B) | 48.6% | Official leaderboard | ||
| 7 | Skywork-OR1-Pre. (32B) | 47.5% | Official leaderboard | ||
| 8 | | 46.4% | Jan 2025 | Official leaderboard | |
| 9 | OpenMath-Nemo. (7B) | 46.2% | Official leaderboard | ||
| 10 | GLM-Z1-32B-0414 | 43.6% | Apr 2025 | Official leaderboard | |
| 11 | Light-R1-DS (32B) | 43.3% | Official leaderboard | ||
| 12 | OpenThinker2 (32B) | 40.3% | Official leaderboard | ||
| 13 | OpenMath-Nemo. (32B) | 40.1% | Official leaderboard | ||
| 14 | | 38.7% | Thinking | Apr 2025 | Official leaderboard |
| 15 | Light-R1-DS (14B) | 33.8% | Official leaderboard | ||
| 16 | | 32.6% | Jan 2025 | Official leaderboard | |
| 17 | AceMath-RL (7B) | 32.3% | Official leaderboard | ||
| 18 | OpenMath-Nemo. (1.5B) | 31.4% | Official leaderboard | ||
| 19 | Skywork-OR1-Math (7B) | 30.7% | Official leaderboard | ||
| 20 | | 29.6% | Jan 2025 | Official leaderboard | |
| 21 | OpenThinker2 (7B) | 29.4% | Official leaderboard | ||
| 22 | Light-R1-DS (7B) | 27.6% | Official leaderboard | ||
| 23 | Skywork-OR1-Pre. (7B) | 26.5% | Official leaderboard | ||
| 24 | | 23.9% | Jan 2025 | Official leaderboard | |
| 25 | DeepScaleR-Pre. (1.5B) | 10.8% | Official leaderboard | ||
| 26 | STILL-3-Pre. (1.5B) | 9.00% | Official leaderboard | ||
| 27 | | 6.88% | Jan 2025 | Official leaderboard | |
| 28 | | 5.13% | Thinking | Apr 2025 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 74.2% | Mar 2025 | Official leaderboard | |
| 2 | o3-mini | 61.5% | High reasoning effort | Jan 2025 | Official leaderboard |
| 3 | | 60.6% | Thinking | Apr 2025 | Official leaderboard |
| 4 | | 53.3% | Thinking | Apr 2025 | Official leaderboard |
| 5 | | 48.9% | Mar 2025 | Official leaderboard | |
| 6 | | 46.4% | Jan 2025 | Official leaderboard | |
| 7 | GLM-Z1-32B-0414 | 43.6% | Apr 2025 | Official leaderboard | |
| 8 | | 38.7% | Thinking | Apr 2025 | Official leaderboard |
| 9 | | 32.6% | Jan 2025 | Official leaderboard | |
| 10 | | 29.6% | Jan 2025 | Official leaderboard | |
| 11 | | 23.9% | Jan 2025 | Official leaderboard | |
| 12 | | 6.88% | Jan 2025 | Official leaderboard | |
| 13 | | 5.13% | Thinking | Apr 2025 | Official leaderboard |
Rankings by part
We read 20 parts from the benchmark’s paper in 5 groups: english hard, language and difficulty, english easy, chinese hard and chinese easy. Each ranks models on one part of the test. Gemini 2.5 Pro leads 15 of them, and 2 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
Algebra
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 71.5%
- 2
Qwen3-235B-A22B ↑1 48%
- 3
Qwen3-30B-A3B ↑1 38.8%
- 4 GLM-Z1-32B-0414 ↑3 35%
- 5
QwQ-32B 32.9%
- 6
DeepSeek-R1 30%
- 7 o3-mini ↓5 29.5%
- 8
DeepSeek-R1-Distill-Qwen-32B ↑1 22.4%
- 9
Qwen3-4B ↓1 18.1%
- 10
DeepSeek-R1-Distill-Qwen-14B 16.1%
Geometry
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 75.5%
- 2
Qwen3-235B-A22B ↑1 49.5%
- 3
Qwen3-30B-A3B ↑1 33.8%
- 4 o3-mini ↓2 29%
- 5
QwQ-32B 26.6%
- 6
DeepSeek-R1 25.5%
- 7 GLM-Z1-32B-0414 21.5%
- 8
DeepSeek-R1-Distill-Qwen-32B ↑1 21.4%
- 9
DeepSeek-R1-Distill-Qwen-14B ↑1 17%
- 10
Qwen3-4B ↓2 14.8%
Number theory
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 59%
- 2 o3-mini 49.5%
- 3
Qwen3-235B-A22B 38%
- 4
QwQ-32B ↑1 26.7%
- 5
Qwen3-30B-A3B ↓1 26.7%
- 6
DeepSeek-R1-Distill-Qwen-32B ↑3 20.3%
- 7
Qwen3-4B ↑1 19.8%
- 8 GLM-Z1-32B-0414 ↓1 19.5%
- 9
DeepSeek-R1 ↓3 18.5%
- 10
DeepSeek-R1-Distill-Qwen-14B 18.1%
Combinatorics
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 27.5%
- 2 o3-mini 17%
- 3
Qwen3-235B-A22B 10.5%
- 4
QwQ-32B ↑1 6.20%
- 5
Qwen3-30B-A3B ↓1 5.90%
- 6 GLM-Z1-32B-0414 ↑1 4.50%
- 7
DeepSeek-R1 ↓1 4.00%
- 8
DeepSeek-R1-Distill-Qwen-32B ↑1 3.40%
- 9
DeepSeek-R1-Distill-Qwen-7B ↑2 3.10%
- 10
Qwen3-4B ↓2 3.10%
English hard
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 58.4%
- 2
Qwen3-235B-A22B ↑1 36.5%
- 3 o3-mini ↓1 31.2%
- 4
Qwen3-30B-A3B 26.3%
- 5
QwQ-32B 23.1%
- 6 GLM-Z1-32B-0414 ↑1 20.1%
- 7
DeepSeek-R1 ↓1 19.5%
- 8
DeepSeek-R1-Distill-Qwen-32B ↑1 16.9%
- 9
Qwen3-4B ↓1 13.9%
- 10
DeepSeek-R1-Distill-Qwen-14B 13.3%
English easy
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 92.2%
- 2 o3-mini 91.4%
- 3
Qwen3-235B-A22B 90.5%
- 4
Qwen3-30B-A3B 87.2%
- 5
QwQ-32B 84%
- 6
DeepSeek-R1 79.6%
- 7 GLM-Z1-32B-0414 76.8%
- 8
Qwen3-4B 72.8%
- 9
DeepSeek-R1-Distill-Qwen-32B 67.3%
- 10
DeepSeek-R1-Distill-Qwen-14B 61.8%
Chinese hard
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 55.4%
- 2 o3-mini 32.9%
- 3
Qwen3-235B-A22B 28.1%
- 4
Qwen3-30B-A3B 20.1%
- 5
DeepSeek-R1 ↑1 15.9%
- 6
QwQ-32B ↓1 14.1%
- 7 GLM-Z1-32B-0414 12.1%
- 8
Qwen3-4B 8.30%
- 9
DeepSeek-R1-Distill-Qwen-32B 5.80%
- 10
DeepSeek-R1-Distill-Qwen-7B ↑1 5.30%
Chinese easy
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 90.8%
- 2 o3-mini 90.5%
- 3
Qwen3-235B-A22B 87.5%
- 4
Qwen3-30B-A3B 79.7%
- 5
QwQ-32B 74.3%
- 6
DeepSeek-R1 70.4%
- 7 GLM-Z1-32B-0414 65.6%
- 8
Qwen3-4B 59.7%
- 9
DeepSeek-R1-Distill-Qwen-32B 40.4%
- 10
DeepSeek-R1-Distill-Qwen-14B 38.4%
Algebra
13 models we track have a score here.
- 1
Qwen3-235B-A22B ↑2 93.5%
- 2 o3-mini 93%
- 3
Gemini 2.5 Pro Mar 2025 ↓2 92%
- 4
QwQ-32B ↑1 91.8%
- 5
Qwen3-30B-A3B ↓1 91.4%
- 6
DeepSeek-R1 90.5%
- 7 GLM-Z1-32B-0414 86.5%
- 8
Qwen3-4B 76.4%
- 9
DeepSeek-R1-Distill-Qwen-32B 73.6%
- 10
DeepSeek-R1-Distill-Qwen-14B 69%
Geometry
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 97%
- 2
Qwen3-30B-A3B ↑2 92.9%
- 3
Qwen3-235B-A22B 92.4%
- 4 o3-mini ↓2 89.8%
- 5
QwQ-32B 87%
- 6
DeepSeek-R1 82.2%
- 7 GLM-Z1-32B-0414 79.5%
- 8
Qwen3-4B 79.1%
- 9
DeepSeek-R1-Distill-Qwen-32B 71.8%
- 10
DeepSeek-R1-Distill-Qwen-14B 65.1%
Number theory
13 models we track have a score here.
- 1
Qwen3-235B-A22B ↑2 99%
- 2
Gemini 2.5 Pro Mar 2025 ↓1 98.1%
- 3 o3-mini ↓1 97.1%
- 4
QwQ-32B ↑1 95%
- 5
DeepSeek-R1 ↑1 94.2%
- 6
Qwen3-30B-A3B ↓2 90.9%
- 7 GLM-Z1-32B-0414 90.4%
- 8
Qwen3-4B 85.1%
- 9
DeepSeek-R1-Distill-Qwen-32B 84.5%
- 10
DeepSeek-R1-Distill-Qwen-14B 79.4%
Combinatorics
13 models we track have a score here.
- 1 o3-mini ↑1 89.2%
- 2
Gemini 2.5 Pro Mar 2025 ↓1 84.5%
- 3
Qwen3-235B-A22B 81.9%
- 4
Qwen3-30B-A3B 75.6%
- 5
QwQ-32B 69%
- 6
DeepSeek-R1 60.8%
- 7 GLM-Z1-32B-0414 59.1%
- 8
Qwen3-4B 57.1%
- 9
DeepSeek-R1-Distill-Qwen-32B 49%
- 10
DeepSeek-R1-Distill-Qwen-14B 44%
Algebra
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 65%
- 2
Qwen3-235B-A22B ↑1 36.5%
- 3
Qwen3-30B-A3B ↑1 35.6%
- 4 o3-mini ↓2 31.5%
- 5
QwQ-32B 20.9%
- 6
DeepSeek-R1 20%
- 7 GLM-Z1-32B-0414 18%
- 8
Qwen3-4B 12.5%
- 9
DeepSeek-R1-Distill-Qwen-32B 6.50%
- 10
DeepSeek-R1-Distill-Qwen-7B ↑1 6.10%
Geometry
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 78%
- 2
Qwen3-235B-A22B ↑1 43.5%
- 3 o3-mini ↓1 32.5%
- 4
DeepSeek-R1 ↑2 25%
- 5
Qwen3-30B-A3B ↓1 24.1%
- 6
QwQ-32B ↓1 15.9%
- 7 GLM-Z1-32B-0414 12%
- 8
DeepSeek-R1-Distill-Qwen-7B ↑3 7.90%
- 9
Qwen3-4B ↓1 7.00%
- 10
DeepSeek-R1-Distill-Qwen-32B ↓1 5.40%
Number theory
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 53.5%
- 2 o3-mini 48.5%
- 3
Qwen3-235B-A22B 28.5%
- 4
Qwen3-30B-A3B 18.1%
- 5
QwQ-32B 17.6%
- 6
DeepSeek-R1 17%
- 7 GLM-Z1-32B-0414 16%
- 8
Qwen3-4B 12.6%
- 9
DeepSeek-R1-Distill-Qwen-32B 10.6%
- 10
DeepSeek-R1-Distill-Qwen-14B 8.70%
Combinatorics
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 25%
- 2 o3-mini 19%
- 3
Qwen3-235B-A22B 4.00%
- 4
Qwen3-30B-A3B 2.70%
- 5 GLM-Z1-32B-0414 ↑2 2.50%
- 6
QwQ-32B ↓1 2.00%
- 7
DeepSeek-R1 ↓1 1.50%
- 8
Qwen3-4B 0.90%
- 9
DeepSeek-R1-Distill-Qwen-32B 0.70%
- 10
DeepSeek-R1-Distill-Qwen-7B ↑1 0.60%
Algebra
13 models we track have a score here.
- 1 o3-mini ↑1 93%
- 2
Qwen3-235B-A22B ↑1 91%
- 3
Gemini 2.5 Pro Mar 2025 ↓2 90.5%
- 4
Qwen3-30B-A3B 87.8%
- 5
QwQ-32B 85.4%
- 6
DeepSeek-R1 79.5%
- 7 GLM-Z1-32B-0414 76%
- 8
Qwen3-4B 70.8%
- 9
DeepSeek-R1-Distill-Qwen-32B 45.2%
- 10
DeepSeek-R1-Distill-Qwen-14B 43.1%
Geometry
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 93.2%
- 2
Qwen3-235B-A22B ↑1 90.2%
- 3 o3-mini ↓1 89.4%
- 4
Qwen3-30B-A3B 84.7%
- 5
QwQ-32B 76.6%
- 6
DeepSeek-R1 74.6%
- 7 GLM-Z1-32B-0414 69.3%
- 8
Qwen3-4B 61%
- 9
DeepSeek-R1-Distill-Qwen-32B 41.8%
- 10
DeepSeek-R1-Distill-Qwen-14B 38.9%
Number theory
13 models we track have a score here.
- 1
Gemini 2.5 Pro Mar 2025 100%
- 2 o3-mini 99%
- 3
Qwen3-235B-A22B 94.2%
- 4
QwQ-32B ↑1 92.9%
- 5
Qwen3-30B-A3B ↓1 91.3%
- 6 GLM-Z1-32B-0414 ↑1 89.4%
- 7
DeepSeek-R1 ↓1 88.5%
- 8
Qwen3-4B 74.8%
- 9
DeepSeek-R1-Distill-Qwen-32B 60.2%
- 10
DeepSeek-R1-Distill-Qwen-14B 58.2%
Combinatorics
13 models we track have a score here.
- 1 o3-mini ↑1 85.8%
- 2
Gemini 2.5 Pro Mar 2025 ↓1 84.1%
- 3
Qwen3-235B-A22B 78.4%
- 4
Qwen3-30B-A3B 61.9%
- 5
QwQ-32B 53.8%
- 6
DeepSeek-R1 49.6%
- 7 GLM-Z1-32B-0414 41.8%
- 8
Qwen3-4B 41.8%
- 9
DeepSeek-R1-Distill-Qwen-32B 26%
- 10
DeepSeek-R1-Distill-Qwen-14B 24.8%
2 models have held number one in the benchmark’s paper since its release in March 2025
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: The OlymMATH paper
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official OlymMATH leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Math
- AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
- IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
- HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
- PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked.
Qwen3.7-Max 86.5%
- MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
- MathArena Apex Shortlist In use Models solve 47 hard final-answer problems that MathArena selected from 2025 competitions, where the best models scored about 50% at selection. Scored by accuracy over repeated attempts. GPT-5.5 98.4%
Data audit
9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 19 months ago, in April 2025, for Qwen3-0.6B. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
- Pass Independent runs Scores from the benchmark’s paper (28 entries).
- Pass Models scored 13 models we track, plus 15 other entries on its leaderboard.
- Warning Newest score The newest came 19 months ago, in April 2025, for Qwen3-0.6B.
- Note Frontier models None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104).
- Pass Headroom The best is 74.2%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on github.com: 13 rows for models we track and 15 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.