Benchmark Almanac

SLR-Bench

GPT-5 81.8% leads since August 2025
Not in model cards

Models write Prolog rules that classify trains by their composition. The verifier runs each rule to determine whether its predictions are correct.

Made by
Lukas Helff and colleagues
Released
June 2025
Kind
Abstract puzzles and logic
Level
University
Scored
Checked automatically
Answers
Code
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Code → Code
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Gemini 3 Flash #60 on the capability index
Last model added
April 2026
Scores refreshed
November 20th, 2026
45
Entries on its own leaderboard, 32 models we track

GPT-5 leads with 81.8% among models released by April 2026 GPT-5 leads with 81.8% among models released by April 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 GPT-5 medium 81.8%
  2. 2 Gemini 3 Flash high 80.8%
  3. 3 GPT-5 mini high 79.5%
  4. 4 gpt-oss-120b high 67.5%
  5. 5 GPT-5.4 Mini medium 66%
  6. 6 GLM-4.7 FP8 65.5%
  7. 7 Gemini 3.1 Flash-Lite high 65.2%
  8. 8 Gemini 2.5 Flash Sep 2025 65%
  9. 9 o3-mini medium 62.2%
  10. 10 o4-mini medium 61.5%
  11. 11 Qwen3.5-27B 60%
  12. 12 GPT-5 Chat 59.8%
  13. 13 Gemini-2.5-pro (snapshot unspecified) 58%
  14. 14 Qwen3.5-35B-A3B 57.8%
  15. 15 GLM-4.5-Air 55%
  16. 16 Gemma 4 31B IT 54.8%
  17. 17 Kimi K2.5 53.5%
  18. 18 Nemotron-3 120B 53%
  19. 19 Kimi K2 Sep 2025 52.8%
  20. 20 gpt-oss-20b high 51.2%
  21. 21 Llama-3.1-Nemotron-Ultra-253B-v1 51%
  22. 22 GPT-5 nano high 51%
  23. 23 Nemotron-Cascade 2 30B-A3B 49.2%
  24. 24 Qwen3.5-9B 49%
  25. 25 Gemini 2.5 Flash-Lite Sep 2025, high 47.8%
  26. 26 Ministral 3 14B 47%
  27. 27 Olmo 3 32B Think 45%
  28. 28 Qwen3.5-4B 44.5%
  29. 29 DeepSeek-R1-Distill-Llama-70B 43.8%
  30. 30 DeepSeek-R1-Distill-Qwen-32B 43.8%
  31. 31 Gemma 4 26B A4B 43.8%
  32. 32 GPT-4.5 41.2%
  33. 33 Ministral-3 8B 41.2%
  34. 34 Gemma 4 E4B 40.8%
  35. 35 Olmo 3.1 32B Think 38.5%
  36. 36 DeepSeek-R1-Distill Qwen 8B 38.2%
  37. 37 Ministral-3 3B 33.8%
  38. 38 Gemma 4 E2B 33%
  39. 39 GPT-4o (snapshot unspecified) 32.2%
  40. 40 GPT-4 Turbo (snapshot unspecified) 28.8%
  41. 41 GPT-4o Mini (snapshot unspecified) 27.5%
  42. 42 Llama-3.3-Nemotron 49B 25%
  43. 43 OLMo-3 7B Think 11.5%
  44. 44 Kimi-VL A3B 8.20%
  45. 45 Nemotron-3 30B 3.00%

Source: Official SLR-Bench leaderboard

Source: Official SLR-Bench leaderboard

#EntryScoreSettingReleasedRun onSource
1 GPT-5 81.8% medium Aug 2025 Apr 2026 Official leaderboard
2 Gemini 3 Flash 80.8% high Dec 2025 Apr 2026 Official leaderboard
3 GPT-5 mini 79.5% high Aug 2025 Apr 2026 Official leaderboard
4 gpt-oss-120b 67.5% high Aug 2025 Apr 2026 Official leaderboard
5 GPT-5.4 Mini 66% medium Mar 2026 Apr 2026 Official leaderboard
6 GLM-4.7 FP8 65.5% Apr 2026 Official leaderboard
7 Gemini 3.1 Flash-Lite 65.2% high Mar 2026 Apr 2026 Official leaderboard
8 Gemini 2.5 Flash Sep 2025 65% Sep 2025 Apr 2026 Official leaderboard
9 o3-mini 62.2% medium Jan 2025 Apr 2026 Official leaderboard
10 o4-mini 61.5% medium Apr 2025 Apr 2026 Official leaderboard
11 Qwen3.5-27B 60% Feb 2026 Apr 2026 Official leaderboard
12 GPT-5 Chat 59.8% Aug 2025 Apr 2026 Official leaderboard
13 Gemini-2.5-pro (snapshot unspecified) 58% Apr 2026 Official leaderboard
14 Qwen3.5-35B-A3B 57.8% Feb 2026 Apr 2026 Official leaderboard
15 GLM-4.5-Air 55% Jul 2025 Apr 2026 Official leaderboard
16 Gemma 4 31B IT 54.8% Apr 2026 Apr 2026 Official leaderboard
17 Kimi K2.5 53.5% Jan 2026 Apr 2026 Official leaderboard
18 Nemotron-3 120B 53% Apr 2026 Official leaderboard
19 Kimi K2 Sep 2025 52.8% Sep 2025 Apr 2026 Official leaderboard
20 gpt-oss-20b 51.2% high Aug 2025 Apr 2026 Official leaderboard
21 Llama-3.1-Nemotron-Ultra-253B-v1 51% Apr 2025 Apr 2026 Official leaderboard
22 GPT-5 nano 51% high Aug 2025 Apr 2026 Official leaderboard
23 Nemotron-Cascade 2 30B-A3B 49.2% Mar 2026 Apr 2026 Official leaderboard
24 Qwen3.5-9B 49% Feb 2026 Apr 2026 Official leaderboard
25 Gemini 2.5 Flash-Lite Sep 2025 47.8% high Sep 2025 Apr 2026 Official leaderboard
26 Ministral 3 14B 47% Dec 2025 Apr 2026 Official leaderboard
27 Olmo 3 32B Think 45% Nov 2025 Apr 2026 Official leaderboard
28 Qwen3.5-4B 44.5% Feb 2026 Apr 2026 Official leaderboard
29 DeepSeek-R1-Distill-Llama-70B 43.8% Jan 2025 Apr 2026 Official leaderboard
30 DeepSeek-R1-Distill-Qwen-32B 43.8% Jan 2025 Apr 2026 Official leaderboard
31 Gemma 4 26B A4B 43.8% Apr 2026 Apr 2026 Official leaderboard
32 GPT-4.5 41.2% Feb 2025 Apr 2026 Official leaderboard
33 Ministral-3 8B 41.2% Apr 2026 Official leaderboard
34 Gemma 4 E4B 40.8% Mar 2026 Apr 2026 Official leaderboard
35 Olmo 3.1 32B Think 38.5% Dec 2025 Apr 2026 Official leaderboard
36 DeepSeek-R1-Distill Qwen 8B 38.2% Apr 2026 Official leaderboard
37 Ministral-3 3B 33.8% Apr 2026 Official leaderboard
38 Gemma 4 E2B 33% Mar 2026 Apr 2026 Official leaderboard
39 GPT-4o (snapshot unspecified) 32.2% Apr 2026 Official leaderboard
40 GPT-4 Turbo (snapshot unspecified) 28.8% Apr 2026 Official leaderboard
41 GPT-4o Mini (snapshot unspecified) 27.5% Apr 2026 Official leaderboard
42 Llama-3.3-Nemotron 49B 25% Apr 2026 Official leaderboard
43 OLMo-3 7B Think 11.5% Apr 2026 Official leaderboard
44 Kimi-VL A3B 8.20% Apr 2026 Official leaderboard
45 Nemotron-3 30B 3.00% Apr 2026 Official leaderboard
#ModelScoreSettingReleasedRun onSource
1 GPT-5 81.8% medium Aug 2025 Apr 2026 Official leaderboard
2 Gemini 3 Flash 80.8% high Dec 2025 Apr 2026 Official leaderboard
3 GPT-5 mini 79.5% high Aug 2025 Apr 2026 Official leaderboard
4 gpt-oss-120b 67.5% high Aug 2025 Apr 2026 Official leaderboard
5 GPT-5.4 Mini 66% medium Mar 2026 Apr 2026 Official leaderboard
6 Gemini 3.1 Flash-Lite 65.2% high Mar 2026 Apr 2026 Official leaderboard
7 Gemini 2.5 Flash Sep 2025 65% Sep 2025 Apr 2026 Official leaderboard
8 o3-mini 62.2% medium Jan 2025 Apr 2026 Official leaderboard
9 o4-mini 61.5% medium Apr 2025 Apr 2026 Official leaderboard
10 Qwen3.5-27B 60% Feb 2026 Apr 2026 Official leaderboard
11 GPT-5 Chat 59.8% Aug 2025 Apr 2026 Official leaderboard
12 Qwen3.5-35B-A3B 57.8% Feb 2026 Apr 2026 Official leaderboard
13 GLM-4.5-Air 55% Jul 2025 Apr 2026 Official leaderboard
14 Gemma 4 31B IT 54.8% Apr 2026 Apr 2026 Official leaderboard
15 Kimi K2.5 53.5% Jan 2026 Apr 2026 Official leaderboard
16 Kimi K2 Sep 2025 52.8% Sep 2025 Apr 2026 Official leaderboard
17 gpt-oss-20b 51.2% high Aug 2025 Apr 2026 Official leaderboard
18 Llama-3.1-Nemotron-Ultra-253B-v1 51% Apr 2025 Apr 2026 Official leaderboard
19 GPT-5 nano 51% high Aug 2025 Apr 2026 Official leaderboard
20 Nemotron-Cascade 2 30B-A3B 49.2% Mar 2026 Apr 2026 Official leaderboard
21 Qwen3.5-9B 49% Feb 2026 Apr 2026 Official leaderboard
22 Gemini 2.5 Flash-Lite Sep 2025 47.8% high Sep 2025 Apr 2026 Official leaderboard
23 Ministral 3 14B 47% Dec 2025 Apr 2026 Official leaderboard
24 Olmo 3 32B Think 45% Nov 2025 Apr 2026 Official leaderboard
25 Qwen3.5-4B 44.5% Feb 2026 Apr 2026 Official leaderboard
26 DeepSeek-R1-Distill-Llama-70B 43.8% Jan 2025 Apr 2026 Official leaderboard
27 DeepSeek-R1-Distill-Qwen-32B 43.8% Jan 2025 Apr 2026 Official leaderboard
28 Gemma 4 26B A4B 43.8% Apr 2026 Apr 2026 Official leaderboard
29 GPT-4.5 41.2% Feb 2025 Apr 2026 Official leaderboard
30 Gemma 4 E4B 40.8% Mar 2026 Apr 2026 Official leaderboard
31 Olmo 3.1 32B Think 38.5% Dec 2025 Apr 2026 Official leaderboard
32 Gemma 4 E2B 33% Mar 2026 Apr 2026 Official leaderboard

Rankings by part

We read 4 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as Basic, Easy and Medium. GPT-5 leads 2 of them, and 2 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

Basic

32 models we track have a score here.

  1. 1 o3-mini ↑7 100%
  2. 2 gpt-oss-120b ↑2 100%
  3. 3 GPT-5 ↓2 100%
  4. 4 GPT-5 Chat ↑7 100%
  5. 5 GPT-5 mini ↓2 100%
  6. 6 GPT-5 nano ↑13 100%
  7. 7 Gemini 2.5 Flash Sep 2025 100%
  8. 8 Gemini 3 Flash ↓6 100%
  9. 9 Qwen3.5-27B ↑1 100%
  10. 10 Qwen3.5-9B ↑11 100%

The best score, 81.8%, costs $103 for the full run

Runs on the cost frontier score higher than any run that's cheaper. GPT-5 mini at high comes within 2.3 points of the best for $28 for the full run, 4 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Mistral AI Google DeepMind Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 $1,000 Cost of the full run, log scale GPT-5, medium GPT-5 mini, high GPT-5 mini, medium gpt-oss-120b, high gpt-oss-120b, medium Ministral 3 14B gpt-oss-20b, low 0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 $1,000 Cost of the full run, log scale GPT-5, medium GPT-5 mini, medium

Source: Official SLR-Bench leaderboard

ModelEffortHarnessScoreCost of the full runOutput tokens
GPT-5 medium 81.8% $103
Gemini 3 Flash high 80.8% $81
GPT-5 mini high 79.5% $28
GPT-5 mini medium 67.8% $12
gpt-oss-120b high 67.5% $3.83
GPT-5.4 Mini medium 66% $81
Gemini 3.1 Flash-Lite high 65.2% $24
Gemini 2.5 Flash Sep 2025 65% $58
o3-mini medium 62.2% $75
o4-mini medium 61.5% $42
Qwen3.5-27B 60% $17
GPT-5 Chat 59.8% $36
Qwen3.5-35B-A3B 57.8% $15
GLM-4.5-Air 55% $8.01
gpt-oss-120b medium 55% $1.18
GPT-5 mini low 54.8% $4.07
Gemma 4 31B IT 54.8% $3.87
Kimi K2.5 53.5% $16
Kimi K2 Sep 2025 52.8% $20
gpt-oss-20b high 51.2% $4.17
Llama-3.1-Nemotron-Ultra-253B-v1 51% $4.98
GPT-5 nano high 51% $6.62
gpt-oss-20b medium 50% $3.90
Nemotron-Cascade 2 30B-A3B 49.2%
Qwen3.5-9B 49% $2.07
Gemini 2.5 Flash-Lite Sep 2025 high 47.8% $8.26
Gemini 3.1 Flash-Lite low 47.5% $3.98
Gemini 3.1 Flash-Lite medium 47.2% $4.57
GPT-5 nano medium 47% $2.81
Ministral 3 14B 47% $0.82
Olmo 3 32B Think 45% $9.04
Qwen3.5-4B 44.5%
gpt-oss-120b low 44.2% $0.44
DeepSeek-R1-Distill-Llama-70B 43.8% $9.55
DeepSeek-R1-Distill-Qwen-32B 43.8% $4.32
Gemma 4 26B A4B 43.8% $5.29
GPT-4.5 41.2% $576
Gemma 4 E4B 40.8%
Olmo 3.1 32B Think 38.5%
GPT-5 nano low 37.8% $0.80
gpt-oss-20b low 37% $0.37
Gemma 4 E2B 33%
Gemini 2.5 Flash-Lite Sep 2025 17.5% $1.16

The best score, 81.8%, costs $103 for the full run

Runs on the cost frontier score higher than any run that's cheaper. GPT-5 mini at high comes within 2.3 points of the best for $28 for the full run, 4 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Mistral AI Google DeepMind Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 $1,000 Cost of the full run, log scale GPT-5, medium GPT-5 mini, high GPT-5 mini, medium gpt-oss-120b, high gpt-oss-120b, medium Ministral 3 14B gpt-oss-20b, low 0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 $1,000 Cost of the full run, log scale GPT-5, medium GPT-5 mini, medium

Source: Official SLR-Bench leaderboard

ModelEffortHarnessScoreCost of the full runOutput tokens
GPT-5 medium 81.8% $103
Gemini 3 Flash high 80.8% $81
GPT-5 mini high 79.5% $28
GPT-5 mini medium 67.8% $12
gpt-oss-120b high 67.5% $3.83
GPT-5.4 Mini medium 66% $81
Gemini 3.1 Flash-Lite high 65.2% $24
Gemini 2.5 Flash Sep 2025 65% $58
o3-mini medium 62.2% $75
o4-mini medium 61.5% $42
Qwen3.5-27B 60% $17
GPT-5 Chat 59.8% $36
Qwen3.5-35B-A3B 57.8% $15
GLM-4.5-Air 55% $8.01
gpt-oss-120b medium 55% $1.18
GPT-5 mini low 54.8% $4.07
Gemma 4 31B IT 54.8% $3.87
Kimi K2.5 53.5% $16
Kimi K2 Sep 2025 52.8% $20
gpt-oss-20b high 51.2% $4.17
Llama-3.1-Nemotron-Ultra-253B-v1 51% $4.98
GPT-5 nano high 51% $6.62
gpt-oss-20b medium 50% $3.90
Nemotron-Cascade 2 30B-A3B 49.2%
Qwen3.5-9B 49% $2.07
Gemini 2.5 Flash-Lite Sep 2025 high 47.8% $8.26
Gemini 3.1 Flash-Lite low 47.5% $3.98
Gemini 3.1 Flash-Lite medium 47.2% $4.57
GPT-5 nano medium 47% $2.81
Ministral 3 14B 47% $0.82
Olmo 3 32B Think 45% $9.04
Qwen3.5-4B 44.5%
gpt-oss-120b low 44.2% $0.44
DeepSeek-R1-Distill-Llama-70B 43.8% $9.55
DeepSeek-R1-Distill-Qwen-32B 43.8% $4.32
Gemma 4 26B A4B 43.8% $5.29
GPT-4.5 41.2% $576
Gemma 4 E4B 40.8%
Olmo 3.1 32B Think 38.5%
GPT-5 nano low 37.8% $0.80
gpt-oss-20b low 37% $0.37
Gemma 4 E2B 33%
Gemini 2.5 Flash-Lite Sep 2025 17.5% $1.16

3 models have held number one on the official leaderboard since its release in June 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Not tested since Apr 2026
GPT-5 81.8%

Source: Official SLR-Bench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) GPT-5 gpt-oss-120b o3-mini DeepSeek-R1-Distill-Llama-70B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) GPT-5 gpt-oss-120b

Source: Official SLR-Bench leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Science and reasoning

All 203 in Science and reasoning
  • GPQA Diamond In use 198 graduate-level 'Google-proof' multiple-choice questions in biology, chemistry and physics. Gemini 4 Carbon 97%
  • Terminal-Bench-Science In use 70 scientific research-workflow tasks across five science areas, run in Harbor containers. Hidden tests give the resolution rate. GPT-6 Astra 68.1%
  • BioMysteryBench In use Bioinformatics problems on anonymized real datasets that require real analysis with tools. Rubric-graded, with human-solvable and human-difficult splits. Claude Fable 5.2 83.7%
  • FrontierScience In use Models solve physics, chemistry, and biology problems in an Olympiad track graded by short answer and a Research track graded by 10-point rubrics. Intern-S2-397B 87%
  • HiPhO In use Models solve theory problems from 2024 and 2025 physics olympiad exams, some with diagrams. Official marking schemes grade answers and steps, mapped to medal levels. Gemini 3 Pro 78.3%
  • LABBench2 In use 11 biology research sub-tasks (literature, figures, tables, patents, trials and more) with tool access. Macro-averaged accuracy. GPT-5.2 60.9%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (45 entries).
  • Pass Models scored 32 models we track, plus 13 other entries on its leaderboard.
  • Warning Newest score The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60).
  • Pass Headroom The best is 81.8%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 43 rows for models we track and 13 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check