Benchmark Almanac

SnorkelGraph

GPT-5.4 84.5% leads since March 2026
Not in model cards

Models solve 200 procedurally generated graph problems, such as minimum vertex cover, given as node and edge lists in text. Answers are checked against verifiable ground truth.

Accuracy@1 with a 16,384-token budget. Part of Snorkel's procedurally generated reasoning series with SnorkelSequences and SnorkelSpatial. The board is archived.

Made by
Snorkel AI
Released
September 2025
Kind
Word problems
Level
University
Scored
Checked automatically
Answers
Short answer
Tasks
Kept private
Strongest model tested
GPT-5.4 #27 on the capability index
Last model added
March 2026
Scores refreshed
November 12th, 2026
33
Entries on its own leaderboard, 24 models we track

GPT-5.4 leads with 84.5% among models released by March 2026

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in March 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

33 entries. Scroll sideways for the rest.

  1. 84.5 GPT-5.4
  2. 75.0 o4-mini
  3. 75.0 Grok 4 Fast
  4. 72.5 GPT-5 mini
  5. 72.0 GPT-5
  6. 71.5 o3
  7. 71.0 o3-mini
  8. 64.5 Claude Opus 4
  9. 64.0 Grok 3
  10. 63.0 GPT-4.1
  11. 62.5 GPT-5 nano
  12. 61.5 Qwen 3 235B [snapshot unspecified]
  13. 61.0 Grok 4
  14. 58.0 Gemini 2.5 Pro [snapshot unspecified]
  15. 58.0 Claude Sonnet 4
  16. 55.0 Gemini 2.5 Flash [snapshot unspecified]
  17. 53.5 Magistral Medium [snapshot unspecified]
  18. 50.0 Claude 3.7 Sonnet
  19. 34.5 Amazon Nova Premier
  20. 34.0 Llama 4 Maverick
  21. 30.0 Mistral Large [snapshot unspecified]
  22. 29.0 Llama-3.3-Nemotron-Super-49B-v1
  23. 28.0 Amazon Nova Pro
  24. 26.0 Llama 4 Scout
  25. 24.5 Codestral [snapshot unspecified]
  26. 23.5 Llama 3.3 70B
  27. 22.5 Nvidia 70B Instruct [snapshot unspecified]
  28. 20.5 Llama 3.1-405B
  29. 19.0 Amazon Nova Lite
  30. 17.5 Amazon Nova Micro
  31. 15.0 Command R+
  32. 10.5 Command-Light
  33. 10.0 Command
  1. 1 GPT-5.4 84.5%
  2. 2 o4-mini 75%
  3. 3 Grok 4 Fast reasoning 75%
  4. 4 GPT-5 mini 72.5%
  5. 5 GPT-5 72%
  6. 6 o3 71.5%
  7. 7 o3-mini 71%
  8. 8 Claude Opus 4 64.5%
  9. 9 Grok 3 64%
  10. 10 GPT-4.1 63%
  11. 11 GPT-5 nano 62.5%
  12. 12 Qwen 3 235B [snapshot unspecified] 61.5%
  13. 13 Grok 4 61%
  14. 14 Gemini 2.5 Pro [snapshot unspecified] 58%
  15. 15 Claude Sonnet 4 58%
  16. 16 Gemini 2.5 Flash [snapshot unspecified] 55%
  17. 17 Magistral Medium [snapshot unspecified] 53.5%
  18. 18 Claude 3.7 Sonnet 50%
  19. 19 Amazon Nova Premier 34.5%
  20. 20 Llama 4 Maverick 34%
  21. 21 Mistral Large [snapshot unspecified] 30%
  22. 22 Llama-3.3-Nemotron-Super-49B-v1 29%
  23. 23 Amazon Nova Pro 28%
  24. 24 Llama 4 Scout 26%
  25. 25 Codestral [snapshot unspecified] 24.5%
  26. 26 Llama 3.3 70B 23.5%
  27. 27 Nvidia 70B Instruct [snapshot unspecified] 22.5%
  28. 28 Llama 3.1-405B 20.5%
  29. 29 Amazon Nova Lite 19%
  30. 30 Amazon Nova Micro 17.5%
  31. 31 Command R+ 15%
  32. 32 Command-Light 10.5%
  33. 33 Command 10%

Source: Official SnorkelGraph leaderboard

#EntryScoreSettingDateSource
1 GPT-5.4 84.5% Mar 2026 Official leaderboard
2 o4-mini 75% Apr 2025 Official leaderboard
3 Grok 4 Fast 75% reasoning Sep 2025 Official leaderboard
4 GPT-5 mini 72.5% Aug 2025 Official leaderboard
5 GPT-5 72% Aug 2025 Official leaderboard
6 o3 71.5% Apr 2025 Official leaderboard
7 o3-mini 71% Jan 2025 Official leaderboard
8 Claude Opus 4 64.5% May 2025 Official leaderboard
9 Grok 3 64% Apr 2025 Official leaderboard
10 GPT-4.1 63% Apr 2025 Official leaderboard
11 GPT-5 nano 62.5% Aug 2025 Official leaderboard
12 Qwen 3 235B [snapshot unspecified] 61.5% Official leaderboard
13 Grok 4 61% Jul 2025 Official leaderboard
14 Gemini 2.5 Pro [snapshot unspecified] 58% Official leaderboard
15 Claude Sonnet 4 58% May 2025 Official leaderboard
16 Gemini 2.5 Flash [snapshot unspecified] 55% Official leaderboard
17 Magistral Medium [snapshot unspecified] 53.5% Official leaderboard
18 Claude 3.7 Sonnet 50% Feb 2025 Official leaderboard
19 Amazon Nova Premier 34.5% Apr 2025 Official leaderboard
20 Llama 4 Maverick 34% Apr 2025 Official leaderboard
21 Mistral Large [snapshot unspecified] 30% Official leaderboard
22 Llama-3.3-Nemotron-Super-49B-v1 29% Mar 2025 Official leaderboard
23 Amazon Nova Pro 28% Dec 2024 Official leaderboard
24 Llama 4 Scout 26% Apr 2025 Official leaderboard
25 Codestral [snapshot unspecified] 24.5% Official leaderboard
26 Llama 3.3 70B 23.5% Dec 2024 Official leaderboard
27 Nvidia 70B Instruct [snapshot unspecified] 22.5% Official leaderboard
28 Llama 3.1-405B 20.5% Jul 2024 Official leaderboard
29 Amazon Nova Lite 19% Dec 2024 Official leaderboard
30 Amazon Nova Micro 17.5% Dec 2024 Official leaderboard
31 Command R+ 15% Aug 2024 Official leaderboard
32 Command-Light 10.5% Official leaderboard
33 Command 10% Official leaderboard

2 models have held number one on the official leaderboard since its release in September 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Not tested since Mar 2026
GPT-5.4 84.5%

Source: Official SnorkelGraph leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) GPT-5.4 o4-mini Amazon Nova Pro Llama 3.1-405B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) GPT-5.4 o4-mini

Source: Official SnorkelGraph leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Math

All 113 in Math
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • MathArena Apex Shortlist In use Models solve 47 hard final-answer problems that MathArena selected from 2025 competitions, where the best models scored about 50% at selection. Scored by accuracy over repeated attempts. GPT-5.5 98.4%