Benchmark Almanac

GraphWalks

Claude Fable 5 99.96% leads since June 2026
In use

A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1.

On February 27th, 2026 OpenAI fixed wrong ground truth in 24 of 400 parents samples under 128K. It also reworded the BFS prompt to ask only for nodes at the exact depth. The name stayed, so results before and after may differ slightly.

Made by
OpenAI
Released
April 2025
Kind
Reasoning over long documents
Level
University
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Gemini 4 Carbon #3 on the capability index
Last in a launch post
November 2026
Last model added
November 2026
29
Models scored in lab cards
25
Launch posts list it, from 9 labs
18
System cards report it, from 7 labs

Claude Fable 5 leads with 99.96%

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

  1. 1 Claude Fable 5 99.96%
  2. 2 Gemini 4 Argon 99.7%
  3. 3 Claude Opus 4.8 99.3%
  4. 4 Gemini 4 Carbon 96.2%
  5. 5 Gemini 4 Neon 95.2%
  6. 6 GPT-5.4 93%
  7. 7 GPT-5.6 Sol 90.7%
  8. 8 MAI-Thinking-1 90%
  9. 9 Nemotron 3.5 Super VL 89.7%
  10. 10 Seed2.0 Pro 83.3%
  11. 11 Gemma 4 31B IT 82.3%
  12. 12 GPT-5.6 Luna 81.3%
  13. 13 Claude Mythos Preview 80%
  14. 14 GPT-5.6 Terra 76.9%
  15. 15 GPT-5.4 Mini 76.3%
  16. 16 GPT-5.5 73.7%
  17. 17 GPT-5 mini 73.4%
  18. 18 GPT-5.4 Nano 73.4%
  19. 19 Gemma 4 26B A4B 72.6%
  20. 20 Inkling 2 Mini 69.7%
  21. 21 GPT-5 nano 64%
  22. 22 Spark-X2.6-4B 63.2%
  23. 23 GPT-4.1 61.7%
  24. 24 GPT-4.1 mini 61.7%
  25. 25 LongCat-Flash-Omni 56%
  26. 26 LongCat-Flash-Chat 51%
  27. 27 Gemma 4 E4B 50.9%
  28. 28 GPT-4.1 nano 25%
  29. 29 Gemma 4 E2B 4.10%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 Claude Fable 5 99.96% Jun 2026 Lab model card
2 Gemini 4 Argon 99.7% Oct 2026 Lab model card
3 Claude Opus 4.8 99.3% May 2026 Lab model card
4 Gemini 4 Carbon 96.2% Oct 2026 Lab model card
5 Gemini 4 Neon 95.2% Nov 2026 Lab model card
6 GPT-5.4 93% Mar 2026 Lab model card
7 GPT-5.6 Sol 90.7% Jul 2026 Lab model card
8 MAI-Thinking-1 90% Jun 2026 Lab model card
9 Nemotron 3.5 Super VL 89.7% Oct 2026 Lab model card
10 Seed2.0 Pro 83.3% Feb 2026 Lab model card
11 Gemma 4 31B IT 82.3% Jul 2026 Lab model card
12 GPT-5.6 Luna 81.3% Jul 2026 Lab model card
13 Claude Mythos Preview 80% Apr 2026 Lab model card
14 GPT-5.6 Terra 76.9% Jul 2026 Lab model card
15 GPT-5.4 Mini 76.3% Mar 2026 Lab model card
16 GPT-5.5 73.7% Apr 2026 Lab model card
17 GPT-5 mini 73.4% Aug 2025 Lab model card
18 GPT-5.4 Nano 73.4% Mar 2026 Lab model card
19 Gemma 4 26B A4B 72.6% Jul 2026 Lab model card
20 Inkling 2 Mini 69.7% Nov 2026 Lab model card
21 GPT-5 nano 64% Aug 2025 Lab model card
22 Spark-X2.6-4B 63.2% Oct 2026 Lab model card
23 GPT-4.1 61.7% Apr 2025 Lab model card
24 GPT-4.1 mini 61.7% Apr 2025 Lab model card
25 LongCat-Flash-Omni 56% Oct 2025 Lab model card
26 LongCat-Flash-Chat 51% Aug 2025 Lab model card
27 Gemma 4 E4B 50.9% Jul 2026 Lab model card
28 GPT-4.1 nano 25% Apr 2025 Lab model card
29 Gemma 4 E2B 4.10% Jul 2026 Lab model card

6 models have held number one in labs’ own numbers since its release in April 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI ByteDance Seed Anthropic
Lab claims Passed 85% in Mar 2026, by GPT-5.4

Source: Labs’ model cards

OpenAI’s own number for GPT-5.4 passed 85% in March 2026

That was 11 months after its release. By the same age, 20% of the benchmarks from 2025 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Claude Fable 5 GPT-5.4 Seed2.0 Pro GPT-5 mini GPT-4.1 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Claude Fable 5 GPT-5.4

Source: Labs’ model cards

7 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

OpenAI
Meituan
ByteDance Seed
Anthropic
Microsoft
Google DeepMind
NVIDIA
iFlytek
Thinking Machines
DateLaunch postScoreAs the post writes it
Nov 20, 2026 Inkling 2 Mini 69.7 GraphWalks BFS 256K
Nov 18, 2026 Gemini 4 Neon 95.2 GraphWalks (256k to 1M, BFS (F1))
Oct 28, 2026 Gemini 4 Carbon 96.2 GraphWalks (256k to 1M, BFS (F1))
Oct 22, 2026 Spark-X2.6-4B 63.2 Mean of Graphwalks bfs 256k f1 92.2 / Graphwalks bfs 1m f1 34.2
Oct 15, 2026 Nemotron 3.5 Super VL 89.7 GraphWalks
Oct 13, 2026 Gemini 4 Argon 99.7 Best of GraphWalks, Up to 128k, BFS (F1) / GraphWalks, 256k to 1M, BFS (F1)
Jul 9, 2026 GPT-5.6 Luna 81.3 GraphWalks BFS 256k f1 81.3 (1mil f1 51.2)
Jul 9, 2026 GPT-5.6 Terra 76.9 GraphWalks BFS 256k f1 76.9 (1mil f1 71.2)
Jul 9, 2026 GPT-5.6 Sol 90.7 GraphWalks BFS 256K F1 90.7% (1M F1 77.1%)
Jun 9, 2026 Claude Fable 5 100 Best of GraphWalks BFS 256K / GraphWalks Parents 256K
Jun 2, 2026 MAI-Thinking-1 90 GraphWalks (≤128k)
May 28, 2026 Claude Opus 4.8 99.3 Best of GraphWalks BFS 256K / GraphWalks Parents 256K
Apr 23, 2026 GPT-5.5 73.7 GraphWalks BFS 256K 73.7% (1M 45.4%)
Apr 7, 2026 Claude Mythos Preview 80 GraphWalks BFS 256K-1M
Mar 17, 2026 GPT-5.4 Nano 73.4 Graphwalks BFS 0K-128K
Mar 17, 2026 GPT-5.4 Mini 76.3 GraphWalks BFS, 0K-128K
Mar 5, 2026 GPT-5.4 93 GraphWalks BFS 0K-128K (21.4% at 256K-1M)
Feb 14, 2026 Seed2.0 Pro 83.3 Mean of Graphwalks Bfs (<128k) 68.9 / Graphwalks Parents (<128k) 97.6
Oct 31, 2025 LongCat-Flash-Omni 56 GraphWalks-128k(precision)
Aug 29, 2025 LongCat-Flash-Chat 51 GraphWalks-128k(precision)
Aug 7, 2025 GPT-5 nano 64 Graphwalks bfs <128k 64.0 (parents <128k 43.8)
Aug 7, 2025 GPT-5 mini 73.4 Graphwalks bfs <128k 73.4 (parents <128k 64.3)
Apr 14, 2025 GPT-4.1 nano 25 Graphwalks bfs <128k 25.0 (parents <128k 9.4)
Apr 14, 2025 GPT-4.1 mini 61.7 Graphwalks bfs <128k 61.7 (parents <128k 60.5)
Apr 14, 2025 GPT-4.1 61.7 Best of Graphwalks bfs <128k / Graphwalks bfs >128k / Graphwalks parents <128k / Graphwalks parents >128k

18 system cards report it, from 7 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Nov 20, 2026 Inkling 2 Mini model card 69.7 GraphWalks BFS 256K Capabilities
Nov 18, 2026 Gemini 4 Neon model card 95.2 GraphWalks (256k to 1M, BFS (F1)) (results table, p.4) Capabilities
Nov 10, 2026 Gemini 4 Carbon model card 96.2 GraphWalks (256k to 1M, BFS (F1)) (results table) Capabilities
Oct 15, 2026 Nemotron 3.5 Super VL technical report 89.7 GraphWalks (<=1M), BFS and parents, 950 prompts, F1 (Table 10) Capabilities
Oct 13, 2026 Gemini 4 Argon model card 99.7 GraphWalks, up to 128k, BFS (F1) (results table, p.5). 256k to 1M BFS (F1) = 84.2% Capabilities
Jul 24, 2026 Gemma 4 E4B technical report 50.9 GraphWalks F1, under 128k, without thinking (Table 9) Capabilities
Jul 24, 2026 Gemma 4 E2B technical report 4.1 GraphWalks F1, under 128k, without thinking (Table 9) Capabilities
Jul 24, 2026 Gemma 4 31B IT technical report 82.3 GraphWalks F1, <128k, without thinking (Table 9) Capabilities
Jul 24, 2026 Gemma 4 26B A4B technical report 72.6 GraphWalks F1, <128k, without thinking (Table 9) Capabilities
Jun 2, 2026 MAI-Thinking-1 technical report 90 GraphWalks (<=128k), F1 (Table 12, rounded) Capabilities
May 28, 2026 Claude Opus 4.8 system card GraphWalks F1, five trials, no overall: BFS 256K 85.9, Parents 256K 99.3, BFS 1M 68.1, Parents 1M 83.3 (Table 8.1.A, Table 8.9.A) Capabilities
Apr 16, 2026 Claude Opus 4.7 system card GraphWalks F1, no overall: BFS 256K-1M 58.6, parents 256K-1M 75.1, 256K subset BFS 76.91 and parents 93.57 (8.7.1) Capabilities
Apr 7, 2026 Claude Mythos Preview system card GraphWalks F1, BFS 256K-1M 80.0 and parents 256K-1M 97.7, average of 5 trials, no overall (Table 6.3.A, 6.9) Capabilities
Feb 17, 2026 Claude Sonnet 4.6 system card GraphWalks F1 at max effort, BFS 1M 73.8, BFS 256K subset 74.5, Parents 1M 86.4 (row label garbled in table), Parents 256K subset 97.9, no overall (Table 2.16.B) Capabilities
Feb 14, 2026 Seed2.0 Pro model card Graphwalks Bfs (<128k) 68.9, Graphwalks Parents (<128k) 97.6, no overall, in-house tokenization (Table 3) Capabilities
Feb 5, 2026 Claude Opus 4.6 system card GraphWalks F1 at max effort, BFS 1M 38.7, BFS 256K subset 61.1, Parents 1M 72.0, Parents 256K subset 95.4, no overall (Table 2.18.A) Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 56 GraphWalks-128K (precision) (Table 13) Capabilities
Sep 1, 2025 LongCat-Flash-Chat technical report 51 GraphWalks-128k (precision) (Table 3) Capabilities

2 logged changes that move scores

Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.

  1. Grader changed September 14th, 2026

    OpenAI corrected the published answer parser and F1 grading code. Parse failures and nonoverlapping answers now receive zero, while matching empty sets receive full credit, changing scores from the earlier code. Source

  2. Questions corrected February 27th, 2026

    OpenAI corrected ground truth for 24 of 400 short-context parents examples and clarified exact-depth breadth-first search prompts. Earlier dataset revisions can yield different answers and scores. Source

More in Long context

All 115 in Long context
  • MRCR v2 In use Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens. Muse Spark 1.3 98.5%
  • LongBench v2 In use 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy. Inkling 2 Small 70.6%
  • OpenAI MRCR v2 In use Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin. Gemini 3.7 Flash 89%
  • CL-bench In use Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. GPT-5.4 27.9%
  • AA-LCR 1.1 In use Models answer 100 questions that need reasoning across sets of real documents of about 100K tokens. An LLM equality checker with a fixed system prompt compares each answer with the key. Step 5 Preview 88.3%
  • LOCA-Bench In use Agentic tasks whose environment state is scaled to grow context length while task semantics stay fixed. Success rate versus context length. GPT-5.2 21.3%

Data audit

All 5 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
  • Note Independent runs Only labs’ own numbers, so no one else has checked them. Scores from launch posts (25 scores) and system cards (12 scores).
  • Pass Models scored 29 models we track.
  • Pass Newest score The newest is from November 2026, for Inkling 2 Mini.
  • Note Frontier models 2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Gemini 4 Carbon (#3).
  • Note Headroom Saturated. The best is 99.96%, past 85% since March 2026.
Leaderboard import 1 note
  • Note Importer No public leaderboard to read.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check