Benchmark Almanac

GraphWalks

Claude Fable 5 99.96% leads since June 2026
In use

A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1.

On February 27th, 2026 OpenAI fixed wrong ground truth in 24 of 400 parents samples under 128K. It also reworded the BFS prompt to ask only for nodes at the exact depth. The name stayed, so results before and after may differ slightly.

Made by
OpenAI
Released
April 2025
Kind
Reasoning over long documents
Level
University
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
Gemini 4 Carbon #2 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
27
Models scored in lab cards
23
Launch posts list it, from 8 labs
15
System cards report it, from 6 labs

Claude Fable 5 leads with 99.96%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

27 models. Scroll sideways for the rest.

  1. 99.96 Claude Fable 5
  2. 99.7 Gemini 4 Argon
  3. 99.3 Claude Opus 4.8
  4. 96.2 Gemini 4 Carbon
  5. 93.0 GPT-5.4
  6. 90.7 GPT-5.6 Sol
  7. 90.0 MAI-Thinking-1
  8. 89.7 Nemotron 3.5 Super VL
  9. 83.3 Seed2.0 Pro
  10. 82.3 Gemma 4 31B IT
  11. 81.3 GPT-5.6 Luna
  12. 80.0 Claude Mythos Preview
  13. 76.9 GPT-5.6 Terra
  14. 76.3 GPT-5.4 Mini
  15. 73.7 GPT-5.5
  16. 73.4 GPT-5 mini
  17. 73.4 GPT-5.4 Nano
  18. 72.6 Gemma 4 26B A4B
  19. 64.0 GPT-5 nano
  20. 63.2 Spark-X2.6-4B
  21. 61.7 GPT-4.1
  22. 61.7 GPT-4.1 mini
  23. 56.0 LongCat-Flash-Omni
  24. 51.0 LongCat-Flash-Chat
  25. 50.9 Gemma 4 E4B
  26. 25.0 GPT-4.1 nano
  27. 4.10 Gemma 4 E2B
  1. 1 Claude Fable 5 99.96%
  2. 2 Gemini 4 Argon 99.7%
  3. 3 Claude Opus 4.8 99.3%
  4. 4 Gemini 4 Carbon 96.2%
  5. 5 GPT-5.4 93%
  6. 6 GPT-5.6 Sol 90.7%
  7. 7 MAI-Thinking-1 90%
  8. 8 Nemotron 3.5 Super VL 89.7%
  9. 9 Seed2.0 Pro 83.3%
  10. 10 Gemma 4 31B IT 82.3%
  11. 11 GPT-5.6 Luna 81.3%
  12. 12 Claude Mythos Preview 80%
  13. 13 GPT-5.6 Terra 76.9%
  14. 14 GPT-5.4 Mini 76.3%
  15. 15 GPT-5.5 73.7%
  16. 16 GPT-5 mini 73.4%
  17. 17 GPT-5.4 Nano 73.4%
  18. 18 Gemma 4 26B A4B 72.6%
  19. 19 GPT-5 nano 64%
  20. 20 Spark-X2.6-4B 63.2%
  21. 21 GPT-4.1 61.7%
  22. 22 GPT-4.1 mini 61.7%
  23. 23 LongCat-Flash-Omni 56%
  24. 24 LongCat-Flash-Chat 51%
  25. 25 Gemma 4 E4B 50.9%
  26. 26 GPT-4.1 nano 25%
  27. 27 Gemma 4 E2B 4.10%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 Claude Fable 5 99.96% Jun 2026 Lab model card
2 Gemini 4 Argon 99.7% Oct 2026 Lab model card
3 Claude Opus 4.8 99.3% May 2026 Lab model card
4 Gemini 4 Carbon 96.2% Oct 2026 Lab model card
5 GPT-5.4 93% Mar 2026 Lab model card
6 GPT-5.6 Sol 90.7% Jul 2026 Lab model card
7 MAI-Thinking-1 90% Jun 2026 Lab model card
8 Nemotron 3.5 Super VL 89.7% Oct 2026 Lab model card
9 Seed2.0 Pro 83.3% Feb 2026 Lab model card
10 Gemma 4 31B IT 82.3% Jul 2026 Lab model card
11 GPT-5.6 Luna 81.3% Jul 2026 Lab model card
12 Claude Mythos Preview 80% Apr 2026 Lab model card
13 GPT-5.6 Terra 76.9% Jul 2026 Lab model card
14 GPT-5.4 Mini 76.3% Mar 2026 Lab model card
15 GPT-5.5 73.7% Apr 2026 Lab model card
16 GPT-5 mini 73.4% Aug 2025 Lab model card
17 GPT-5.4 Nano 73.4% Mar 2026 Lab model card
18 Gemma 4 26B A4B 72.6% Jul 2026 Lab model card
19 GPT-5 nano 64% Aug 2025 Lab model card
20 Spark-X2.6-4B 63.2% Oct 2026 Lab model card
21 GPT-4.1 61.7% Apr 2025 Lab model card
22 GPT-4.1 mini 61.7% Apr 2025 Lab model card
23 LongCat-Flash-Omni 56% Oct 2025 Lab model card
24 LongCat-Flash-Chat 51% Aug 2025 Lab model card
25 Gemma 4 E4B 50.9% Jul 2026 Lab model card
26 GPT-4.1 nano 25% Apr 2025 Lab model card
27 Gemma 4 E2B 4.10% Jul 2026 Lab model card

6 models have held number one in labs’ own numbers since its release in April 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI ByteDance Seed Anthropic
Lab claims Passed 85% in Mar 2026, by GPT-5.4
Claude Fable 5 99.96%

Source: Labs’ model cards

OpenAI’s own number for GPT-5.4 passed 85% in March 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Claude Fable 5 GPT-5.4 Seed2.0 Pro GPT-5 mini GPT-4.1 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Claude Fable 5 GPT-5.4

Source: Labs’ model cards

6 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

OpenAI
Meituan
ByteDance Seed
Anthropic
Microsoft
Google DeepMind
NVIDIA
iFlytek
DateLaunch postScoreAs the post writes it
Oct 28, 2026 Gemini 4 Carbon 96.2 GraphWalks (256k to 1M, BFS (F1))
Oct 22, 2026 Spark-X2.6-4B 63.2 Mean of Graphwalks bfs 256k f1 92.2 / Graphwalks bfs 1m f1 34.2
Oct 15, 2026 Nemotron 3.5 Super VL 89.7 GraphWalks
Oct 13, 2026 Gemini 4 Argon 99.7 Best of GraphWalks, Up to 128k, BFS (F1) / GraphWalks, 256k to 1M, BFS (F1)
Jul 9, 2026 GPT-5.6 Luna 81.3 GraphWalks BFS 256k f1 81.3 (1mil f1 51.2)
Jul 9, 2026 GPT-5.6 Terra 76.9 GraphWalks BFS 256k f1 76.9 (1mil f1 71.2)
Jul 9, 2026 GPT-5.6 Sol 90.7 GraphWalks BFS 256K F1 90.7% (1M F1 77.1%)
Jun 9, 2026 Claude Fable 5 100 Best of GraphWalks BFS 256K / GraphWalks Parents 256K
Jun 2, 2026 MAI-Thinking-1 90 GraphWalks (≤128k)
May 28, 2026 Claude Opus 4.8 99.3 Best of GraphWalks BFS 256K / GraphWalks Parents 256K
Apr 23, 2026 GPT-5.5 73.7 GraphWalks BFS 256K 73.7% (1M 45.4%)
Apr 7, 2026 Claude Mythos Preview 80 GraphWalks BFS 256K-1M
Mar 17, 2026 GPT-5.4 Nano 73.4 Graphwalks BFS 0K-128K
Mar 17, 2026 GPT-5.4 Mini 76.3 GraphWalks BFS, 0K-128K
Mar 5, 2026 GPT-5.4 93 GraphWalks BFS 0K-128K (21.4% at 256K-1M)
Feb 14, 2026 Seed2.0 Pro 83.3 Mean of Graphwalks Bfs (<128k) 68.9 / Graphwalks Parents (<128k) 97.6
Oct 31, 2025 LongCat-Flash-Omni 56 GraphWalks-128k(precision)
Aug 29, 2025 LongCat-Flash-Chat 51 GraphWalks-128k(precision)
Aug 7, 2025 GPT-5 nano 64 Graphwalks bfs <128k 64.0 (parents <128k 43.8)
Aug 7, 2025 GPT-5 mini 73.4 Graphwalks bfs <128k 73.4 (parents <128k 64.3)
Apr 14, 2025 GPT-4.1 nano 25 Graphwalks bfs <128k 25.0 (parents <128k 9.4)
Apr 14, 2025 GPT-4.1 mini 61.7 Graphwalks bfs <128k 61.7 (parents <128k 60.5)
Apr 14, 2025 GPT-4.1 61.7 Best of Graphwalks bfs <128k / Graphwalks bfs >128k / Graphwalks parents <128k / Graphwalks parents >128k

15 system cards report it, from 6 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 15, 2026 Nemotron 3.5 Super VL technical report 89.7 GraphWalks (<=1M), BFS and parents, 950 prompts, F1 (Table 10) Capabilities
Oct 13, 2026 Gemini 4 Argon model card 99.7 GraphWalks, up to 128k, BFS (F1) (results table, p.5). 256k to 1M BFS (F1) = 84.2% Capabilities
Jul 24, 2026 Gemma 4 E4B technical report 50.9 GraphWalks F1, under 128k, without thinking (Table 9) Capabilities
Jul 24, 2026 Gemma 4 E2B technical report 4.1 GraphWalks F1, under 128k, without thinking (Table 9) Capabilities
Jul 24, 2026 Gemma 4 31B IT technical report 82.3 GraphWalks F1, <128k, without thinking (Table 9) Capabilities
Jul 24, 2026 Gemma 4 26B A4B technical report 72.6 GraphWalks F1, <128k, without thinking (Table 9) Capabilities
Jun 2, 2026 MAI-Thinking-1 technical report 90 GraphWalks (<=128k), F1 (Table 12, rounded) Capabilities
May 28, 2026 Claude Opus 4.8 system card GraphWalks F1, five trials, no overall: BFS 256K 85.9, Parents 256K 99.3, BFS 1M 68.1, Parents 1M 83.3 (Table 8.1.A, Table 8.9.A) Capabilities
Apr 16, 2026 Claude Opus 4.7 system card GraphWalks F1, no overall: BFS 256K-1M 58.6, parents 256K-1M 75.1, 256K subset BFS 76.91 and parents 93.57 (8.7.1) Capabilities
Apr 7, 2026 Claude Mythos Preview system card GraphWalks F1, BFS 256K-1M 80.0 and parents 256K-1M 97.7, average of 5 trials, no overall (Table 6.3.A, 6.9) Capabilities
Feb 17, 2026 Claude Sonnet 4.6 system card GraphWalks F1 at max effort, BFS 1M 73.8, BFS 256K subset 74.5, Parents 1M 86.4 (row label garbled in table), Parents 256K subset 97.9, no overall (Table 2.16.B) Capabilities
Feb 14, 2026 Seed2.0 Pro model card Graphwalks Bfs (<128k) 68.9, Graphwalks Parents (<128k) 97.6, no overall, in-house tokenization (Table 3) Capabilities
Feb 5, 2026 Claude Opus 4.6 system card GraphWalks F1 at max effort, BFS 1M 38.7, BFS 256K subset 61.1, Parents 1M 72.0, Parents 256K subset 95.4, no overall (Table 2.18.A) Capabilities
Oct 31, 2025 LongCat-Flash-Omni technical report 56 GraphWalks-128K (precision) (Table 13) Capabilities
Sep 1, 2025 LongCat-Flash-Chat technical report 51 GraphWalks-128k (precision) (Table 3) Capabilities

2 logged changes that move scores

Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.

  1. Grader changed September 14th, 2026

    OpenAI corrected the published answer parser and F1 grading code. Parse failures and nonoverlapping answers now receive zero, while matching empty sets receive full credit, changing scores from the earlier code. Source

  2. Questions corrected February 27th, 2026

    OpenAI corrected ground truth for 24 of 400 short-context parents examples and clarified exact-depth breadth-first search prompts. Earlier dataset revisions can yield different answers and scores. Source

More in Long context

All 79 in Long context
  • MRCR v2 In use Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens. Muse Spark 1.3 98.5%
  • LongBench v2 In use 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy. Inkling 2 Small 70.6%
  • OpenAI MRCR v2 In use Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin. Gemini 3.7 Flash 89%
  • CL-bench In use Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. GPT-5.4 27.9%
  • AA-LCR 1.1 In use Models answer 100 questions that need reasoning across sets of real documents of about 100K tokens. An LLM equality checker with a fixed system prompt compares each answer with the key. Step 5 Preview 88.3%
  • LOCA-Bench In use Agentic tasks whose environment state is scaled to grow context length while task semantics stay fixed. Success rate versus context length. GPT-5.2 21.3%