Benchmark Almanac

AA-LCR Extended

GPT-5.6 Sol 79.5% leads since August 2026
Not in model cards

Models answer questions that need reasoning across about 100k tokens of documents. Mercor adds 89 new expert-written tasks to the 100 public ones and reports both scores.

Held-out extension of aa-lcr-artificial-analysis-long-context-reasoning. Mercor shows the original and extended score for each model. The month is approximate. The Wayback Machine first saved the earliest extended pages on August 15th, 2026.

Made by
Mercor
Released
August 2026
Kind
Reasoning over long documents
Level
Expert
Scored
Checked automatically
Answers
Short answer
Tasks
Public, plus a private test set
Strongest model tested
Claude Opus 5.5 #4 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
51
Entries on its own leaderboard, 49 models we track

GPT-5.6 Sol leads with 79.5%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

51 entries. Scroll sideways for the rest.

  1. 79.5 GPT-5.6 Sol
  2. 76.8 GLM-5.3
  3. 74.7 Gemini 4 Argon
  4. 74.4 GPT-5.6 Terra
  5. 74.4 Muse Wonder
  6. 74.2 Claude Opus 5.5
  7. 73.9 Claude Opus 5
  8. 73.8 Claude Sonnet 5.5
  9. 73.3 Qwen 3.8 Max
  10. 73.0 Claude Fable 5.1
  11. 72.2 GPT-5.6 Luna
  12. 71.1 Claude Fable 5
  13. 70.8 DeepSeek V4.1 Flash
  14. 70.5 GPT-5.5
  15. 70.2 GPT-6 Sol
  16. 69.1 Kimi K3
  17. 68.3 GPT-5.4
  18. 68.3 GPT-6 Luna
  19. 68.2 Grok 4.7
  20. 66.9 GLM-5.3-Flash
  21. 66.9 Muse Spark 1.3
  22. 65.2 Grok 4.6
  23. 63.8 Claude Haiku 5.5
  24. 63.2 Claude Opus 4.8
  25. 62.9 DeepSeek V4 Pro 0813
  26. 61.8 DeepSeek-V4-Flash
  27. 61.8 Claude Sonnet 5
  28. 61.8 Muse Spark Open
  29. 61.5 Muse Spark 1.2
  30. 60.4 Grok 4.5
  31. 60.1 Beam
  32. 59.8 Claude Opus 4.7
  33. 58.1 Claude Sonnet 4.6
  34. 55.6 Gemini 3.8 Flash
  35. 50.6 Inkling
  36. 49.2 Kimi K2.7 Code
  37. 48.9 Gemini 3.7 Flash
  38. 46.3 Gemini 3.6 Flash
  39. 45.5 MiniMax-M3
  40. 42.7 GLM-5.2
  41. 41.6 Nemotron 3 Ultra
  42. 39.9 Gemini 3.5 Flash
  43. 37.9 GLM-5.1
  44. 37.6 DeepSeek-V4-Pro
  45. 35.4 Gemini 3.1 Pro
  46. 33.4 Qwen3.5
  47. 32.9 Kimi K2
  48. 28.4 MiniMax-M2.7
  49. 25.8 gpt-oss-120b
  50. 24.7 DeepSeek-V3.2
  51. 22.7 Gemma 4 31B IT
  1. 1 GPT-5.6 Sol max 79.5%
  2. 2 GLM-5.3 max 76.8%
  3. 3 Gemini 4 Argon high 74.7%
  4. 4 GPT-5.6 Terra max 74.4%
  5. 5 Muse Wonder max 74.4%
  6. 6 Claude Opus 5.5 max 74.2%
  7. 7 Claude Opus 5 max 73.9%
  8. 8 Claude Sonnet 5.5 max 73.8%
  9. 9 Qwen 3.8 Max xhigh 73.3%
  10. 10 Claude Fable 5.1 high 73%
  11. 11 GPT-5.6 Luna max 72.2%
  12. 12 Claude Fable 5 max 71.1%
  13. 13 DeepSeek V4.1 Flash max 70.8%
  14. 14 GPT-5.5 xhigh 70.5%
  15. 15 GPT-6 Sol max 70.2%
  16. 16 Kimi K3 max 69.1%
  17. 17 GPT-5.4 xhigh 68.3%
  18. 18 GPT-6 Luna max 68.3%
  19. 19 Grok 4.7 xhigh 68.2%
  20. 20 GLM-5.3-Flash max 66.9%
  21. 21 Muse Spark 1.3 max 66.9%
  22. 22 Grok 4.6 xhigh 65.2%
  23. 23 Claude Haiku 5.5 max 63.8%
  24. 24 Claude Opus 4.8 max 63.2%
  25. 25 DeepSeek V4 Pro 0813 max 62.9%
  26. 26 DeepSeek-V4-Flash max 61.8%
  27. 27 Claude Sonnet 5 max 61.8%
  28. 28 Muse Spark Open xhigh 61.8%
  29. 29 Muse Spark 1.2 xhigh 61.5%
  30. 30 Grok 4.5 high 60.4%
  31. 31 Beam max 60.1%
  32. 32 Claude Opus 4.7 max 59.8%
  33. 33 Claude Sonnet 4.6 high 58.1%
  34. 34 Gemini 3.8 Flash high 55.6%
  35. 35 Inkling high 50.6%
  36. 36 Kimi K2.7 Code high 49.2%
  37. 37 Gemini 3.7 Flash high 48.9%
  38. 38 Gemini 3.6 Flash high 46.3%
  39. 39 MiniMax-M3 high 45.5%
  40. 40 GLM-5.2 max 42.7%
  41. 41 Nemotron 3 Ultra high 41.6%
  42. 42 Gemini 3.5 Flash high 39.9%
  43. 43 GLM-5.1 37.9%
  44. 44 DeepSeek-V4-Pro max 37.6%
  45. 45 Gemini 3.1 Pro high 35.4%
  46. 46 Qwen3.5 33.4%
  47. 47 Kimi K2 Sep 2025, high 32.9%
  48. 48 MiniMax-M2.7 high 28.4%
  49. 49 gpt-oss-120b high 25.8%
  50. 50 DeepSeek-V3.2 24.7%
  51. 51 Gemma 4 31B IT 22.7%

Source: Official AA-LCR Extended leaderboard

#EntryScoreSettingDateSource
1 GPT-5.6 Sol 79.5% max Jul 2026 Official leaderboard
2 GLM-5.3 76.8% max Aug 2026 Official leaderboard
3 Gemini 4 Argon 74.7% high Sep 2026 Official leaderboard
4 GPT-5.6 Terra 74.4% max Jul 2026 Official leaderboard
5 Muse Wonder 74.4% max Oct 2026 Official leaderboard
6 Claude Opus 5.5 74.2% max Sep 2026 Official leaderboard
7 Claude Opus 5 73.9% max Jul 2026 Official leaderboard
8 Claude Sonnet 5.5 73.8% max Sep 2026 Official leaderboard
9 Qwen 3.8 Max 73.3% xhigh Aug 2026 Official leaderboard
10 Claude Fable 5.1 73% high Sep 2026 Official leaderboard
11 GPT-5.6 Luna 72.2% max Jul 2026 Official leaderboard
12 Claude Fable 5 71.1% max Jun 2026 Official leaderboard
13 DeepSeek V4.1 Flash 70.8% max Sep 2026 Official leaderboard
14 GPT-5.5 70.5% xhigh Apr 2026 Official leaderboard
15 GPT-6 Sol 70.2% max Sep 2026 Official leaderboard
16 Kimi K3 69.1% max Jul 2026 Official leaderboard
17 GPT-5.4 68.3% xhigh Mar 2026 Official leaderboard
18 GPT-6 Luna 68.3% max Sep 2026 Official leaderboard
19 Grok 4.7 68.2% xhigh Sep 2026 Official leaderboard
20 GLM-5.3-Flash 66.9% max Aug 2026 Official leaderboard
21 Muse Spark 1.3 66.9% max Sep 2026 Official leaderboard
22 Grok 4.6 65.2% xhigh Aug 2026 Official leaderboard
23 Claude Haiku 5.5 63.8% max Oct 2026 Official leaderboard
24 Claude Opus 4.8 63.2% max May 2026 Official leaderboard
25 DeepSeek V4 Pro 0813 62.9% max Aug 2026 Official leaderboard
26 DeepSeek-V4-Flash 61.8% max Apr 2026 Official leaderboard
27 Claude Sonnet 5 61.8% max Jun 2026 Official leaderboard
28 Muse Spark Open 61.8% xhigh Oct 2026 Official leaderboard
29 Muse Spark 1.2 61.5% xhigh Aug 2026 Official leaderboard
30 Grok 4.5 60.4% high Jul 2026 Official leaderboard
31 Beam 60.1% max Official leaderboard
32 Claude Opus 4.7 59.8% max Apr 2026 Official leaderboard
33 Claude Sonnet 4.6 58.1% high Feb 2026 Official leaderboard
34 Gemini 3.8 Flash 55.6% high Sep 2026 Official leaderboard
35 Inkling 50.6% high Jul 2026 Official leaderboard
36 Kimi K2.7 Code 49.2% high Jun 2026 Official leaderboard
37 Gemini 3.7 Flash 48.9% high Aug 2026 Official leaderboard
38 Gemini 3.6 Flash 46.3% high Jul 2026 Official leaderboard
39 MiniMax-M3 45.5% high Jun 2026 Official leaderboard
40 GLM-5.2 42.7% max Jun 2026 Official leaderboard
41 Nemotron 3 Ultra 41.6% high Jun 2026 Official leaderboard
42 Gemini 3.5 Flash 39.9% high May 2026 Official leaderboard
43 GLM-5.1 37.9% Apr 2026 Official leaderboard
44 DeepSeek-V4-Pro 37.6% max Apr 2026 Official leaderboard
45 Gemini 3.1 Pro 35.4% high Feb 2026 Official leaderboard
46 Qwen3.5 33.4% Official leaderboard
47 Kimi K2 Sep 2025 32.9% high Sep 2025 Official leaderboard
48 MiniMax-M2.7 28.4% high Mar 2026 Official leaderboard
49 gpt-oss-120b 25.8% high Aug 2025 Official leaderboard
50 DeepSeek-V3.2 24.7% Dec 2025 Official leaderboard
51 Gemma 4 31B IT 22.7% Apr 2026 Official leaderboard

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard 1 leader
GPT-5.6 Sol 79.5%

Source: Official AA-LCR Extended leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.6 Sol Claude Fable 5 GPT-5.4 Claude Sonnet 4.6 Kimi K2 gpt-oss-120b 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.6 Sol Claude Fable 5

Source: Official AA-LCR Extended leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Long context

All 79 in Long context
  • MRCR v2 In use Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens. Muse Spark 1.3 98.5%
  • LongBench v2 In use 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy. Inkling 2 Small 70.6%
  • GraphWalks In use A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1. Claude Fable 5 99.96%
  • OpenAI MRCR v2 In use Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin. Gemini 3.7 Flash 89%
  • CL-bench In use Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. GPT-5.4 27.9%
  • AA-LCR 1.1 In use Models answer 100 questions that need reasoning across sets of real documents of about 100K tokens. An LLM equality checker with a fixed system prompt compares each answer with the key. Step 5 Preview 88.3%