Benchmark Almanac

AA-LCR (Artificial Analysis Long Context Reasoning)

Muse Glimmer 80% leads since August 2026
In use

Questions requiring reasoning across multiple real documents totaling about 100K tokens. Answers judged by an LLM.

Version 1.0.0 stays available at dataset revision bdae010. Version 1.1 (September 2026) fixed 16 answer keys, so scores differ.

Released
August 2025
Kind
Reasoning over long documents
Level
University
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
Kimi K3 #24 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
38
Models scored in lab cards
36
Launch posts list it, from 13 labs
18
System cards report it, from 7 labs

Muse Glimmer leads with 80%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

38 models. Scroll sideways for the rest.

  1. 80.0 Muse Glimmer
  2. 79.3 Beam 1.0
  3. 74.7 Kimi K3
  4. 73.4 Tencent Hy3
  5. 72.0 MiniMax-M2.7
  6. 71.2 Mistral Small 4
  7. 71.0 Solar Pro 4
  8. 70.0 Kimi K2.5
  9. 69.5 MiniMax-M2.5
  10. 68.7 Qwen3-Max
  11. 68.7 Qwen3.5 397B-A17B
  12. 68.3 Qwen 3.6 Plus
  13. 68.0 Mercury 2.5
  14. 66.9 Qwen3.5-122B-A10B
  15. 66.3 Tencent Hy3 preview
  16. 66.1 Qwen3.5-27B
  17. 65.4 Nemotron 3 Ultra
  18. 64.1 Nemotron 3.5 Bolt 74B-A9B
  19. 63.9 Step 3.7 Flash
  20. 63.0 Qwen3.5-9B
  21. 62.3 Solar Open 2
  22. 62.0 MiniMax-M2.1
  23. 61.0 MiniMax-M2
  24. 58.5 Qwen3.5-35B-A3B
  25. 58.3 Nemotron 3 Super
  26. 57.0 Qwen3.5-4B
  27. 57.0 Qwen3.5-Omni-Plus
  28. 56.2 K-EXAONE 2.0
  29. 53.5 K-EXAONE
  30. 52.8 LFM3-1.7B
  31. 52.4 Nemotron 3.5 Lightning 30B-A3B
  32. 50.6 EXAONE 4.5 33B
  33. 41.0 Nemotron 3 Nano Omni 30B-A3B
  34. 39.1 Nemotron-Cascade 2 30B-A3B
  35. 35.9 Nemotron 3 Nano 30B-A3B
  36. 35.0 Solar Open 100B
  37. 25.6 Qwen3.5-2B
  38. 4.70 Qwen3.5-0.8B
  1. 1 Muse Glimmer 80%
  2. 2 Beam 1.0 79.3%
  3. 3 Kimi K3 74.7%
  4. 4 Tencent Hy3 73.4%
  5. 5 MiniMax-M2.7 72%
  6. 6 Mistral Small 4 71.2%
  7. 7 Solar Pro 4 71%
  8. 8 Kimi K2.5 70%
  9. 9 MiniMax-M2.5 69.5%
  10. 10 Qwen3-Max 68.7%
  11. 11 Qwen3.5 397B-A17B 68.7%
  12. 12 Qwen 3.6 Plus 68.3%
  13. 13 Mercury 2.5 68%
  14. 14 Qwen3.5-122B-A10B 66.9%
  15. 15 Tencent Hy3 preview 66.3%
  16. 16 Qwen3.5-27B 66.1%
  17. 17 Nemotron 3 Ultra 65.4%
  18. 18 Nemotron 3.5 Bolt 74B-A9B 64.1%
  19. 19 Step 3.7 Flash 63.9%
  20. 20 Qwen3.5-9B 63%
  21. 21 Solar Open 2 62.3%
  22. 22 MiniMax-M2.1 62%
  23. 23 MiniMax-M2 61%
  24. 24 Qwen3.5-35B-A3B 58.5%
  25. 25 Nemotron 3 Super 58.3%
  26. 26 Qwen3.5-4B 57%
  27. 27 Qwen3.5-Omni-Plus 57%
  28. 28 K-EXAONE 2.0 56.2%
  29. 29 K-EXAONE 53.5%
  30. 30 LFM3-1.7B 52.8%
  31. 31 Nemotron 3.5 Lightning 30B-A3B 52.4%
  32. 32 EXAONE 4.5 33B 50.6%
  33. 33 Nemotron 3 Nano Omni 30B-A3B 41%
  34. 34 Nemotron-Cascade 2 30B-A3B 39.1%
  35. 35 Nemotron 3 Nano 30B-A3B 35.9%
  36. 36 Solar Open 100B 35%
  37. 37 Qwen3.5-2B 25.6%
  38. 38 Qwen3.5-0.8B 4.70%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 Muse Glimmer 80% Aug 2026 Lab model card
2 Beam 1.0 79.3% Oct 2026 Lab model card
3 Kimi K3 74.7% Jul 2026 Lab model card
4 Tencent Hy3 73.4% Jul 2026 Lab model card
5 MiniMax-M2.7 72% May 2026 Lab model card
6 Mistral Small 4 71.2% Mar 2026 Lab model card
7 Solar Pro 4 71% Aug 2026 Lab model card
8 Kimi K2.5 70% Jan 2026 Lab model card
9 MiniMax-M2.5 69.5% Feb 2026 Lab model card
10 Qwen3-Max 68.7% Jan 2026 Lab model card
11 Qwen3.5 397B-A17B 68.7% Feb 2026 Lab model card
12 Qwen 3.6 Plus 68.3% Apr 2026 Lab model card
13 Mercury 2.5 68% Sep 2026 Lab model card
14 Qwen3.5-122B-A10B 66.9% Feb 2026 Lab model card
15 Tencent Hy3 preview 66.3% Apr 2026 Lab model card
16 Qwen3.5-27B 66.1% Feb 2026 Lab model card
17 Nemotron 3 Ultra 65.4% Jun 2026 Lab model card
18 Nemotron 3.5 Bolt 74B-A9B 64.1% Oct 2026 Lab model card
19 Step 3.7 Flash 63.9% May 2026 Lab model card
20 Qwen3.5-9B 63% Mar 2026 Lab model card
21 Solar Open 2 62.3% Jul 2026 Lab model card
22 MiniMax-M2.1 62% Dec 2025 Lab model card
23 MiniMax-M2 61% Oct 2025 Lab model card
24 Qwen3.5-35B-A3B 58.5% Feb 2026 Lab model card
25 Nemotron 3 Super 58.3% Mar 2026 Lab model card
26 Qwen3.5-4B 57% Mar 2026 Lab model card
27 Qwen3.5-Omni-Plus 57% Mar 2026 Lab model card
28 K-EXAONE 2.0 56.2% Jul 2026 Lab model card
29 K-EXAONE 53.5% Dec 2025 Lab model card
30 LFM3-1.7B 52.8% Oct 2026 Lab model card
31 Nemotron 3.5 Lightning 30B-A3B 52.4% Oct 2026 Lab model card
32 EXAONE 4.5 33B 50.6% Apr 2026 Lab model card
33 Nemotron 3 Nano Omni 30B-A3B 41% Apr 2026 Lab model card
34 Nemotron-Cascade 2 30B-A3B 39.1% Mar 2026 Lab model card
35 Nemotron 3 Nano 30B-A3B 35.9% Dec 2025 Lab model card
36 Solar Open 100B 35% Dec 2025 Lab model card
37 Qwen3.5-2B 25.6% Mar 2026 Lab model card
38 Qwen3.5-0.8B 4.70% Mar 2026 Lab model card

9 models have held number one in labs’ own numbers since its release in August 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

MiniMax Qwen Moonshot AI Mistral AI Tencent Meta
Lab claims 9 leaders
Muse Glimmer 80%

Source: Labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Muse Glimmer Kimi K3 Mistral Small 4 MiniMax-M2.1 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Muse Glimmer Tencent Hy3

Source: Labs’ model cards

9 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

MiniMax
NVIDIA
LG AI Research
Upstage
Qwen
Moonshot AI
Mistral AI
Tencent
StepFun
Meta
Inception Labs
Reflection
Liquid AI
DateLaunch postScoreAs the post writes it
Oct 27, 2026 LFM3-1.7B 52.8 AA-LCR
Oct 15, 2026 Nemotron 3.5 Bolt 74B-A9B 64.1 AA-LCR
Oct 5, 2026 Beam 1.0 79.3 AA-LCR
Sep 8, 2026 Mercury 2.5 68 AA-LCR (version not stated)
Aug 11, 2026 Nemotron 3.5 Lightning 30B-A3B 52 AA-LCR
Aug 11, 2026 Solar Pro 4 71 AA-LCR (Artificial Analysis score quoted by Upstage)
Aug 10, 2026 Muse Glimmer 80 AA-LCR
Jul 31, 2026 K-EXAONE 2.0 56.2 AA-LCR
Jul 22, 2026 Solar Open 2 62.3 AA-LCR
Jul 16, 2026 Kimi K3 74.7 AA-LCR
Jul 2, 2026 Tencent Hy3 73.4 AA-LCR
Jun 4, 2026 Nemotron 3 Ultra 65.4 AA-LCR
May 29, 2026 Step 3.7 Flash 63.9 AA-LCR (avg@16/acc)
Apr 23, 2026 Tencent Hy3 preview 66.3 AA-LCR
Apr 9, 2026 EXAONE 4.5 33B 50.6 AA-LCR
Apr 2, 2026 Qwen 3.6 Plus 68.3 AA-LCR
Mar 30, 2026 Qwen3.5-Omni-Plus 57 AA-LCR
Mar 19, 2026 Nemotron-Cascade 2 30B-A3B 39.1 AA-LCR
Mar 16, 2026 Mistral Small 4 71.2 LCR [Mistral Small 4 - High] (text: 'AA LCR ... 0.72')
Mar 11, 2026 Nemotron 3 Super 58.3 AA-LCR
Mar 2, 2026 Qwen3.5-0.8B 4.7 AA-LCR
Mar 2, 2026 Qwen3.5-2B 25.6 AA-LCR
Mar 2, 2026 Qwen3.5-4B 57 AA-LCR
Mar 2, 2026 Qwen3.5-9B 63 AA-LCR
Feb 24, 2026 Qwen3.5-35B-A3B 58.5 AA-LCR
Feb 24, 2026 Qwen3.5-27B 66.1 AA-LCR
Feb 24, 2026 Qwen3.5-122B-A10B 66.9 AA-LCR
Feb 16, 2026 Qwen3.5 397B-A17B 68.7 AA-LCR
Feb 12, 2026 MiniMax-M2.5 69.5 AA-LCR
Jan 27, 2026 Kimi K2.5 70 AA-LCR 70.0 (evaluation table)
Jan 23, 2026 Qwen3-Max 68.7 AA-LCR
Dec 31, 2025 Solar Open 100B 35 AA-LCR
Dec 31, 2025 K-EXAONE 53.5 AA-LCR
Dec 23, 2025 MiniMax-M2.1 62 AA-LCR
Dec 15, 2025 Nemotron 3 Nano 30B-A3B 35.9 AA-LCR
Oct 27, 2025 MiniMax-M2 61 AA-LCR

18 system cards report it, from 7 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 19, 2026 Muse Glimmer technical report 80 AA-LCR, high (Tables 2 and 5) Capabilities
Oct 15, 2026 Nemotron 3.5 Lightning 30B-A3B technical report 52.4 AA-LCR, BF16 acc. (Table 22) Capabilities
Oct 15, 2026 Nemotron 3.5 Bolt 74B-A9B technical report 64.1 AA-LCR, released 24GB checkpoint (Table 7). BF16 67.38 Capabilities
Aug 5, 2026 K-EXAONE 2.0 technical report 56.2 AA-LCR (Table 6, Reasoning mode) Capabilities
Jul 27, 2026 Kimi K3 technical report 74.7 AA-LCR, cited from Artificial Analysis (Table 2) Capabilities
Jul 22, 2026 Solar Open 2 technical report 62.3 AA-LCR (Table 3) Capabilities
Jun 9, 2026 Nemotron 3 Ultra technical report 65.4 AA-LCR (Table 10) Capabilities
May 26, 2026 MiniMax-M2 technical report 61 AA-LCR, MiniMax-M2 bar, printed label (Figure 9) Capabilities
May 26, 2026 MiniMax-M2.7 technical report 72 AA-LCR, AA Index v4.0 protocol (Table 4) Capabilities
May 26, 2026 MiniMax-M2.5 technical report 65 AA-LCR, AA Index v4.0 protocol (Table 4) Capabilities
Apr 27, 2026 Nemotron 3 Nano Omni 30B-A3B technical report 41 AA-LCR (Table 10) Capabilities
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 57 AA-LCR (Table 4) Capabilities
Apr 9, 2026 EXAONE 4.5 33B technical report 50.6 AA-LCR (Table 3, Reasoning mode) Capabilities
Mar 19, 2026 Nemotron-Cascade 2 30B-A3B technical report 39.1 AA-LCR (Table 1) Capabilities
Mar 11, 2026 Nemotron 3 Super technical report 58.3 AA-LCR (Table 5) Capabilities
Jan 11, 2026 Solar Open 100B technical report 35 AA-LCR (Table 6) Capabilities
Jan 5, 2026 K-EXAONE technical report 53.5 AA-LCR (Table 3, Reasoning mode). Non-reasoning 45.2 (Table 4) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B technical report 35.9 AA-LCR, reasoning on (Table 3) Capabilities

One logged change that moves scores

Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.

  1. Questions corrected November 26th, 2025

    Artificial Analysis fixed question-string encoding in the AA-LCR dataset. Evaluations using older revisions can present corrupted question text, so comparisons should identify the dataset revision. Source

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

MiniMax Qwen Moonshot AI Mistral AI Tencent Meta StepFun
AA-LCR 1.1 Passed 85% in Sep 2026, by Step 5 Preview
Step 5 Preview 88.3%
AA-LCR (Artificial Analysis Long Context Reasoning) 9 leaders
Muse Glimmer 80%
AA-LCR 1.1 Passed 85% in Sep 2026, by Step 5 Preview
Step 5 Preview 88.3%

Source: Labs’ model cards

Replaced by AA-LCR 1.1

More in Long context

All 79 in Long context
  • MRCR v2 In use Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens. Muse Spark 1.3 98.5%
  • LongBench v2 In use 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy. Inkling 2 Small 70.6%
  • GraphWalks In use A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1. Claude Fable 5 99.96%
  • OpenAI MRCR v2 In use Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin. Gemini 3.7 Flash 89%
  • CL-bench In use Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. GPT-5.4 27.9%
  • LOCA-Bench In use Agentic tasks whose environment state is scaled to grow context length while task semantics stay fixed. Success rate versus context length. GPT-5.2 21.3%