Benchmark Almanac

LongBench Pro

Gemini 2.5 Pro Jun 2025 73.4% leads since January 2026
Not in model cards

Models answer English and Chinese questions over documents from 8,000 to 256,000 tokens. The benchmark applies task-specific metrics to 1,500 examples.

The authors publish results for 46 models. Prime Intellect independently ran the English subset.

Made by
Ziyang Chen and colleagues
Released
January 2026
Kind
Reasoning over long documents
Level
University
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Measures
Capability
Language
English, Chinese
In and out
Text → Text
Interaction
Single turn
Items from
Another benchmark
Run by
Anyone
Based on
LongBench
Strongest model tested
GPT-5 #71 on the capability index
Last model added
December 2025
Scores refreshed
November 21st, 2026
45
Entries in its paper, 36 models we track

Gemini 2.5 Pro leads with 73.4% among models released by December 2025 Gemini 2.5 Pro leads with 73.4% among models released by December 2025

Best score per entry from the benchmark's paper, read on November 21st, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

The newest model with a score here came out in December 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Gemini 2.5 Pro Jun 2025, thinking; context 1000000 73.4%
  2. 2 GPT-5 thinking; context 272000 72.6%
  3. 3 Claude Sonnet 4 thinking; context 1000000 69.9%
  4. 4 DeepSeek-V3.2 thinking; context 120000 67.8%
  5. 5 Gemini 2.5 Flash Sep 2025, thinking; context 1000000 67.4%
  6. 6 Qwen3-235B-A22B-Thinking Jul 2025, thinking; context 224000 67%
  7. 7 DeepSeek-V3.1 thinking; context 120000 66.2%
  8. 8 Qwen3-Next-80B-A3B thinking; context 224000 64%
  9. 9 Qwen3-235B-A22B-Instruct Jul 2025, thinking; context 224000 63.8%
  10. 10 DeepSeek-R1 May 2025, thinking; context 120000 61.9%
  11. 11 Qwen3-Next-80B-A3B-Instruct thinking; context 224000 60.8%
  12. 12 Qwen3-30B-A3B-Thinking Jul 2025, thinking; context 224000 59.7%
  13. 13 Claude 3.7 Sonnet thinking; context 200000 59.7%
  14. 14 GLM-4.6 thinking; context 120000 58.2%
  15. 15 DeepSeek-V3 Mar 2025, thinking; context 120000 56.7%
  16. 16 Kimi K2 Sep 2025, thinking; context 224000 55.5%
  17. 17 GLM-4.5 thinking; context 120000 55.5%
  18. 18 Qwen3-30B-A3B-Instruct Jul 2025, thinking; context 224000 54.5%
  19. 19 MiniMax-M2 thinking; context 1000000 53.2%
  20. 20 gpt-oss-120b thinking; context 120000 52.6%
  21. 21 Qwen3-32B thinking; context 120000 51.1%
  22. 22 Qwen3-4B-Thinking-2507 thinking; context 224000 50.1%
  23. 23 GPT-4o Mar 2025, thinking; context 120000 49.4%
  24. 24 Qwen3-14B thinking; context 120000 47.1%
  25. 25 Ministral-3-14B-Instruct-2512 thinking; context 224000 45.8%
  26. 26 qwen3-4b-instruct-2507 thinking; context 224000 45.7%
  27. 27 MiniMax-Text-01 thinking; context 1000000 45%
  28. 28 gpt-oss-20b thinking; context 120000 44.7%
  29. 29 Ministral 3 8B Instruct thinking; context 224000 44.5%
  30. 30 Qwen3-8B thinking; context 120000 44.3%
  31. 31 Qwen2.5-72B thinking; context 120000 44.1%
  32. 32 Qwen3-4B thinking; context 120000 40.8%
  33. 33 Llama-3.1-405B-Instruct thinking; context 120000 40.7%
  34. 34 Mistral-Small-3.2-24B-Instruct-2506 thinking; context 120000 39.9%
  35. 35 Magistral-Small-2509 thinking; context 120000 38.4%
  36. 36 Gemma 3 27B thinking; context 120000 37.3%
  37. 37 Mistral-Large-Instruct-2411 thinking; context 120000 36.3%
  38. 38 Ministral 3 3B Instruct thinking; context 224000 34.5%
  39. 39 Llama 3.3 70B Instruct thinking; context 120000 33.7%
  40. 40 Gemma 3 12B non-thinking; context 120000 32.2%
  41. 41 Llama-3.1-70B-Instruct thinking; context 120000 32.1%
  42. 42 Gemma 3 4B non-thinking; context 120000 21.8%
  43. 43 Llama-3.1-8B-Instruct non-thinking; context 120000 21.1%
  44. 44 Ministral-8B-Instruct-2410 non-thinking; context 120000 17.6%
  45. 45 Llama-3.2-3B-Instruct non-thinking; context 120000 15.7%

Source: The LongBench Pro paper

  1. 1 Gemini 2.5 Pro Jun 2025, thinking; context 1000000 73.4%
  2. 2 GPT-5 thinking; context 272000 72.6%
  3. 3 Claude Sonnet 4 thinking; context 1000000 69.9%
  4. 4 DeepSeek-V3.2 thinking; context 120000 67.8%
  5. 5 Gemini 2.5 Flash Sep 2025, thinking; context 1000000 67.4%
  6. 6 Qwen3-235B-A22B-Thinking Jul 2025, thinking; context 224000 67%
  7. 7 DeepSeek-V3.1 thinking; context 120000 66.2%
  8. 8 Qwen3-Next-80B-A3B thinking; context 224000 64%
  9. 9 Qwen3-235B-A22B-Instruct Jul 2025, thinking; context 224000 63.8%
  10. 10 DeepSeek-R1 May 2025, thinking; context 120000 61.9%
  11. 11 Qwen3-Next-80B-A3B-Instruct thinking; context 224000 60.8%
  12. 12 Qwen3-30B-A3B-Thinking Jul 2025, thinking; context 224000 59.7%
  13. 13 Claude 3.7 Sonnet thinking; context 200000 59.7%
  14. 14 GLM-4.6 thinking; context 120000 58.2%
  15. 15 DeepSeek-V3 Mar 2025, thinking; context 120000 56.7%
  16. 16 Kimi K2 Sep 2025, thinking; context 224000 55.5%
  17. 17 GLM-4.5 thinking; context 120000 55.5%
  18. 18 Qwen3-30B-A3B-Instruct Jul 2025, thinking; context 224000 54.5%
  19. 19 MiniMax-M2 thinking; context 1000000 53.2%
  20. 20 gpt-oss-120b thinking; context 120000 52.6%
  21. 21 Qwen3-32B thinking; context 120000 51.1%
  22. 22 Qwen3-4B-Thinking-2507 thinking; context 224000 50.1%
  23. 23 GPT-4o Mar 2025, thinking; context 120000 49.4%
  24. 24 Qwen3-14B thinking; context 120000 47.1%
  25. 25 qwen3-4b-instruct-2507 thinking; context 224000 45.7%
  26. 26 MiniMax-Text-01 thinking; context 1000000 45%
  27. 27 gpt-oss-20b thinking; context 120000 44.7%
  28. 28 Ministral 3 8B Instruct thinking; context 224000 44.5%
  29. 29 Qwen3-8B thinking; context 120000 44.3%
  30. 30 Qwen2.5-72B thinking; context 120000 44.1%
  31. 31 Qwen3-4B thinking; context 120000 40.8%
  32. 32 Gemma 3 27B thinking; context 120000 37.3%
  33. 33 Ministral 3 3B Instruct thinking; context 224000 34.5%
  34. 34 Llama 3.3 70B Instruct thinking; context 120000 33.7%
  35. 35 Gemma 3 12B non-thinking; context 120000 32.2%
  36. 36 Gemma 3 4B non-thinking; context 120000 21.8%

Source: The LongBench Pro paper

#EntryScoreSettingReleasedRun onSource
1 Gemini 2.5 Pro Jun 2025 73.4% thinking; context 1000000 Jun 2025 Dec 2025 Official leaderboard
2 GPT-5 72.6% thinking; context 272000 Aug 2025 Dec 2025 Official leaderboard
3 Claude Sonnet 4 69.9% thinking; context 1000000 May 2025 Dec 2025 Official leaderboard
4 DeepSeek-V3.2 67.8% thinking; context 120000 Dec 2025 Dec 2025 Official leaderboard
5 Gemini 2.5 Flash Sep 2025 67.4% thinking; context 1000000 Sep 2025 Dec 2025 Official leaderboard
6 Qwen3-235B-A22B-Thinking Jul 2025 67% thinking; context 224000 Jul 2025 Dec 2025 Official leaderboard
7 DeepSeek-V3.1 66.2% thinking; context 120000 Aug 2025 Dec 2025 Official leaderboard
8 Qwen3-Next-80B-A3B 64% thinking; context 224000 Sep 2025 Dec 2025 Official leaderboard
9 Qwen3-235B-A22B-Instruct Jul 2025 63.8% thinking; context 224000 Jul 2025 Dec 2025 Official leaderboard
10 DeepSeek-R1 May 2025 61.9% thinking; context 120000 May 2025 Dec 2025 Official leaderboard
11 Qwen3-Next-80B-A3B-Instruct 60.8% thinking; context 224000 Sep 2025 Dec 2025 Official leaderboard
12 Qwen3-30B-A3B-Thinking Jul 2025 59.7% thinking; context 224000 Jul 2025 Dec 2025 Official leaderboard
13 Claude 3.7 Sonnet 59.7% thinking; context 200000 Feb 2025 Dec 2025 Official leaderboard
14 GLM-4.6 58.2% thinking; context 120000 Sep 2025 Dec 2025 Official leaderboard
15 DeepSeek-V3 Mar 2025 56.7% thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard
16 Kimi K2 Sep 2025 55.5% thinking; context 224000 Sep 2025 Dec 2025 Official leaderboard
17 GLM-4.5 55.5% thinking; context 120000 Aug 2025 Dec 2025 Official leaderboard
18 Qwen3-30B-A3B-Instruct Jul 2025 54.5% thinking; context 224000 Jul 2025 Dec 2025 Official leaderboard
19 MiniMax-M2 53.2% thinking; context 1000000 Oct 2025 Dec 2025 Official leaderboard
20 gpt-oss-120b 52.6% thinking; context 120000 Aug 2025 Dec 2025 Official leaderboard
21 Qwen3-32B 51.1% thinking; context 120000 Apr 2025 Dec 2025 Official leaderboard
22 Qwen3-4B-Thinking-2507 50.1% thinking; context 224000 Aug 2025 Dec 2025 Official leaderboard
23 GPT-4o Mar 2025 49.4% thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard
24 Qwen3-14B 47.1% thinking; context 120000 Apr 2025 Dec 2025 Official leaderboard
25 Ministral-3-14B-Instruct-2512 45.8% thinking; context 224000 Dec 2025 Official leaderboard
26 qwen3-4b-instruct-2507 45.7% thinking; context 224000 Aug 2025 Dec 2025 Official leaderboard
27 MiniMax-Text-01 45% thinking; context 1000000 Jan 2025 Dec 2025 Official leaderboard
28 gpt-oss-20b 44.7% thinking; context 120000 Aug 2025 Dec 2025 Official leaderboard
29 Ministral 3 8B Instruct 44.5% thinking; context 224000 Dec 2025 Dec 2025 Official leaderboard
30 Qwen3-8B 44.3% thinking; context 120000 Apr 2025 Dec 2025 Official leaderboard
31 Qwen2.5-72B 44.1% thinking; context 120000 Sep 2024 Dec 2025 Official leaderboard
32 Qwen3-4B 40.8% thinking; context 120000 Apr 2025 Dec 2025 Official leaderboard
33 Llama-3.1-405B-Instruct 40.7% thinking; context 120000 Dec 2025 Official leaderboard
34 Mistral-Small-3.2-24B-Instruct-2506 39.9% thinking; context 120000 Dec 2025 Official leaderboard
35 Magistral-Small-2509 38.4% thinking; context 120000 Dec 2025 Official leaderboard
36 Gemma 3 27B 37.3% thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard
37 Mistral-Large-Instruct-2411 36.3% thinking; context 120000 Dec 2025 Official leaderboard
38 Ministral 3 3B Instruct 34.5% thinking; context 224000 Dec 2025 Dec 2025 Official leaderboard
39 Llama 3.3 70B Instruct 33.7% thinking; context 120000 Dec 2024 Dec 2025 Official leaderboard
40 Gemma 3 12B 32.2% non-thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard
41 Llama-3.1-70B-Instruct 32.1% thinking; context 120000 Dec 2025 Official leaderboard
42 Gemma 3 4B 21.8% non-thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard
43 Llama-3.1-8B-Instruct 21.1% non-thinking; context 120000 Dec 2025 Official leaderboard
44 Ministral-8B-Instruct-2410 17.6% non-thinking; context 120000 Dec 2025 Official leaderboard
45 Llama-3.2-3B-Instruct 15.7% non-thinking; context 120000 Dec 2025 Official leaderboard
#ModelScoreSettingReleasedRun onSource
1 Gemini 2.5 Pro Jun 2025 73.4% thinking; context 1000000 Jun 2025 Dec 2025 Official leaderboard
2 GPT-5 72.6% thinking; context 272000 Aug 2025 Dec 2025 Official leaderboard
3 Claude Sonnet 4 69.9% thinking; context 1000000 May 2025 Dec 2025 Official leaderboard
4 DeepSeek-V3.2 67.8% thinking; context 120000 Dec 2025 Dec 2025 Official leaderboard
5 Gemini 2.5 Flash Sep 2025 67.4% thinking; context 1000000 Sep 2025 Dec 2025 Official leaderboard
6 Qwen3-235B-A22B-Thinking Jul 2025 67% thinking; context 224000 Jul 2025 Dec 2025 Official leaderboard
7 DeepSeek-V3.1 66.2% thinking; context 120000 Aug 2025 Dec 2025 Official leaderboard
8 Qwen3-Next-80B-A3B 64% thinking; context 224000 Sep 2025 Dec 2025 Official leaderboard
9 Qwen3-235B-A22B-Instruct Jul 2025 63.8% thinking; context 224000 Jul 2025 Dec 2025 Official leaderboard
10 DeepSeek-R1 May 2025 61.9% thinking; context 120000 May 2025 Dec 2025 Official leaderboard
11 Qwen3-Next-80B-A3B-Instruct 60.8% thinking; context 224000 Sep 2025 Dec 2025 Official leaderboard
12 Qwen3-30B-A3B-Thinking Jul 2025 59.7% thinking; context 224000 Jul 2025 Dec 2025 Official leaderboard
13 Claude 3.7 Sonnet 59.7% thinking; context 200000 Feb 2025 Dec 2025 Official leaderboard
14 GLM-4.6 58.2% thinking; context 120000 Sep 2025 Dec 2025 Official leaderboard
15 DeepSeek-V3 Mar 2025 56.7% thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard
16 Kimi K2 Sep 2025 55.5% thinking; context 224000 Sep 2025 Dec 2025 Official leaderboard
17 GLM-4.5 55.5% thinking; context 120000 Aug 2025 Dec 2025 Official leaderboard
18 Qwen3-30B-A3B-Instruct Jul 2025 54.5% thinking; context 224000 Jul 2025 Dec 2025 Official leaderboard
19 MiniMax-M2 53.2% thinking; context 1000000 Oct 2025 Dec 2025 Official leaderboard
20 gpt-oss-120b 52.6% thinking; context 120000 Aug 2025 Dec 2025 Official leaderboard
21 Qwen3-32B 51.1% thinking; context 120000 Apr 2025 Dec 2025 Official leaderboard
22 Qwen3-4B-Thinking-2507 50.1% thinking; context 224000 Aug 2025 Dec 2025 Official leaderboard
23 GPT-4o Mar 2025 49.4% thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard
24 Qwen3-14B 47.1% thinking; context 120000 Apr 2025 Dec 2025 Official leaderboard
25 qwen3-4b-instruct-2507 45.7% thinking; context 224000 Aug 2025 Dec 2025 Official leaderboard
26 MiniMax-Text-01 45% thinking; context 1000000 Jan 2025 Dec 2025 Official leaderboard
27 gpt-oss-20b 44.7% thinking; context 120000 Aug 2025 Dec 2025 Official leaderboard
28 Ministral 3 8B Instruct 44.5% thinking; context 224000 Dec 2025 Dec 2025 Official leaderboard
29 Qwen3-8B 44.3% thinking; context 120000 Apr 2025 Dec 2025 Official leaderboard
30 Qwen2.5-72B 44.1% thinking; context 120000 Sep 2024 Dec 2025 Official leaderboard
31 Qwen3-4B 40.8% thinking; context 120000 Apr 2025 Dec 2025 Official leaderboard
32 Gemma 3 27B 37.3% thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard
33 Ministral 3 3B Instruct 34.5% thinking; context 224000 Dec 2025 Dec 2025 Official leaderboard
34 Llama 3.3 70B Instruct 33.7% thinking; context 120000 Dec 2024 Dec 2025 Official leaderboard
35 Gemma 3 12B 32.2% non-thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard
36 Gemma 3 4B 21.8% non-thinking; context 120000 Mar 2025 Dec 2025 Official leaderboard

Rankings by part

We read 12 parts from the benchmark’s paper in 3 groups: output length, contextual requirement and difficulty. Each ranks models on one part of the test. Gemini 2.5 Pro leads 7 of them, and 3 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

8k

36 models we track have a score here.

  1. 1 DeepSeek-V3.2 ↑3 75.5%
  2. 2 GPT-5 75.4%
  3. 3 DeepSeek-V3.1 ↑4 74.9%
  4. 4 Gemini 2.5 Pro Jun 2025 ↓3 74.5%
  5. 5 Claude Sonnet 4 ↓2 72.7%
  6. 6 Qwen3-235B-A22B-Thinking Jul 2025 72.1%
  7. 7 DeepSeek-R1 May 2025 ↑3 71.7%
  8. 8 Gemini 2.5 Flash Sep 2025 ↓3 71.4%
  9. 9 GLM-4.6 ↑5 71.2%
  10. 10 Qwen3-Next-80B-A3B ↓2 70.9%

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Sonnet 4
non-thinkin… thinking; c…
69.9%
DeepSeek-V3.2
non-thinkin… thinking; c…
67.8%
Gemini 2.5 Flash
non-thinkin… thinking; c…
67.4%
DeepSeek-V3.1
non-thinkin… thinking; c…
66.2%
Qwen3-235B-A22B-Instruct
non-thinkin… thinking; c…
63.8%
Qwen3-Next-80B-A3B-Instruct
non-thinkin… thinking; c…
60.8%
Claude 3.7 Sonnet
non-thinkin… thinking; c…
59.7%
GLM-4.6
non-thinkin… thinking; c…
58.2%
DeepSeek-V3
non-thinkin… thinking; c…
56.7%
Kimi K2
non-thinkin… thinking; c…
55.5%
GLM-4.5
non-thinkin… thinking; c…
55.5%
Qwen3-30B-A3B-Instruct
non-thinkin… thinking; c…
54.5%

Source: The LongBench Pro paper

ModelEffortHarnessScoreCost per taskOutput tokens
Gemini 2.5 Pro Jun 2025 thinking; context 1000000 73.4%
GPT-5 thinking; context 272000 72.6%
Claude Sonnet 4 thinking; context 1000000 69.9%
DeepSeek-V3.2 thinking; context 120000 67.8%
Gemini 2.5 Flash Sep 2025 thinking; context 1000000 67.4%
Qwen3-235B-A22B-Thinking Jul 2025 thinking; context 224000 67%
DeepSeek-V3.1 thinking; context 120000 66.2%
Qwen3-Next-80B-A3B thinking; context 224000 64%
Qwen3-235B-A22B-Instruct Jul 2025 thinking; context 224000 63.8%
DeepSeek-R1 May 2025 thinking; context 120000 61.9%
Qwen3-Next-80B-A3B-Instruct thinking; context 224000 60.8%
DeepSeek-R1 May 2025 thinking; context 120000 60.1%
Qwen3-30B-A3B-Thinking Jul 2025 thinking; context 224000 59.7%
Claude 3.7 Sonnet thinking; context 200000 59.7%
GLM-4.6 thinking; context 120000 58.2%
DeepSeek-V3 Mar 2025 thinking; context 120000 56.7%
Claude Sonnet 4 non-thinking; context 1000000 56.1%
Gemini 2.5 Flash Sep 2025 non-thinking; context 1000000 55.9%
Kimi K2 Sep 2025 thinking; context 224000 55.5%
GLM-4.5 thinking; context 120000 55.5%
Qwen3-30B-A3B-Instruct Jul 2025 thinking; context 224000 54.5%
MiniMax-M2 thinking; context 1000000 53.2%
gpt-oss-120b thinking; context 120000 52.6%
Qwen3-235B-A22B-Instruct Jul 2025 non-thinking; context 224000 52.5%
DeepSeek-V3 Mar 2025 non-thinking; context 120000 51.7%
DeepSeek-V3.2 non-thinking; context 120000 51.7%
Qwen3-Next-80B-A3B-Instruct non-thinking; context 224000 51.5%
Claude 3.7 Sonnet non-thinking; context 200000 51.5%
DeepSeek-V3.1 non-thinking; context 120000 51.4%
Qwen3-32B thinking; context 120000 51.1%
Qwen3-4B-Thinking-2507 thinking; context 224000 50.1%
Kimi K2 Sep 2025 non-thinking; context 224000 50.1%
GPT-4o Mar 2025 thinking; context 120000 49.4%
Qwen3-14B thinking; context 120000 47.1%
GPT-4o Mar 2025 non-thinking; context 120000 46.7%
GLM-4.6 non-thinking; context 120000 45.9%
qwen3-4b-instruct-2507 thinking; context 224000 45.7%
MiniMax-Text-01 thinking; context 1000000 45%
gpt-oss-20b thinking; context 120000 44.7%
Ministral 3 8B Instruct thinking; context 224000 44.5%
Qwen3-8B thinking; context 120000 44.3%
Qwen2.5-72B thinking; context 120000 44.1%
Qwen3-30B-A3B-Instruct Jul 2025 non-thinking; context 224000 43.8%
GLM-4.5 non-thinking; context 120000 43%
MiniMax-Text-01 non-thinking; context 1000000 41.1%
Qwen3-4B thinking; context 120000 40.8%
Qwen3-32B non-thinking; context 120000 40.3%
Qwen2.5-72B non-thinking; context 120000 39.6%
Ministral 3 8B Instruct non-thinking; context 224000 37.8%
Gemma 3 27B thinking; context 120000 37.3%
Qwen3-14B non-thinking; context 120000 37.1%
qwen3-4b-instruct-2507 non-thinking; context 224000 36.8%
Gemma 3 27B non-thinking; context 120000 36.1%
Ministral 3 3B Instruct thinking; context 224000 34.5%
Llama 3.3 70B Instruct thinking; context 120000 33.7%
Qwen3-8B non-thinking; context 120000 33.4%
Gemma 3 12B non-thinking; context 120000 32.2%
Gemma 3 12B thinking; context 120000 31.9%
Llama 3.3 70B Instruct non-thinking; context 120000 31.9%
Qwen3-4B non-thinking; context 120000 31.3%
Ministral 3 3B Instruct non-thinking; context 224000 30.2%
Gemma 3 4B non-thinking; context 120000 21.8%
Gemma 3 4B thinking; context 120000 21.2%

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Sonnet 4
non-thinkin… thinking; c…
69.9%
DeepSeek-V3.2
non-thinkin… thinking; c…
67.8%
Gemini 2.5 Flash
non-thinkin… thinking; c…
67.4%
DeepSeek-V3.1
non-thinkin… thinking; c…
66.2%
Qwen3-235B-A22B-Instruct
non-thinkin… thinking; c…
63.8%
Qwen3-Next-80B-A3B-Instruct
non-thinkin… thinking; c…
60.8%
Claude 3.7 Sonnet
non-thinkin… thinking; c…
59.7%
GLM-4.6
non-thinkin… thinking; c…
58.2%
DeepSeek-V3
non-thinkin… thinking; c…
56.7%
Kimi K2
non-thinkin… thinking; c…
55.5%
GLM-4.5
non-thinkin… thinking; c…
55.5%
Qwen3-30B-A3B-Instruct
non-thinkin… thinking; c…
54.5%

Source: The LongBench Pro paper

ModelEffortHarnessScoreCost per taskOutput tokens
Gemini 2.5 Pro Jun 2025 thinking; context 1000000 73.4%
GPT-5 thinking; context 272000 72.6%
Claude Sonnet 4 thinking; context 1000000 69.9%
DeepSeek-V3.2 thinking; context 120000 67.8%
Gemini 2.5 Flash Sep 2025 thinking; context 1000000 67.4%
Qwen3-235B-A22B-Thinking Jul 2025 thinking; context 224000 67%
DeepSeek-V3.1 thinking; context 120000 66.2%
Qwen3-Next-80B-A3B thinking; context 224000 64%
Qwen3-235B-A22B-Instruct Jul 2025 thinking; context 224000 63.8%
DeepSeek-R1 May 2025 thinking; context 120000 61.9%
Qwen3-Next-80B-A3B-Instruct thinking; context 224000 60.8%
DeepSeek-R1 May 2025 thinking; context 120000 60.1%
Qwen3-30B-A3B-Thinking Jul 2025 thinking; context 224000 59.7%
Claude 3.7 Sonnet thinking; context 200000 59.7%
GLM-4.6 thinking; context 120000 58.2%
DeepSeek-V3 Mar 2025 thinking; context 120000 56.7%
Claude Sonnet 4 non-thinking; context 1000000 56.1%
Gemini 2.5 Flash Sep 2025 non-thinking; context 1000000 55.9%
Kimi K2 Sep 2025 thinking; context 224000 55.5%
GLM-4.5 thinking; context 120000 55.5%
Qwen3-30B-A3B-Instruct Jul 2025 thinking; context 224000 54.5%
MiniMax-M2 thinking; context 1000000 53.2%
gpt-oss-120b thinking; context 120000 52.6%
Qwen3-235B-A22B-Instruct Jul 2025 non-thinking; context 224000 52.5%
DeepSeek-V3 Mar 2025 non-thinking; context 120000 51.7%
DeepSeek-V3.2 non-thinking; context 120000 51.7%
Qwen3-Next-80B-A3B-Instruct non-thinking; context 224000 51.5%
Claude 3.7 Sonnet non-thinking; context 200000 51.5%
DeepSeek-V3.1 non-thinking; context 120000 51.4%
Qwen3-32B thinking; context 120000 51.1%
Qwen3-4B-Thinking-2507 thinking; context 224000 50.1%
Kimi K2 Sep 2025 non-thinking; context 224000 50.1%
GPT-4o Mar 2025 thinking; context 120000 49.4%
Qwen3-14B thinking; context 120000 47.1%
GPT-4o Mar 2025 non-thinking; context 120000 46.7%
GLM-4.6 non-thinking; context 120000 45.9%
qwen3-4b-instruct-2507 thinking; context 224000 45.7%
MiniMax-Text-01 thinking; context 1000000 45%
gpt-oss-20b thinking; context 120000 44.7%
Ministral 3 8B Instruct thinking; context 224000 44.5%
Qwen3-8B thinking; context 120000 44.3%
Qwen2.5-72B thinking; context 120000 44.1%
Qwen3-30B-A3B-Instruct Jul 2025 non-thinking; context 224000 43.8%
GLM-4.5 non-thinking; context 120000 43%
MiniMax-Text-01 non-thinking; context 1000000 41.1%
Qwen3-4B thinking; context 120000 40.8%
Qwen3-32B non-thinking; context 120000 40.3%
Qwen2.5-72B non-thinking; context 120000 39.6%
Ministral 3 8B Instruct non-thinking; context 224000 37.8%
Gemma 3 27B thinking; context 120000 37.3%
Qwen3-14B non-thinking; context 120000 37.1%
qwen3-4b-instruct-2507 non-thinking; context 224000 36.8%
Gemma 3 27B non-thinking; context 120000 36.1%
Ministral 3 3B Instruct thinking; context 224000 34.5%
Llama 3.3 70B Instruct thinking; context 120000 33.7%
Qwen3-8B non-thinking; context 120000 33.4%
Gemma 3 12B non-thinking; context 120000 32.2%
Gemma 3 12B thinking; context 120000 31.9%
Llama 3.3 70B Instruct non-thinking; context 120000 31.9%
Qwen3-4B non-thinking; context 120000 31.3%
Ministral 3 3B Instruct non-thinking; context 224000 30.2%
Gemma 3 4B non-thinking; context 120000 21.8%
Gemma 3 4B thinking; context 120000 21.2%

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind
Benchmark paper Not tested since Jan 2026

Source: The LongBench Pro paper

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
In the benchmark’s paper Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Gemini 2.5 Pro Claude 3.7 Sonnet MiniMax-Text-01 Qwen2.5-72B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Gemini 2.5 Pro Claude 3.7 Sonnet

Source: Official LongBench Pro leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Long context

All 115 in Long context
  • MRCR v2 In use Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens. Muse Spark 1.3 98.5%
  • LongBench v2 In use 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy. Inkling 2 Small 70.6%
  • GraphWalks In use A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1. Claude Fable 5 99.96%
  • OpenAI MRCR v2 In use Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin. Gemini 3.7 Flash 89%
  • CL-bench In use Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. GPT-5.4 27.9%
  • AA-LCR 1.1 In use Models answer 100 questions that need reasoning across sets of real documents of about 100K tokens. An LLM equality checker with a fixed system prompt compares each answer with the key. Step 5 Preview 88.3%

Data audit

9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 11 months ago, in December 2025, for Ministral 3 3B Instruct. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
  • Pass Independent runs Scores from the benchmark’s paper (45 entries).
  • Pass Models scored 36 models we track, plus 9 other entries on its leaderboard.
  • Warning Newest score The newest came 11 months ago, in December 2025, for Ministral 3 3B Instruct.
  • Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5 (#71).
  • Pass Headroom The best is 73.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 63 rows for models we track and 9 other entries at the last read.
  • Pass Last read November 21st, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check