LongBench Pro
Models answer English and Chinese questions over documents from 8,000 to 256,000 tokens. The benchmark applies task-specific metrics to 1,500 examples.
The authors publish results for 46 models. Prime Intellect independently ran the English subset.
- Made by
- Ziyang Chen and colleagues
- Released
- January 2026
- Kind
- Reasoning over long documents
- Level
- University
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Measures
- Capability
- Interaction
- Single turn
- Items from
- Another benchmark
- Refresh
- New versions replace it
- Run by
- Anyone
- Based on
- LongBench
- Strongest model tested
- GPT-5 #71 on the capability index
- Last model added
- December 2025
- Scores refreshed
- November 21st, 2026
Gemini 2.5 Pro leads with 73.4% among models released by December 2025 Gemini 2.5 Pro leads with 73.4% among models released by December 2025
Best score per entry from the benchmark's paper, read on November 21st, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in December 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 73.4
Gemini 2.5 Pro
- 72.6 GPT-5
- 69.9 Claude Sonnet 4
- 67.8
DeepSeek-V3.2
- 67.4
Gemini 2.5 Flash
- 67.0
Qwen3-235B-A22B-Thinking
- 66.2
DeepSeek-V3.1
- 64.0
Qwen3-Next-80B-A3B
- 63.8
Qwen3-235B-A22B-Instruct
- 61.9
DeepSeek-R1
- 60.8
Qwen3-Next-80B-A3B-Instruct
- 59.7
Qwen3-30B-A3B-Thinking
- 59.7 Claude 3.7 Sonnet
- 58.2 GLM-4.6
- 56.7
DeepSeek-V3
- 55.5 Kimi K2
- 55.5 GLM-4.5
- 54.5
Qwen3-30B-A3B-Instruct
- 53.2
MiniMax-M2
- 52.6 gpt-oss-120b
- 51.1
Qwen3-32B
- 50.1
Qwen3-4B-Thinking-2507
- 49.4 GPT-4o
- 47.1
Qwen3-14B
- 45.8 Ministral-3-14B-Instruct-2512
- 45.7 qwen3-4b-instruct-2507
- 45.0
MiniMax-Text-01
- 44.7 gpt-oss-20b
- 44.5
Ministral 3 8B Instruct
- 44.3
Qwen3-8B
- 44.1
Qwen2.5-72B
- 40.8
Qwen3-4B
- 40.7 Llama-3.1-405B-Instruct
- 39.9 Mistral-Small-3.2-24B-Instruct-2506
- 38.4 Magistral-Small-2509
- 37.3
Gemma 3 27B
- 36.3 Mistral-Large-Instruct-2411
- 34.5
Ministral 3 3B Instruct
- 33.7
Llama 3.3 70B Instruct
- 32.2
Gemma 3 12B
- 32.1 Llama-3.1-70B-Instruct
- 21.8
Gemma 3 4B
- 21.1 Llama-3.1-8B-Instruct
- 17.6 Ministral-8B-Instruct-2410
- 15.7 Llama-3.2-3B-Instruct
- 1
Gemini 2.5 Pro Jun 2025, thinking; context 1000000 73.4%
- 2 GPT-5 thinking; context 272000 72.6%
- 3 Claude Sonnet 4 thinking; context 1000000 69.9%
- 4
DeepSeek-V3.2 thinking; context 120000 67.8%
- 5
Gemini 2.5 Flash Sep 2025, thinking; context 1000000 67.4%
- 6
Qwen3-235B-A22B-Thinking Jul 2025, thinking; context 224000 67%
- 7
DeepSeek-V3.1 thinking; context 120000 66.2%
- 8
Qwen3-Next-80B-A3B thinking; context 224000 64%
- 9
Qwen3-235B-A22B-Instruct Jul 2025, thinking; context 224000 63.8%
- 10
DeepSeek-R1 May 2025, thinking; context 120000 61.9%
- 11
Qwen3-Next-80B-A3B-Instruct thinking; context 224000 60.8%
- 12
Qwen3-30B-A3B-Thinking Jul 2025, thinking; context 224000 59.7%
- 13 Claude 3.7 Sonnet thinking; context 200000 59.7%
- 14 GLM-4.6 thinking; context 120000 58.2%
- 15
DeepSeek-V3 Mar 2025, thinking; context 120000 56.7%
- 16 Kimi K2 Sep 2025, thinking; context 224000 55.5%
- 17 GLM-4.5 thinking; context 120000 55.5%
- 18
Qwen3-30B-A3B-Instruct Jul 2025, thinking; context 224000 54.5%
- 19
MiniMax-M2 thinking; context 1000000 53.2%
- 20 gpt-oss-120b thinking; context 120000 52.6%
- 21
Qwen3-32B thinking; context 120000 51.1%
- 22
Qwen3-4B-Thinking-2507 thinking; context 224000 50.1%
- 23 GPT-4o Mar 2025, thinking; context 120000 49.4%
- 24
Qwen3-14B thinking; context 120000 47.1%
- 25 Ministral-3-14B-Instruct-2512 thinking; context 224000 45.8%
- 26 qwen3-4b-instruct-2507 thinking; context 224000 45.7%
- 27
MiniMax-Text-01 thinking; context 1000000 45%
- 28 gpt-oss-20b thinking; context 120000 44.7%
- 29
Ministral 3 8B Instruct thinking; context 224000 44.5%
- 30
Qwen3-8B thinking; context 120000 44.3%
- 31
Qwen2.5-72B thinking; context 120000 44.1%
- 32
Qwen3-4B thinking; context 120000 40.8%
- 33 Llama-3.1-405B-Instruct thinking; context 120000 40.7%
- 34 Mistral-Small-3.2-24B-Instruct-2506 thinking; context 120000 39.9%
- 35 Magistral-Small-2509 thinking; context 120000 38.4%
- 36
Gemma 3 27B thinking; context 120000 37.3%
- 37 Mistral-Large-Instruct-2411 thinking; context 120000 36.3%
- 38
Ministral 3 3B Instruct thinking; context 224000 34.5%
- 39
Llama 3.3 70B Instruct thinking; context 120000 33.7%
- 40
Gemma 3 12B non-thinking; context 120000 32.2%
- 41 Llama-3.1-70B-Instruct thinking; context 120000 32.1%
- 42
Gemma 3 4B non-thinking; context 120000 21.8%
- 43 Llama-3.1-8B-Instruct non-thinking; context 120000 21.1%
- 44 Ministral-8B-Instruct-2410 non-thinking; context 120000 17.6%
- 45 Llama-3.2-3B-Instruct non-thinking; context 120000 15.7%
Source: The LongBench Pro paper
- 73.4
Gemini 2.5 Pro
- 72.6 GPT-5
- 69.9 Claude Sonnet 4
- 67.8
DeepSeek-V3.2
- 67.4
Gemini 2.5 Flash
- 67.0
Qwen3-235B-A22B-Thinking
- 66.2
DeepSeek-V3.1
- 64.0
Qwen3-Next-80B-A3B
- 63.8
Qwen3-235B-A22B-Instruct
- 61.9
DeepSeek-R1
- 60.8
Qwen3-Next-80B-A3B-Instruct
- 59.7
Qwen3-30B-A3B-Thinking
- 59.7 Claude 3.7 Sonnet
- 58.2 GLM-4.6
- 56.7
DeepSeek-V3
- 55.5 Kimi K2
- 55.5 GLM-4.5
- 54.5
Qwen3-30B-A3B-Instruct
- 53.2
MiniMax-M2
- 52.6 gpt-oss-120b
- 51.1
Qwen3-32B
- 50.1
Qwen3-4B-Thinking-2507
- 49.4 GPT-4o
- 47.1
Qwen3-14B
- 45.7 qwen3-4b-instruct-2507
- 45.0
MiniMax-Text-01
- 44.7 gpt-oss-20b
- 44.5
Ministral 3 8B Instruct
- 44.3
Qwen3-8B
- 44.1
Qwen2.5-72B
- 40.8
Qwen3-4B
- 37.3
Gemma 3 27B
- 34.5
Ministral 3 3B Instruct
- 33.7
Llama 3.3 70B Instruct
- 32.2
Gemma 3 12B
- 21.8
Gemma 3 4B
- 1
Gemini 2.5 Pro Jun 2025, thinking; context 1000000 73.4%
- 2 GPT-5 thinking; context 272000 72.6%
- 3 Claude Sonnet 4 thinking; context 1000000 69.9%
- 4
DeepSeek-V3.2 thinking; context 120000 67.8%
- 5
Gemini 2.5 Flash Sep 2025, thinking; context 1000000 67.4%
- 6
Qwen3-235B-A22B-Thinking Jul 2025, thinking; context 224000 67%
- 7
DeepSeek-V3.1 thinking; context 120000 66.2%
- 8
Qwen3-Next-80B-A3B thinking; context 224000 64%
- 9
Qwen3-235B-A22B-Instruct Jul 2025, thinking; context 224000 63.8%
- 10
DeepSeek-R1 May 2025, thinking; context 120000 61.9%
- 11
Qwen3-Next-80B-A3B-Instruct thinking; context 224000 60.8%
- 12
Qwen3-30B-A3B-Thinking Jul 2025, thinking; context 224000 59.7%
- 13 Claude 3.7 Sonnet thinking; context 200000 59.7%
- 14 GLM-4.6 thinking; context 120000 58.2%
- 15
DeepSeek-V3 Mar 2025, thinking; context 120000 56.7%
- 16 Kimi K2 Sep 2025, thinking; context 224000 55.5%
- 17 GLM-4.5 thinking; context 120000 55.5%
- 18
Qwen3-30B-A3B-Instruct Jul 2025, thinking; context 224000 54.5%
- 19
MiniMax-M2 thinking; context 1000000 53.2%
- 20 gpt-oss-120b thinking; context 120000 52.6%
- 21
Qwen3-32B thinking; context 120000 51.1%
- 22
Qwen3-4B-Thinking-2507 thinking; context 224000 50.1%
- 23 GPT-4o Mar 2025, thinking; context 120000 49.4%
- 24
Qwen3-14B thinking; context 120000 47.1%
- 25 qwen3-4b-instruct-2507 thinking; context 224000 45.7%
- 26
MiniMax-Text-01 thinking; context 1000000 45%
- 27 gpt-oss-20b thinking; context 120000 44.7%
- 28
Ministral 3 8B Instruct thinking; context 224000 44.5%
- 29
Qwen3-8B thinking; context 120000 44.3%
- 30
Qwen2.5-72B thinking; context 120000 44.1%
- 31
Qwen3-4B thinking; context 120000 40.8%
- 32
Gemma 3 27B thinking; context 120000 37.3%
- 33
Ministral 3 3B Instruct thinking; context 224000 34.5%
- 34
Llama 3.3 70B Instruct thinking; context 120000 33.7%
- 35
Gemma 3 12B non-thinking; context 120000 32.2%
- 36
Gemma 3 4B non-thinking; context 120000 21.8%
Source: The LongBench Pro paper
| # | Entry | Score | Setting | Released | Run on | Source |
|---|---|---|---|---|---|---|
| 1 | | 73.4% | thinking; context 1000000 | Jun 2025 | Dec 2025 | Official leaderboard |
| 2 | GPT-5 | 72.6% | thinking; context 272000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 3 | Claude Sonnet 4 | 69.9% | thinking; context 1000000 | May 2025 | Dec 2025 | Official leaderboard |
| 4 | | 67.8% | thinking; context 120000 | Dec 2025 | Dec 2025 | Official leaderboard |
| 5 | | 67.4% | thinking; context 1000000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 6 | | 67% | thinking; context 224000 | Jul 2025 | Dec 2025 | Official leaderboard |
| 7 | | 66.2% | thinking; context 120000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 8 | | 64% | thinking; context 224000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 9 | | 63.8% | thinking; context 224000 | Jul 2025 | Dec 2025 | Official leaderboard |
| 10 | | 61.9% | thinking; context 120000 | May 2025 | Dec 2025 | Official leaderboard |
| 11 | | 60.8% | thinking; context 224000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 12 | | 59.7% | thinking; context 224000 | Jul 2025 | Dec 2025 | Official leaderboard |
| 13 | Claude 3.7 Sonnet | 59.7% | thinking; context 200000 | Feb 2025 | Dec 2025 | Official leaderboard |
| 14 | GLM-4.6 | 58.2% | thinking; context 120000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 15 | | 56.7% | thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
| 16 | Kimi K2 Sep 2025 | 55.5% | thinking; context 224000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 17 | GLM-4.5 | 55.5% | thinking; context 120000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 18 | | 54.5% | thinking; context 224000 | Jul 2025 | Dec 2025 | Official leaderboard |
| 19 | | 53.2% | thinking; context 1000000 | Oct 2025 | Dec 2025 | Official leaderboard |
| 20 | gpt-oss-120b | 52.6% | thinking; context 120000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 21 | | 51.1% | thinking; context 120000 | Apr 2025 | Dec 2025 | Official leaderboard |
| 22 | | 50.1% | thinking; context 224000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 23 | GPT-4o Mar 2025 | 49.4% | thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
| 24 | | 47.1% | thinking; context 120000 | Apr 2025 | Dec 2025 | Official leaderboard |
| 25 | Ministral-3-14B-Instruct-2512 | 45.8% | thinking; context 224000 | Dec 2025 | Official leaderboard | |
| 26 | qwen3-4b-instruct-2507 | 45.7% | thinking; context 224000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 27 | | 45% | thinking; context 1000000 | Jan 2025 | Dec 2025 | Official leaderboard |
| 28 | gpt-oss-20b | 44.7% | thinking; context 120000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 29 | | 44.5% | thinking; context 224000 | Dec 2025 | Dec 2025 | Official leaderboard |
| 30 | | 44.3% | thinking; context 120000 | Apr 2025 | Dec 2025 | Official leaderboard |
| 31 | | 44.1% | thinking; context 120000 | Sep 2024 | Dec 2025 | Official leaderboard |
| 32 | | 40.8% | thinking; context 120000 | Apr 2025 | Dec 2025 | Official leaderboard |
| 33 | Llama-3.1-405B-Instruct | 40.7% | thinking; context 120000 | Dec 2025 | Official leaderboard | |
| 34 | Mistral-Small-3.2-24B-Instruct-2506 | 39.9% | thinking; context 120000 | Dec 2025 | Official leaderboard | |
| 35 | Magistral-Small-2509 | 38.4% | thinking; context 120000 | Dec 2025 | Official leaderboard | |
| 36 | | 37.3% | thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
| 37 | Mistral-Large-Instruct-2411 | 36.3% | thinking; context 120000 | Dec 2025 | Official leaderboard | |
| 38 | | 34.5% | thinking; context 224000 | Dec 2025 | Dec 2025 | Official leaderboard |
| 39 | | 33.7% | thinking; context 120000 | Dec 2024 | Dec 2025 | Official leaderboard |
| 40 | | 32.2% | non-thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
| 41 | Llama-3.1-70B-Instruct | 32.1% | thinking; context 120000 | Dec 2025 | Official leaderboard | |
| 42 | | 21.8% | non-thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
| 43 | Llama-3.1-8B-Instruct | 21.1% | non-thinking; context 120000 | Dec 2025 | Official leaderboard | |
| 44 | Ministral-8B-Instruct-2410 | 17.6% | non-thinking; context 120000 | Dec 2025 | Official leaderboard | |
| 45 | Llama-3.2-3B-Instruct | 15.7% | non-thinking; context 120000 | Dec 2025 | Official leaderboard |
| # | Model | Score | Setting | Released | Run on | Source |
|---|---|---|---|---|---|---|
| 1 | | 73.4% | thinking; context 1000000 | Jun 2025 | Dec 2025 | Official leaderboard |
| 2 | GPT-5 | 72.6% | thinking; context 272000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 3 | Claude Sonnet 4 | 69.9% | thinking; context 1000000 | May 2025 | Dec 2025 | Official leaderboard |
| 4 | | 67.8% | thinking; context 120000 | Dec 2025 | Dec 2025 | Official leaderboard |
| 5 | | 67.4% | thinking; context 1000000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 6 | | 67% | thinking; context 224000 | Jul 2025 | Dec 2025 | Official leaderboard |
| 7 | | 66.2% | thinking; context 120000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 8 | | 64% | thinking; context 224000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 9 | | 63.8% | thinking; context 224000 | Jul 2025 | Dec 2025 | Official leaderboard |
| 10 | | 61.9% | thinking; context 120000 | May 2025 | Dec 2025 | Official leaderboard |
| 11 | | 60.8% | thinking; context 224000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 12 | | 59.7% | thinking; context 224000 | Jul 2025 | Dec 2025 | Official leaderboard |
| 13 | Claude 3.7 Sonnet | 59.7% | thinking; context 200000 | Feb 2025 | Dec 2025 | Official leaderboard |
| 14 | GLM-4.6 | 58.2% | thinking; context 120000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 15 | | 56.7% | thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
| 16 | Kimi K2 Sep 2025 | 55.5% | thinking; context 224000 | Sep 2025 | Dec 2025 | Official leaderboard |
| 17 | GLM-4.5 | 55.5% | thinking; context 120000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 18 | | 54.5% | thinking; context 224000 | Jul 2025 | Dec 2025 | Official leaderboard |
| 19 | | 53.2% | thinking; context 1000000 | Oct 2025 | Dec 2025 | Official leaderboard |
| 20 | gpt-oss-120b | 52.6% | thinking; context 120000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 21 | | 51.1% | thinking; context 120000 | Apr 2025 | Dec 2025 | Official leaderboard |
| 22 | | 50.1% | thinking; context 224000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 23 | GPT-4o Mar 2025 | 49.4% | thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
| 24 | | 47.1% | thinking; context 120000 | Apr 2025 | Dec 2025 | Official leaderboard |
| 25 | qwen3-4b-instruct-2507 | 45.7% | thinking; context 224000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 26 | | 45% | thinking; context 1000000 | Jan 2025 | Dec 2025 | Official leaderboard |
| 27 | gpt-oss-20b | 44.7% | thinking; context 120000 | Aug 2025 | Dec 2025 | Official leaderboard |
| 28 | | 44.5% | thinking; context 224000 | Dec 2025 | Dec 2025 | Official leaderboard |
| 29 | | 44.3% | thinking; context 120000 | Apr 2025 | Dec 2025 | Official leaderboard |
| 30 | | 44.1% | thinking; context 120000 | Sep 2024 | Dec 2025 | Official leaderboard |
| 31 | | 40.8% | thinking; context 120000 | Apr 2025 | Dec 2025 | Official leaderboard |
| 32 | | 37.3% | thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
| 33 | | 34.5% | thinking; context 224000 | Dec 2025 | Dec 2025 | Official leaderboard |
| 34 | | 33.7% | thinking; context 120000 | Dec 2024 | Dec 2025 | Official leaderboard |
| 35 | | 32.2% | non-thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
| 36 | | 21.8% | non-thinking; context 120000 | Mar 2025 | Dec 2025 | Official leaderboard |
Rankings by part
We read 12 parts from the benchmark’s paper in 3 groups: output length, contextual requirement and difficulty. Each ranks models on one part of the test. Gemini 2.5 Pro leads 7 of them, and 3 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
8k
36 models we track have a score here.
- 1
DeepSeek-V3.2 ↑3 75.5%
- 2 GPT-5 75.4%
- 3
DeepSeek-V3.1 ↑4 74.9%
- 4
Gemini 2.5 Pro Jun 2025 ↓3 74.5%
- 5 Claude Sonnet 4 ↓2 72.7%
- 6
Qwen3-235B-A22B-Thinking Jul 2025 72.1%
- 7
DeepSeek-R1 May 2025 ↑3 71.7%
- 8
Gemini 2.5 Flash Sep 2025 ↓3 71.4%
- 9 GLM-4.6 ↑5 71.2%
- 10
Qwen3-Next-80B-A3B ↓2 70.9%
16k
36 models we track have a score here.
- 1 GPT-5 ↑1 76.3%
- 2
Gemini 2.5 Pro Jun 2025 ↓1 74.8%
- 3
DeepSeek-V3.2 ↑1 74.5%
- 4
DeepSeek-V3.1 ↑3 71.6%
- 5 Claude Sonnet 4 ↓2 71.5%
- 6
Qwen3-235B-A22B-Instruct Jul 2025 ↑3 71.1%
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 70.4%
- 8
Gemini 2.5 Flash Sep 2025 ↓3 68.6%
- 9
Qwen3-Next-80B-A3B ↓1 68.5%
- 10
DeepSeek-R1 May 2025 68.3%
32k
36 models we track have a score here.
- 1
Gemini 2.5 Pro Jun 2025 75.3%
- 2 GPT-5 74.3%
- 3 Claude Sonnet 4 72.8%
- 4
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 69.7%
- 5
DeepSeek-V3.2 ↓1 69.5%
- 6
Gemini 2.5 Flash Sep 2025 ↓1 68.2%
- 7
Qwen3-235B-A22B-Instruct Jul 2025 ↑2 67.3%
- 8
DeepSeek-V3.1 ↓1 66.9%
- 9
Qwen3-Next-80B-A3B ↓1 65%
- 10 Claude 3.7 Sonnet ↑3 64.2%
64k
36 models we track have a score here.
- 1 GPT-5 ↑1 76.5%
- 2
Gemini 2.5 Pro Jun 2025 ↓1 74.2%
- 3 Claude Sonnet 4 70.5%
- 4
DeepSeek-V3.1 ↑3 70.3%
- 5
Gemini 2.5 Flash Sep 2025 70.3%
- 6
DeepSeek-V3.2 ↓2 69.5%
- 7
Qwen3-235B-A22B-Thinking Jul 2025 ↓1 66.9%
- 8
Qwen3-Next-80B-A3B 64%
- 9
DeepSeek-R1 May 2025 ↑1 62.9%
- 10 Claude 3.7 Sonnet ↑3 62.4%
128k
36 models we track have a score here.
- 1
Gemini 2.5 Pro Jun 2025 70%
- 2 GPT-5 69.4%
- 3 Claude Sonnet 4 66.4%
- 4
DeepSeek-V3.2 64.8%
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 64%
- 6
Gemini 2.5 Flash Sep 2025 ↓1 64%
- 7
DeepSeek-V3.1 61.5%
- 8
Qwen3-Next-80B-A3B 60.8%
- 9
Qwen3-235B-A22B-Instruct Jul 2025 60.2%
- 10
DeepSeek-R1 May 2025 58.1%
256k
36 models we track have a score here.
- 1
Gemini 2.5 Pro Jun 2025 71.8%
- 2 Claude Sonnet 4 ↑1 65.3%
- 3 GPT-5 ↓1 63.8%
- 4
Gemini 2.5 Flash Sep 2025 ↑1 62%
- 5
Qwen3-235B-A22B-Thinking Jul 2025 ↑1 58.8%
- 6
Qwen3-Next-80B-A3B-Instruct ↑5 56.7%
- 7
Qwen3-Next-80B-A3B ↑1 54.5%
- 8
DeepSeek-V3.2 ↓4 53.1%
- 9
Qwen3-235B-A22B-Instruct Jul 2025 52.8%
- 10
DeepSeek-V3.1 ↓3 52.1%
Full
36 models we track have a score here.
- 1
Gemini 2.5 Pro Jun 2025 70.1%
- 2 GPT-5 69.2%
- 3 Claude Sonnet 4 66.2%
- 4
DeepSeek-V3.2 64.6%
- 5
Gemini 2.5 Flash Sep 2025 63.7%
- 6
Qwen3-235B-A22B-Thinking Jul 2025 63.5%
- 7
DeepSeek-V3.1 63.1%
- 8
Qwen3-235B-A22B-Instruct Jul 2025 ↑1 59.1%
- 9
Qwen3-Next-80B-A3B ↓1 58.5%
- 10
DeepSeek-R1 May 2025 58.4%
Partial
36 models we track have a score here.
- 1
Gemini 2.5 Pro Jun 2025 77.7%
- 2 GPT-5 77%
- 3 Claude Sonnet 4 74.6%
- 4
Gemini 2.5 Flash Sep 2025 ↑1 72.2%
- 5
DeepSeek-V3.2 ↓1 71.9%
- 6
Qwen3-235B-A22B-Thinking Jul 2025 71.4%
- 7
Qwen3-Next-80B-A3B ↑1 70.9%
- 8
DeepSeek-V3.1 ↓1 70.2%
- 9
Qwen3-235B-A22B-Instruct Jul 2025 69.7%
- 10
Qwen3-Next-80B-A3B-Instruct ↑1 67%
Easy
36 models we track have a score here.
- 1
DeepSeek-V3.1 ↑6 85.7%
- 2 GPT-5 85.2%
- 3
DeepSeek-V3.2 ↑1 85%
- 4
Gemini 2.5 Pro Jun 2025 ↓3 84.4%
- 5 Claude Sonnet 4 ↓2 83.8%
- 6
Qwen3-235B-A22B-Thinking Jul 2025 83.5%
- 7
Qwen3-235B-A22B-Instruct Jul 2025 ↑2 83%
- 8
DeepSeek-R1 May 2025 ↑2 82.7%
- 9
Qwen3-Next-80B-A3B ↓1 81.9%
- 10
Qwen3-Next-80B-A3B-Instruct ↑1 80.8%
Moderate
36 models we track have a score here.
- 1 GPT-5 ↑1 82.3%
- 2
Gemini 2.5 Pro Jun 2025 ↓1 82%
- 3 Claude Sonnet 4 76.6%
- 4
Qwen3-235B-A22B-Thinking Jul 2025 ↑2 75.1%
- 5
DeepSeek-V3.2 ↓1 75.1%
- 6
DeepSeek-V3.1 ↑1 73.5%
- 7
Gemini 2.5 Flash Sep 2025 ↓2 72.4%
- 8
Qwen3-Next-80B-A3B 69.2%
- 9
Qwen3-235B-A22B-Instruct Jul 2025 68.1%
- 10
DeepSeek-R1 May 2025 66.5%
Hard
36 models we track have a score here.
- 1
Gemini 2.5 Pro Jun 2025 81%
- 2 GPT-5 78.7%
- 3 Claude Sonnet 4 74.7%
- 4
Gemini 2.5 Flash Sep 2025 ↑1 72.2%
- 5
DeepSeek-V3.2 ↓1 67.7%
- 6
Qwen3-235B-A22B-Thinking Jul 2025 67.1%
- 7
DeepSeek-V3.1 62.2%
- 8
Qwen3-Next-80B-A3B 61.5%
- 9
Qwen3-235B-A22B-Instruct Jul 2025 58.6%
- 10 Claude 3.7 Sonnet ↑3 56.6%
Extreme
36 models we track have a score here.
- 1
Gemini 2.5 Pro Jun 2025 50.8%
- 2 GPT-5 48.4%
- 3
Gemini 2.5 Flash Sep 2025 ↑2 47.4%
- 4 Claude Sonnet 4 ↓1 47.1%
- 5
DeepSeek-V3.2 ↓1 44.3%
- 6
Qwen3-235B-A22B-Thinking Jul 2025 43.4%
- 7
Qwen3-235B-A22B-Instruct Jul 2025 ↑2 43.2%
- 8
DeepSeek-V3.1 ↓1 42.7%
- 9
Qwen3-Next-80B-A3B ↓1 42.5%
- 10
DeepSeek-R1 May 2025 41.5%
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: The LongBench Pro paper
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| thinking; context 1000000 | 73.4% | ||||
| GPT-5 | thinking; context 272000 | 72.6% | |||
| Claude Sonnet 4 | thinking; context 1000000 | 69.9% | |||
| thinking; context 120000 | 67.8% | ||||
| thinking; context 1000000 | 67.4% | ||||
| thinking; context 224000 | 67% | ||||
| thinking; context 120000 | 66.2% | ||||
| thinking; context 224000 | 64% | ||||
| thinking; context 224000 | 63.8% | ||||
| thinking; context 120000 | 61.9% | ||||
| thinking; context 224000 | 60.8% | ||||
| thinking; context 120000 | 60.1% | ||||
| thinking; context 224000 | 59.7% | ||||
| Claude 3.7 Sonnet | thinking; context 200000 | 59.7% | |||
| GLM-4.6 | thinking; context 120000 | 58.2% | |||
| thinking; context 120000 | 56.7% | ||||
| Claude Sonnet 4 | non-thinking; context 1000000 | 56.1% | |||
| non-thinking; context 1000000 | 55.9% | ||||
| Kimi K2 Sep 2025 | thinking; context 224000 | 55.5% | |||
| GLM-4.5 | thinking; context 120000 | 55.5% | |||
| thinking; context 224000 | 54.5% | ||||
| thinking; context 1000000 | 53.2% | ||||
| gpt-oss-120b | thinking; context 120000 | 52.6% | |||
| non-thinking; context 224000 | 52.5% | ||||
| non-thinking; context 120000 | 51.7% | ||||
| non-thinking; context 120000 | 51.7% | ||||
| non-thinking; context 224000 | 51.5% | ||||
| Claude 3.7 Sonnet | non-thinking; context 200000 | 51.5% | |||
| non-thinking; context 120000 | 51.4% | ||||
| thinking; context 120000 | 51.1% | ||||
| thinking; context 224000 | 50.1% | ||||
| Kimi K2 Sep 2025 | non-thinking; context 224000 | 50.1% | |||
| GPT-4o Mar 2025 | thinking; context 120000 | 49.4% | |||
| thinking; context 120000 | 47.1% | ||||
| GPT-4o Mar 2025 | non-thinking; context 120000 | 46.7% | |||
| GLM-4.6 | non-thinking; context 120000 | 45.9% | |||
| qwen3-4b-instruct-2507 | thinking; context 224000 | 45.7% | |||
| thinking; context 1000000 | 45% | ||||
| gpt-oss-20b | thinking; context 120000 | 44.7% | |||
| thinking; context 224000 | 44.5% | ||||
| thinking; context 120000 | 44.3% | ||||
| thinking; context 120000 | 44.1% | ||||
| non-thinking; context 224000 | 43.8% | ||||
| GLM-4.5 | non-thinking; context 120000 | 43% | |||
| non-thinking; context 1000000 | 41.1% | ||||
| thinking; context 120000 | 40.8% | ||||
| non-thinking; context 120000 | 40.3% | ||||
| non-thinking; context 120000 | 39.6% | ||||
| non-thinking; context 224000 | 37.8% | ||||
| thinking; context 120000 | 37.3% | ||||
| non-thinking; context 120000 | 37.1% | ||||
| qwen3-4b-instruct-2507 | non-thinking; context 224000 | 36.8% | |||
| non-thinking; context 120000 | 36.1% | ||||
| thinking; context 224000 | 34.5% | ||||
| thinking; context 120000 | 33.7% | ||||
| non-thinking; context 120000 | 33.4% | ||||
| non-thinking; context 120000 | 32.2% | ||||
| thinking; context 120000 | 31.9% | ||||
| non-thinking; context 120000 | 31.9% | ||||
| non-thinking; context 120000 | 31.3% | ||||
| non-thinking; context 224000 | 30.2% | ||||
| non-thinking; context 120000 | 21.8% | ||||
| thinking; context 120000 | 21.2% |
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
Source: The LongBench Pro paper
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| thinking; context 1000000 | 73.4% | ||||
| GPT-5 | thinking; context 272000 | 72.6% | |||
| Claude Sonnet 4 | thinking; context 1000000 | 69.9% | |||
| thinking; context 120000 | 67.8% | ||||
| thinking; context 1000000 | 67.4% | ||||
| thinking; context 224000 | 67% | ||||
| thinking; context 120000 | 66.2% | ||||
| thinking; context 224000 | 64% | ||||
| thinking; context 224000 | 63.8% | ||||
| thinking; context 120000 | 61.9% | ||||
| thinking; context 224000 | 60.8% | ||||
| thinking; context 120000 | 60.1% | ||||
| thinking; context 224000 | 59.7% | ||||
| Claude 3.7 Sonnet | thinking; context 200000 | 59.7% | |||
| GLM-4.6 | thinking; context 120000 | 58.2% | |||
| thinking; context 120000 | 56.7% | ||||
| Claude Sonnet 4 | non-thinking; context 1000000 | 56.1% | |||
| non-thinking; context 1000000 | 55.9% | ||||
| Kimi K2 Sep 2025 | thinking; context 224000 | 55.5% | |||
| GLM-4.5 | thinking; context 120000 | 55.5% | |||
| thinking; context 224000 | 54.5% | ||||
| thinking; context 1000000 | 53.2% | ||||
| gpt-oss-120b | thinking; context 120000 | 52.6% | |||
| non-thinking; context 224000 | 52.5% | ||||
| non-thinking; context 120000 | 51.7% | ||||
| non-thinking; context 120000 | 51.7% | ||||
| non-thinking; context 224000 | 51.5% | ||||
| Claude 3.7 Sonnet | non-thinking; context 200000 | 51.5% | |||
| non-thinking; context 120000 | 51.4% | ||||
| thinking; context 120000 | 51.1% | ||||
| thinking; context 224000 | 50.1% | ||||
| Kimi K2 Sep 2025 | non-thinking; context 224000 | 50.1% | |||
| GPT-4o Mar 2025 | thinking; context 120000 | 49.4% | |||
| thinking; context 120000 | 47.1% | ||||
| GPT-4o Mar 2025 | non-thinking; context 120000 | 46.7% | |||
| GLM-4.6 | non-thinking; context 120000 | 45.9% | |||
| qwen3-4b-instruct-2507 | thinking; context 224000 | 45.7% | |||
| thinking; context 1000000 | 45% | ||||
| gpt-oss-20b | thinking; context 120000 | 44.7% | |||
| thinking; context 224000 | 44.5% | ||||
| thinking; context 120000 | 44.3% | ||||
| thinking; context 120000 | 44.1% | ||||
| non-thinking; context 224000 | 43.8% | ||||
| GLM-4.5 | non-thinking; context 120000 | 43% | |||
| non-thinking; context 1000000 | 41.1% | ||||
| thinking; context 120000 | 40.8% | ||||
| non-thinking; context 120000 | 40.3% | ||||
| non-thinking; context 120000 | 39.6% | ||||
| non-thinking; context 224000 | 37.8% | ||||
| thinking; context 120000 | 37.3% | ||||
| non-thinking; context 120000 | 37.1% | ||||
| qwen3-4b-instruct-2507 | non-thinking; context 224000 | 36.8% | |||
| non-thinking; context 120000 | 36.1% | ||||
| thinking; context 224000 | 34.5% | ||||
| thinking; context 120000 | 33.7% | ||||
| non-thinking; context 120000 | 33.4% | ||||
| non-thinking; context 120000 | 32.2% | ||||
| thinking; context 120000 | 31.9% | ||||
| non-thinking; context 120000 | 31.9% | ||||
| non-thinking; context 120000 | 31.3% | ||||
| non-thinking; context 224000 | 30.2% | ||||
| non-thinking; context 120000 | 21.8% | ||||
| thinking; context 120000 | 21.2% |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: The LongBench Pro paper
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official LongBench Pro leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Long context
- MRCR v2 In use Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens.
Muse Spark 1.3 98.5%
- LongBench v2 In use 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy.
Inkling 2 Small 70.6%
- GraphWalks In use A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1. Claude Fable 5 99.96%
- OpenAI MRCR v2 In use Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin.
Gemini 3.7 Flash 89%
- CL-bench In use Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. GPT-5.4 27.9%
- AA-LCR 1.1 In use Models answer 100 questions that need reasoning across sets of real documents of about 100K tokens. An LLM equality checker with a fixed system prompt compares each answer with the key.
Step 5 Preview 88.3%
Data audit
9 of 10 checks pass, 1 warning. Look first at the Newest score check. The newest came 11 months ago, in December 2025, for Ministral 3 3B Instruct. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 3 pass 1 note
- Pass Independent runs Scores from the benchmark’s paper (45 entries).
- Pass Models scored 36 models we track, plus 9 other entries on its leaderboard.
- Warning Newest score The newest came 11 months ago, in December 2025, for Ministral 3 3B Instruct.
- Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5 (#71).
- Pass Headroom The best is 73.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on huggingface.co: 63 rows for models we track and 9 other entries at the last read.
- Pass Last read November 21st, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.