RULERv2
Models solve synthetic multi-key and multi-value needle tasks and QA tasks at basic, easy, medium and hard levels. Contexts run from 8K to 1M tokens, and accuracy is averaged across lengths.
Runs through NVIDIA NeMo-Skills as the ruler2 benchmark, and the RULER README now points users to it. No paper and no lab report found yet. Tasks differ from RULER, so scores are not comparable.
- Made by
- NVIDIA
- Released
- October 2025
- Kind
- Finding facts in long inputs
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Strongest model tested
-
Qwen3-235B-A22B-Thinking Jul 2025 #107 on the capability index - Last model added
- July 2025
- Scores refreshed
- November 12th, 2026
Gemini 2.5 Flash (snapshot unspecified) tops the leaderboard with 91.4%
Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is GPT-4.1, at 89.2%.
The newest model with a score here came out in July 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- Claude Fable 5.2
-
Gemini 4 Carbon - GPT-6 Astra
- Claude Opus 5.5
- and 16 more
- 1 Gemini 2.5 Flash (snapshot unspecified) Think On 91.4%
- 2 GPT-4.1 89.2%
- 3 Qwen3-235B-A22B-Thinking Jul 2025 85.2%
- 4 Qwen3-235B-A22B-Instruct Jul 2025 83.7%
Source: Official RULERv2 leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Gemini 2.5 Flash (snapshot unspecified) | 91.4% | Think On | Official leaderboard | |
| 2 | GPT-4.1 | 89.2% | Apr 2025 | Official leaderboard | |
| 3 | Qwen3-235B-A22B-Thinking Jul 2025 | 85.2% | Jul 2025 | Official leaderboard | |
| 4 | Qwen3-235B-A22B-Instruct Jul 2025 | 83.7% | Jul 2025 | Official leaderboard |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official RULERv2 leaderboard
GPT-4.1 on the official leaderboard passed 85% in April 2025
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official RULERv2 leaderboard
No lab model card reports it yet
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.
Versions
We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.
Mistral Medium 3 
Sources: Official leaderboards and labs’ model cards
More in Long context
- MRCR v2 In use Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens.
Muse Spark 1.3 98.5% - LongBench v2 In use 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy.
Inkling 2 Small 70.6% - GraphWalks In use A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1. Claude Fable 5 99.96%
- OpenAI MRCR v2 In use Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin. Gemini 3.7 Flash 89%
- CL-bench In use Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. GPT-5.4 27.9%
- AA-LCR 1.1 In use Models answer 100 questions that need reasoning across sets of real documents of about 100K tokens. An LLM equality checker with a fixed system prompt compares each answer with the key.
Step 5 Preview 88.3%