RNG-Bench
Agents play Matching Pairs and navigate 3D mazes using remembered observations. Scores measure matched pairs and combine maze success, efficiency, and exploration.
The suite also includes duels and an oracle minimap comparison. Main-table Matching Pairs seeds and repeat counts are not fully documented.
- Made by
- Fudan University, Shanghai Innovation Institute, and Shanghai AI Laboratory with collaborators
- Released
- June 2026
- Kind
- Puzzles and chess
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Agent task
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Multi-agent
- Environment
- Concentration, A simulator
- Items from
- Generated, Live play or votes
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
- GPT-5.4 #28 on the capability index
- Last model added
- June 2026
- Scores refreshed
- November 20th, 2026
In the authors’ words
Deploying multimodal foundation models as closed-loop policies increasingly requires conditioning actions on observations that are no longer visible. However, existing benchmarks either expose the full state, conflate hidden-state reconstruction with other agent skills, or test recall only after an episode has ended. We introduce RNG-Bench (Reconstructive Non-Markov Games), a benchmark suite designed to isolate a base model's ability to reconstruct past observations and act on them during multi-step interaction. RNG-Bench includes two complementary games: Matching Pairs, where card identities briefly revealed at specific locations must later be recalled, and 3D Maze, where egocentric views must be integrated into a spatial map. Both games are evaluated under a unified harness with three controlled difficulty axes: grid size, visual pattern, and observation modality. The benchmark further introduces a head-to-head duel protocol to control for instance-level variance and a Memory Gap metric that disentangles forgetting from poor action selection. The hardest configurations require contexts of roughly 128K tokens and 350 image inputs per episode, and remain far from saturated by frontier MLLMs. Memory Gap analysis shows that most residual errors stem from forgetting earlier observations rather than from suboptimal decision making. Finally, fine-tuning Qwen3.5-9B on optimal-policy rollouts and filtered model demonstrations improves performance on RNG-Bench and transfers to existing benchmarks without degrading general multimodal capability.
Gemini 3.1 Pro leads with 49.9% among models released by June 2026
Best score per model from the benchmark's own leaderboard, read on November 20th, 2026. The board reports no overall score, so this is the plain mean of its two single-player game scores. Where a model ran with several settings or harnesses, we keep its best.
The newest model with a score here came out in June 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 1
Gemini 3.1 Pro Single-player, 10×10 image pairs and 13×13 maze without minimap 49.9%
- 2
Seed2.1 Pro Single-player, 10×10 image pairs and 13×13 maze without minimap 48.3%
- 3 GPT-5.4 Single-player, 10×10 image pairs and 13×13 maze without minimap 46.4%
- 4
Seed2.0 Lite (0215) Single-player, 10×10 image pairs and 13×13 maze without minimap 32.5%
- 5 Kimi K2.5 Single-player, 10×10 image pairs and 13×13 maze without minimap 27.1%
- 6
Qwen3.5 397B-A17B Single-player, 10×10 image pairs and 13×13 maze without minimap 17.9%
Source: Official RNG-Bench leaderboard
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 49.9% | Single-player, 10×10 image pairs and 13×13 maze without minimap | Feb 2026 | Official leaderboard |
| 2 | | 48.3% | Single-player, 10×10 image pairs and 13×13 maze without minimap | Jun 2026 | Official leaderboard |
| 3 | GPT-5.4 | 46.4% | Single-player, 10×10 image pairs and 13×13 maze without minimap | Mar 2026 | Official leaderboard |
| 4 | | 32.5% | Single-player, 10×10 image pairs and 13×13 maze without minimap | Feb 2026 | Official leaderboard |
| 5 | Kimi K2.5 | 27.1% | Single-player, 10×10 image pairs and 13×13 maze without minimap | Jan 2026 | Official leaderboard |
| 6 | | 17.9% | Single-player, 10×10 image pairs and 13×13 maze without minimap | Feb 2026 | Official leaderboard |
Rankings by part
We read 2 parts from the benchmark’s own leaderboard: Matching Pairs and 3D Maze. Each ranks models on one part of the test. Seed2.1 Pro leads Matching Pairs and Gemini 3.1 Pro leads 3D Maze. Arrows show how far a model moved from its overall rank among the models we track.
Matching Pairs
6 models we track have a score here.
- 1
Seed2.1 Pro ↑1 64.6%
- 2 GPT-5.4 ↑1 62.3%
- 3
Gemini 3.1 Pro ↓2 50%
- 4
Seed2.0 Lite (0215) 43.2%
- 5 Kimi K2.5 38%
- 6
Qwen3.5 397B-A17B 25.3%
3D Maze
6 models we track have a score here.
- 1
Gemini 3.1 Pro 49.7%
- 2
Seed2.1 Pro 32%
- 3 GPT-5.4 30.5%
- 4
Seed2.0 Lite (0215) 21.7%
- 5 Kimi K2.5 16.1%
- 6
Qwen3.5 397B-A17B 10.5%
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official RNG-Bench leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official RNG-Bench leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Games and long tasks
- ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6.2 Astra 100%
- Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games.
Gemini 2.5 Pro 3.90
- MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks.
Seed2.0 Pro 49%
- TextQuests Only in system cards Agents play 25 classic interactive fiction games. The score is harmful actions per 100 steps, where lower is better. The board also reports game progress.
Llama 4 Scout 7.00
- GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. GPT-6.1 Terra 76.8%
- Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%
Data audit
All 10 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (6 entries).
- Pass Models scored 6 models we track.
- Pass Newest score The newest is from June 2026, for Seed2.1 Pro.
- Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5.4 (#28).
- Pass Headroom The best is 49.9%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on github.com: 6 rows for models we track at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.