AgentWorldBench
Language world models predict the next environment observation after an agent action, in 7 domains from MCP and search to Android and desktop OS. A judge model scores each prediction against the real observation on five dimensions.
2,170 samples built from real trajectories on Toolathlon, Terminal-Bench 1.0 and 2.0, and OSWorld-Verified. The judge is gpt-5.2. It looks like a public form of the in-house QwenWorldBench (same 7 domains and 5 rubric dimensions), but the scores differ, so it is kept separate.
- Made by
- Qwen Team, Alibaba Group
- Released
- June 2026
- Kind
- Interactive world models
- Level
- University
- Scored
- Graded by a model
- Answers
- Open-ended
- Tasks
- All public
- Strongest model tested
- Claude Opus 4.8 #21 on the capability index
- Last in a launch post
- June 2026
- Last model added
- June 2026
- Scores refreshed
- November 12th, 2026
Qwen-AgentWorld-397B-A17B tops the leaderboard with 58.7
Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is GPT-5.4, at 58.3.
The newest model with a score here came out in June 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- Claude Fable 5.2
-
Gemini 4 Carbon - GPT-6 Astra
- Claude Opus 5.5
- and 16 more
- 1 Qwen-AgentWorld-397B-A17B 58.7
- 2 GPT-5.4 58.3
- 3 Claude Opus 4.6 57.8
- 4 Claude Opus 4.8 56.6
- 5
Qwen-AgentWorld-35B-A3B 56.4 - 6 Claude Sonnet 4.6 56.0
- 7 Qwen3.5 397B-A17B 54.7
- 8 Gemini 3.1 Pro 54.6
- 9 Kimi K2.6 53.4
- 10
DeepSeek-V4-Pro 53.0 - 11 GLM-5.1 51.3
- 12 Qwen 3.6 Plus 50.8
- 13 Qwen3.5-35B-A3B 47.7
- 14
MiniMax-M2.7 46.1
Source: Official AgentWorldBench leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Qwen-AgentWorld-397B-A17B | 58.7 | Official leaderboard | ||
| 2 | GPT-5.4 | 58.3 | Mar 2026 | Official leaderboard | |
| 3 | Claude Opus 4.6 | 57.8 | Feb 2026 | Official leaderboard | |
| 4 | Claude Opus 4.8 | 56.6 | May 2026 | Official leaderboard | |
| 5 | Qwen-AgentWorld-35B-A3B | 56.4 | Jun 2026 | Official leaderboard | |
| 6 | Claude Sonnet 4.6 | 56.0 | Feb 2026 | Official leaderboard | |
| 7 | Qwen3.5 397B-A17B | 54.7 | Feb 2026 | Official leaderboard | |
| 8 | Gemini 3.1 Pro | 54.6 | Feb 2026 | Official leaderboard | |
| 9 | Kimi K2.6 | 53.4 | Apr 2026 | Official leaderboard | |
| 10 | DeepSeek-V4-Pro | 53.0 | Apr 2026 | Official leaderboard | |
| 11 | GLM-5.1 | 51.3 | Apr 2026 | Official leaderboard | |
| 12 | Qwen 3.6 Plus | 50.8 | Mar 2026 | Official leaderboard | |
| 13 | Qwen3.5-35B-A3B | 47.7 | Feb 2026 | Official leaderboard | |
| 14 | MiniMax-M2.7 | 46.1 | Mar 2026 | Official leaderboard |
Labs report scores for 1 model in their cards. We plot every source in the charts below.
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official AgentWorldBench leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official AgentWorldBench leaderboard and labs’ model cards
One lab put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Jun 22, 2026 | Qwen-AgentWorld-35B-A3B | 56.4 | AgentWorldBench Overall (five-dimension rubric mean, 0 to 100) |
One system card reports it
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Jun 23, 2026 | Qwen-AgentWorld-35B-A3B technical report | 56.4 | AgentWorldBench Avg., five-dimension rubric mean 0 to 100 (Table 5). MCP 64.79, Search 36.69, Terminal 53.96, SWE 65.63, Android 58.17, Web 49.55, OS 65.92. Rule-based verification average 57.28 % (Table 10) | Capabilities |
One lab claim next to the leaderboard
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.
| Model | Lab says | Board says | Gap |
|---|---|---|---|
| Qwen-AgentWorld-35B-A3B | 56.4 AgentWorldBench Overall (five-dimension rubric mean, 0 to 100) | 56.4 Leaderboard | Same |
More in World models
- IntPhys 2 In use Models watch simulated videos and judge whether each event is physically possible, testing object permanence, immutability, continuity and solidity. Scored by accuracy.
Muse Spark Open 68.3% - QwenWorldBench Fading Models act as world models that simulate agent environments in 7 domains, such as terminal, web, and Android. A rubric judge scores five dimensions. Qwen3.7-Plus 62.1%
- VPCT Fading Models view an image of a ball, ramps, and three buckets and predict which bucket the ball falls into. Scored by accuracy on 100 problems. Gemini 3 Pro 91%
- ContPhy Fading Models infer physical properties such as mass and density of fluids and soft bodies from videos and predict their dynamics.
Seed2.0 Pro 67.4% - RBench Only in system cards 650 image+text prompts for robot video generation (tasks and embodiments). MLLM judge scores task completion, physical plausibility, stability.
Wan 2.6 60.7% - PAI-Bench Only in system cards Physical-AI video generation from 1,044 prompts in six domains, VLM-QA plus quality metrics. Companion control and understanding tracks.
Cosmos3-Nano 83.7%