1X World Model Challenge
Predict future egocentric frames of 1X's EVE humanoid from video and state. Compression scored by token cross-entropy, Sampling by PSNR on private tests.
Prize-based phases. 1X scores submissions on a held-out set and warns that nearest-neighbor retrieval gets good validation scores. The Evaluation (policy-ranking) challenge was announced but not released. Paper is the winning team's report. Competition spaces on HF.
- Made by
- 1X Technologies
- Released
- June 2024
- Kind
- Video world models
- Scored
- Checked automatically
- Answers
- Open-ended
- Tasks
- Public, plus a private test set
In the authors’ words
World models are a powerful paradigm in AI and robotics, enabling agents to reason about the future by predicting visual observations or compact latent states. The 1X World Model Challenge introduces an open-source benchmark of real-world humanoid interaction, with two complementary tracks: sampling, focused on forecasting future image frames, and compression, focused on predicting future discrete latent codes. For the sampling track, we adapt the video generation foundation model Wan-2.2 TI2V-5B to video-state-conditioned future frame prediction. We condition the video generation on robot states using AdaLN-Zero, and further post-train the model using LoRA. For the compression track, we train a Spatio-Temporal Transformer model from scratch. Our models achieve 23.0 dB PSNR in the sampling task and a Top-500 CE of 6.6386 in the compression task, securing 1st place in both challenges.
1x-technologies/GENIE_138M tops the leaderboard with 8.79
Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.
- 1 1x-technologies/GENIE_138M maskgit_steps 2 8.79
- 2 1x-technologies/GENIE_35M maskgit_steps 2 8.99
Source: Official 1X World Model Challenge leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | 1x-technologies/GENIE_138M | 8.79 | maskgit_steps 2 | Official leaderboard | |
| 2 | 1x-technologies/GENIE_35M | 8.99 | maskgit_steps 2 | Official leaderboard |
No lab model card reports it yet
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.
More in World models
- AgentWorldBench In use Language world models predict the next environment observation after an agent action, in 7 domains from MCP and search to Android and desktop OS. A judge model scores each prediction against the real observation on five dimensions. GPT-5.4 58.3
- IntPhys 2 In use Models watch simulated videos and judge whether each event is physically possible, testing object permanence, immutability, continuity and solidity. Scored by accuracy.
Muse Spark Open 68.3% - QwenWorldBench Fading Models act as world models that simulate agent environments in 7 domains, such as terminal, web, and Android. A rubric judge scores five dimensions.
Qwen3.7-Plus 62.1% - VPCT Fading Models view an image of a ball, ramps, and three buckets and predict which bucket the ball falls into. Scored by accuracy on 100 problems.
Gemini 3 Pro 91% - ContPhy Fading Models infer physical properties such as mass and density of fluids and soft bodies from videos and predict their dynamics.
Seed2.0 Pro 67.4% - RBench Only in system cards 650 image+text prompts for robot video generation (tasks and embodiments). MLLM judge scores task completion, physical plausibility, stability.
Wan 2.6 60.7%