Superintelligence Labs
We track 9 benchmarks that Superintelligence Labs made or helped make.
0 in use 9 with public code 0 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| AI Agent Privacy Web agents complete tasks while exposed to private information. The benchmark measures privacy leakage and unsafe disclosure during autonomous operation. | Deception and alignment | Mar 2025 | Not in cards | |
| CRAG-MM Multimodal retrieval agents answer single-turn and multi-turn questions grounded in images and changing web content. The benchmark includes wearable-style egocentric images. | Questions that need search | Oct 2025 | Not in cards | |
| E2E-SWE Agents build complete software repositories from natural language specifications in empty workspaces. Hidden tests require every generated project to install and run. | Building whole projects | Sep 2026 | Not in cards | |
| EgoToM Video models infer a camera wearer’s goals, beliefs, and future actions from egocentric clips. Multiple-choice questions test theory of mind. | General visual questions | Mar 2025 | Not in cards | |
| HalluLens Language models answer dynamically generated prompts that test factual and contextual hallucination. The evaluator builds fresh examples to reduce test leakage. | Hallucination rates | Apr 2025 | Not in cards | |
| KernelBench-Verified Language models generate CUDA kernels that must match PyTorch outputs and improve runtime. The verifier checks correctness with robust performance baselines. | GPU kernels | Jan 2026 | Not in cards | |
| PARTNR Agents follow natural language instructions in human-robot collaboration scenes. Tasks require planning and coordination among multiple embodied agents. | Embodied reasoning | Nov 2024 | Not in cards | |
| PHYRE Agents solve two-dimensional physics puzzles by placing new bodies into a simulated scene. The benchmark measures success on unseen puzzles and the number of attempts. | Physical prediction | May 2020 | Not in cards | |
| WorldPrediction Agents predict and plan in procedural environments with long horizons. Tasks test whether models maintain a useful world model while acting. | Interactive world models | Jun 2025 | Not in cards |