Caltech
We track 4 benchmarks that Caltech made or helped make.
0 in use 3 with public code 1 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| MineDojo Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. | Video games | Jun 2022 | | Fading |
| FrontierPhysics Agents do end-to-end physics research on 56 tasks that 42 physicists from 26 institutions built from their own work, across nine domains of theoretical, computational and experimental physics. Each task asks the agent to review the literature, plan, run the research and write a manuscript. Deterministic verifiers and expert rubrics grade whether the research is correct and publishable, over 3 trials. | Research science | Aug 2026 | 27.4% | Not in cards |
| LeanDojo Benchmark Models prove theorems from Lean's mathlib library inside the Lean proof assistant, which checks every proof. The benchmark has 98,734 theorems and proofs with premise annotations. A novel-premises split tests whether a prover can use lemmas it never saw in training. | Formal proofs | Jun 2023 | Not in cards | |
| Natural World Tasks (NeWT 2021) Models distinguish pairs of labels in wildlife photographs. The 164 tasks include recognizing life stages, animal behavior and scene context. | General visual questions | Mar 2021 | Not in cards |