Bespoke Labs
We track 3 benchmarks that Bespoke Labs made or helped make.
0 in use 3 with public code 3 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| AutoResearchExam Agents get 24 hours on a CPU or GPU machine to improve solutions to open-ended ML research tasks through experiments. The score is the area under the improvement curve on a hidden test set, rewarding both quality and speed. | Machine learning engineering | Sep 2026 | 0.61 | Not in cards |
| OpenThoughts-TBLite Agents solve 100 Terminal-Bench tasks picked to track Terminal-Bench 2 scores while running faster. Tasks are balanced across four difficulty bands using Claude Haiku 4.5 pass rates. | Terminal tasks | Feb 2026 | 75.1% | Not in cards |
| Multi Agent Coordination Bench Teams of eight agents work for four hours on twelve tasks built so each agent depends on its teammates. Each model also runs alone and as a team that can't talk, which isolates the gain from coordination. | Multi-agent games | Nov 2026 | 0.53 | Not in cards |