Stony Brook University
We track 4 benchmarks that Stony Brook University made or helped make.
0 in use 4 with public code 3 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| AppWorld Test-Challenge Agents complete 417 app tasks that require Amazon or Gmail APIs held out from training. The score is the percentage passing every task test. | Apps and MCP servers | Jul 2024 | 92.8% | Not in cards |
| AppWorld Test-Normal Agents complete 168 everyday app tasks from familiar applications by calling APIs through code. The score is the percentage passing every task test. | Apps and MCP servers | Jul 2024 | 94% | Not in cards |
| PRMBench Models identify errors and redundant steps in mathematical solutions. PRMScore combines positive and negative F1 scores across simplicity, soundness, and sensitivity tests. | Word problems | Jan 2025 | 68.8 | Not in cards |
| AppWorld Agents write code against 457 APIs of 9 simulated everyday apps to finish 750 tasks for about 100 fictional users. State-based unit tests check the outcome and collateral damage. | Apps and MCP servers | Jul 2024 | Not in cards |