Harvard University
We track 6 benchmarks that Harvard University made or helped make.
4 in use 6 with public code 5 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| ProgramBench tests passed Agents rebuild 200 programs from only their compiled binaries and docs. The score is the share of hidden behavioral tests the rebuilt program passes, averaged over tasks. | Building whole projects | May 2026 |  88.5% | In use |
| ProgramBench 200 tasks: rebuild a full program from only its compiled binary and docs. Hidden behavioral tests (248k total) must match the original. | Building whole projects | May 2026 | 18.5% | In use |
| ProgramBench almost solved Agents rebuild 200 programs from only their compiled binaries and docs. The score is the share of tasks whose rebuilt program passes at least 95% of the hidden behavioral tests. | Building whole projects | May 2026 | 70.5% | In use |
| MTOB Models learn to translate between English and Kalamang, a language with under 200 speakers, from one grammar book. Scored by chrF. | Learning from the context | Sep 2023 | 54.0 | Dropped |
| HorizonMath Models attack 113 mostly unsolved problems in eight areas of mathematics and the mathematical sciences. A program checks each proposed closed form, construction or bound. | Research mathematics | Mar 2026 | 11.5% | In use |
| PrivacyLens Language model agents write emails, posts or messages from a prior tool trajectory without leaking sensitive details. A judge scores information leakage and helpfulness. | Refusing harmful requests | Aug 2024 | 49.2% | System cards only |