Columbia University
We track 8 benchmarks that Columbia University made or helped make.
0 in use 7 with public code 6 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| PrimeVul Models classify C and C++ functions as vulnerable or not, with a paired setting that pairs each vulnerable function with its patch. | Finding and exploiting real bugs | Mar 2024 | | System cards only |
| CWEval Models write code for security-critical tasks in several languages. Outcome-driven test oracles check both functionality (func@k) and security (func-sec@k). | Secure code and review | Jan 2025 | 50.2% | System cards only |
| MBABench Agents build complete financial models in Excel from a case prompt and documents. An LLM judge grades accuracy, formulas and formatting on a 0 to 100 scale. | Finance | May 2026 | 31.2 | Not in cards |
| MBABench 2.0 Agents build spreadsheet financial models for 101 tasks written by finance experts. A task passes only if every number matches the reference and the workbook meets a quality bar. | Finance | Oct 2026 | 15.8% | Not in cards |
| Seeing What Execution Needs Agents choose camera views that help a robot complete a tabletop task. The evaluator runs the action and checks whether the requested state holds. | Simulated robots | Nov 2026 | 42.9% | Not in cards |
| CREATE Models find a word that connects three photographs. A judge grades each answer against the intended connection. | General visual questions | Oct 2026 | | Not in cards |
| DyCodeEval Models solve dynamically rewritten HumanEval and MBPP problems with equivalent program behavior. The evaluator runs tests and computes DyPass@K across generated variants. | Function-level problems | Mar 2025 | | Not in cards |
| TailOR Models generate images and videos of regular, unconventional, and impossible tool interactions. Annotators score instruction adherence, interaction accuracy, physical realism, and perceptual quality. | Physical prediction | Jun 2026 | Not in cards |