Benchmark Almanac

Northeastern University

We track 12 benchmarks that Northeastern University made or helped make.

1 in use 12 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
RefAdv-S Models locate the object a referring expression describes among hard distractors in real images. Scored by accuracy at 0.5 box overlap on 1,142 cases. Detection and captioning Feb 2026 68% In use
PrivacyLens Language model agents write emails, posts or messages from a prior tool trajectory without leaking sensitive details. A judge scores information leakage and helpfulness. Refusing harmful requests Aug 2024 49.2% System cards only
MatCha Models answer multiple-choice questions about material structure, shape and properties using images from materials characterization instruments. Lab and data work Sep 2025 68.1% Not in cards
Numerical results extraction from randomized controlled trials Models extract group sizes, event counts, means, and standard deviations from trial reports. The test contains 656 treatment comparisons across 110 papers. Medicine May 2024 77.9% Not in cards
CircuitSense Models read circuit schematics and block diagrams, derive equations, and select designs. SymPy checks symbolic answers. Other tasks use exact matches, a model judge, or simulation. Visual math and diagrams Sep 2025 68.9% Not in cards
MobileAgentBench Agents complete 100 tasks across ten Android apps. The grader checks app state and accessibility events, so agents can take different paths. Phones Jun 2024 8.00% Not in cards
EBM-NLP Models mark the words that describe participants, interventions, and outcomes in clinical trial abstracts. Medical professionals supplied the test labels. Medicine Jun 2018 Not in cards
Evidence Inference 2.0 Models classify the direction of a treatment effect in a trial report and identify text that supports their answer. Medicine May 2020 Not in cards
MS² Models synthesize the findings of multiple medical studies into a short answer to the research question of a systematic review. Medicine Apr 2021 Not in cards
MultiCochrane Models rewrite medical review sentences in plain language. The collection pairs technical and simplified wording in English, Spanish, French, and Farsi. Medicine May 2023 Not in cards
PHEE Models extract adverse effects and potential therapeutic effects from published medical cases. They identify the event triggers and details such as the treatment and the patient. Medicine Oct 2022 Not in cards
Trialstreamer clinical evidence extraction Models identify treatments, comparators, outcomes, and the direction of each reported effect in trial abstracts. Physicians annotated 100 test abstracts. Medicine Jan 2020 Not in cards