Johns Hopkins University
We track 7 benchmarks that Johns Hopkins University made or helped make.
0 in use 5 with public code 3 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Winogender Models resolve pronouns in sentences about occupations. Accuracy shows whether gender stereotypes sway coreference. | Bias, fairness and politics | Apr 2018 | | System cards only |
| ProteinLMBench Models answer 944 hand-checked multiple choice questions on protein sequence, structure and function. | Genomics and life sciences | Jun 2024 | 57.9% | Dropped |
| ReCoRD Models fill in a masked entity in a query about a news article, using the article as context. Scoring is exact match and token F1. | Reading comprehension | Oct 2018 | 93.8% | Dropped |
| AL-QASIDA Models write in eight dialectal Arabic varieties. Scored on fidelity, understanding, quality and diglossia, including the ADI2 dialect identification score. | Chat and instructions in many languages | Dec 2024 | System cards only | |
| ManyIH-Bench Models resolve conflicts among instructions at up to twelve privilege levels. A response must satisfy all winning instructions and any applicable coding tests. | Following precise instructions | Apr 2026 | 60.9% | Not in cards |
| HANS An entailment test built to catch three shallow shortcuts: lexical overlap, subsequence and constituent. Every premise and hypothesis pair shares words, but only some pairs are entailments. A model that relies on word overlap fails the non-entailment cases. | Language reasoning | Feb 2019 | Not in cards | |
| StyleInstructCaps Models describe a speaker's voice according to different instructions. The evaluator grades instruction following, accuracy about the recording, and consistency across the speaker's clips. | Speech recognition and translation | Oct 2026 | Not in cards |