Benchmark Almanac

U.S. National Library of Medicine

We track 9 benchmarks that U.S. National Library of Medicine made or helped make.

0 in use 2 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
MedGenVidQA 2026 Visual Answer Localization Models mark the start and end of the video segment that answers a medical question. The organizers created 80 test examples from professional medical videos. Medicine Jan 2026 Not in cards
MedVidQA 2023 Medical Instructional Question Generation Models write a question that a supplied medical video segment answers. They can use both the images and the subtitles. Medicine Feb 2023 Not in cards
MedVidQA 2023 Video Corpus Visual Answer Localization Models search 12,657 medical videos and return the times when a video shows the answer. Half of the 40 questions require visual evidence. Medicine Feb 2023 Not in cards
MedVidQA 2024 Query-Focused Instructional Step Captioning Models divide a medical demonstration into steps that answer a question. They give each step a caption and start and end times. Medicine Feb 2024 Not in cards
MedVidQA 2024 Video Corpus Visual Answer Localization Models search 48,605 medical videos for demonstrations that answer 52 new questions. They return ranked videos and the start and end times of each answer. Medicine Feb 2024 Not in cards
PLABA 2023 Abstract Adaptation Models rewrite biomedical abstracts for patients. Four annotators independently adapted each of the 400 test abstracts. Medicine May 2023 Not in cards
PLABA 2024 Complete Abstract Adaptation Models rewrite 400 new biomedical abstracts for patients. Reviewers grade how well the rewrites preserve information and explain it clearly. Medicine Jan 2024 Not in cards
PLABA 2024 Term Replacement Models identify difficult medical terms, classify how to adapt them, and supply wording for patients. The organizers held out 300 abstracts. Medicine Aug 2024 Not in cards
Plain Language Adaptation of Biomedical Abstracts Models rewrite biomedical abstracts in plain language for people who asked health questions. The authors collected 750 abstracts for 75 questions. Medicine Jul 2022 Not in cards