Benchmark Almanac

UNC Chapel Hill

We track 5 benchmarks that UNC Chapel Hill made or helped make.

1 in use 5 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
RefCOCO / RefCOCO+ / RefCOCOg Model grounds a referring expression to a bounding box in a COCO image. Scored by accuracy at IoU 0.5. Detection and captioning Jul 2016 95% In use
QVHighlights Natural-language queries over YouTube videos. Model localizes relevant moments, scored by Recall@1 at IoU 0.5. Video understanding Jul 2021 77.2% Fading
TVQA Models answer 152,545 multiple-choice questions about clips from 6 TV shows, using video frames and subtitles. Scoring is accuracy. Video understanding Sep 2018 87.3% Dropped
LoCoMo Models answer questions, summarize events and continue dialogue over very long multi-session chats of about 300 turns and 9K tokens, up to 35 sessions. Answers are scored against references. Finding facts in long inputs Feb 2024 Not in cards
MMIE Models interpret and generate interleaved text and images. MMIE-Score, a fine-tuned InternVL-2-4B model, grades the resulting multimodal responses. General visual questions Oct 2024 Not in cards