Benchmark Almanac

University of Oregon NLP Group

We track 4 benchmarks that University of Oregon NLP Group made or helped make.

0 in use 4 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Okapi MMLU Models answer MMLU multiple choice questions that ChatGPT translated into 26 languages. Scoring is accuracy. Knowledge in many languages Jul 2023 80% Dropped
Okapi ARC Models choose answers to translated science questions from ARC-Challenge. Science questions Jul 2023 Not in cards
Okapi HellaSwag Models choose the most plausible next event in a translated description of an everyday activity. Reasoning in many languages Jul 2023 Not in cards
Okapi TruthfulQA Models distinguish true answers from common misconceptions in translated multiple-choice questions. Short factual questions Jul 2023 Not in cards