Benchmark Almanac

Guokan Shang

We track 3 benchmarks that Guokan Shang made or helped make.

0 in use 3 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
DarijaBench Models translate, summarize and classify sentiment in Moroccan Arabic. A transliteration task also tests conversion between Arabic and Latin scripts. Reading in many languages Sep 2024 Not in cards
DarijaHellaSwag Models read a situation in Moroccan Arabic and choose its most plausible continuation. Reasoning in many languages Sep 2024 Not in cards
DarijaMMLU Models answer academic multiple-choice questions in Moroccan Arabic. The evaluation adapts MMLU to the dialect. Knowledge in many languages Sep 2024 Not in cards