Benchmark Almanac

Masakhane

We track 4 benchmarks that Masakhane made or helped make.

0 in use 0 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Uhura ARC-Easy Models answer ARC-Easy science questions translated into Amharic, Hausa, Northern Sotho, Swahili and Yoruba. Scoring is accuracy. Reasoning in many languages Aug 2024 System cards only
Uhura TruthfulQA Models answer TruthfulQA questions translated into Amharic, Hausa, Northern Sotho, Swahili and Yoruba. Scoring is accuracy. Knowledge in many languages Aug 2024 System cards only
AfroBench-Lite Models complete language understanding, question answering, reasoning, and generation tasks across 14 African languages. Seven datasets use task-specific metrics, including accuracy, F1, character-level translation scores, and exact match. Knowledge in many languages Jun 2025 77.7 Not in cards
IrokoBench Models answer human-translated knowledge, math and inference questions in 16 African languages. The parts are AfriMMLU, AfriMGSM and AfriXNLI. Knowledge in many languages Jun 2024 59% Not in cards