Benchmark Almanac

Fab AI

We track 6 benchmarks that Fab AI made or helped make.

0 in use 5 with public code 6 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
SEND Pedagogy Benchmark Models choose how a teacher should support students with disabilities or special educational needs. The questions come from Chilean teacher exams. Exam questions Mar 2025 89.5% Not in cards
Pedagogy Benchmark Models answer multiple-choice questions about teaching strategies and assessment from Chilean teacher exams. Exam questions Dec 2024 92.9% Not in cards
Linguistic Knowledge Benchmark (Fab AI) Models answer multiple-choice and short-answer questions about Kiswahili and English grammar, phonics, and spelling. Three model judges vote on short answers. Language reasoning Sep 2026 97.8% Not in cards
Visual Reasoning Benchmark (Fab AI) Models choose which picture completes a pattern, matches another shape, or shows a folded figure. Questions come from school exams in Zambia and India. Spatial reasoning and counting Sep 2025 89.9% Not in cards
Decodability Assessment Benchmark Models flag words a child cannot yet sound out in a passage, given the phonics lessons the child has completed. The leaderboard reports adjusted balanced accuracy. Language reasoning Sep 2026 95.9% Not in cards
Read-aloud Assessment Benchmark Speech models transcribe children reading aloud in Kiswahili. The evaluation measures transcription errors and disagreements with human assessors about which items the child misread. Speech recognition and translation Aug 2026 54.5% Not in cards