Benchmark Almanac

AI4Bharat

We track 5 benchmarks that AI4Bharat made or helped make.

0 in use 4 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
MILU Multi-task Indic Language Understanding: culturally grounded multiple-choice questions in 11 Indic languages. Knowledge in many languages Nov 2024 94.1% Dropped
IndicIFEval Models follow instructions in 14 Indic languages. Rule-based checks test constraints such as required words, paragraph structure and word counts. Chat and instructions in many languages Feb 2026 82.4% Not in cards
INCLUDE Indian Sign Language Recognition Models identify isolated Indian Sign Language words in video. The authors provide separate test files for the full vocabulary and its smaller selection. Video understanding Oct 2020 Not in cards
IndicQA Models extract answers from Wikipedia passages in Indian languages. Native speakers wrote the questions and answers. Reading in many languages Jul 2023 Not in cards
Vistaar Models transcribe speech across 59 combinations of language and dataset in 12 Indian languages. Each test has its own word error rate. Speech recognition and translation May 2023 Not in cards