Benchmark Almanac

Sarvam AI

We track 4 benchmarks that Sarvam AI made or helped make. Its models are on its lab page.

0 in use 2 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
IndiVibe Models answer 110 source prompts in chat, STEM, math and coding, translated into 22 Indian languages in native and romanized script. A judge model compares answers pairwise. Chat and instructions in many languages Mar 2026 90% Fading
IndicContextEval Models transcribe speech in eight Indian languages using hints about the topic or names. One prompt deliberately supplies names from an unrelated field. Speech recognition and translation Jun 2026 17.5% Not in cards
Indic DiarBench Models transcribe conversations in 22 Indian languages and identify who spoke each word. Human annotators supply speaker labels and timestamps, including for overlapping speech. Speech recognition and translation Jul 2026 36.2% Not in cards
Synthetic Contextual ASR Benchmark (Indic) Models transcribe a synthetic caller’s response with or without the preceding conversation. A language model scores whether each transcript preserves the caller’s intent and named entities. Speech recognition and translation Feb 2026 Not in cards