Benchmark Almanac

Indian Institute of Science

We track 11 benchmarks that Indian Institute of Science made or helped make.

0 in use 10 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
HEPToolBench 1.2 Models generate inputs for particle physics simulations and inspect the resulting files. The main suite has 28 tasks and a separate extension adds three debugging tasks. Lab and data work Aug 2026 71.4% Not in cards
DISPLACE Challenge 2023 Models mark who speaks when and which language each person speaks in multilingual conversations. The scorer counts missed speech, false alarms and incorrect assignments, including overlaps. Audio understanding and voice Dec 2022 Not in cards
DISPLACE Challenge 2024 Models mark speakers and languages in multilingual conversations and transcribe individual speakers. The organizers score timing errors on room recordings and word errors on close microphone recordings. Audio understanding and voice Dec 2023 Not in cards
DISPLACE-M 2026 Models identify speakers, transcribe their words, identify medical topics and summarize conversations between community health workers and patients. Each task has its own score. Audio understanding and voice Jan 2026 Not in cards
LIMMITS 2024 Speech models read text in a target voice across seven languages. Native listeners score naturalness and speaker similarity on the same prompts. Audio understanding and voice Sep 2023 Not in cards
LIMMITS 2025 Speech models synthesize text in four languages. Listeners rate naturalness and voice similarity. Separate tracks also measure streaming latency, codec bitrate, or both. Audio understanding and voice Aug 2024 Not in cards
MADASR 2.0 Models transcribe read speech and spontaneous conversations across eight Indian languages. The organizers rank each track by character error and also score language and dialect identification. Speech recognition and translation Apr 2025 Not in cards
MADASR 2023 Models transcribe dialectal Bengali and Bhojpuri read aloud by new speakers. Four tracks vary which speech recordings and written texts teams can use. Speech recognition and translation May 2023 Not in cards
Pixel2Perspective Models locate and segment objects from aerial and ground views. Tests vary the camera angle, weather, and use of visible or infrared images. Detection and captioning Nov 2026 Not in cards
RESPIN-S1.0 Models transcribe dialect speakers reading sentences about farming and finance in nine Indian languages. The test holds out speakers and reports errors by dialect. Speech recognition and translation May 2025 Not in cards
Vaani Benchmark V1.0 Models transcribe speakers describing pictures. The scorer compares each prediction with three human transcripts and allows wording that matches any reference. Speech recognition and translation Jun 2026 Not in cards