KAIST
We track 15 benchmarks that KAIST made or helped make.
2 in use 10 with public code 12 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| CLIcK Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. | Knowledge in many languages | Mar 2024 | | In use |
| KorMedMCQA Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. | Medicine | Mar 2024 | 92.7% | In use |
| BLEnD Models answer 52.6k questions about everyday culture in 16 countries and regions across 13 languages. Scoring is accuracy on short-answer and multiple-choice formats. | Knowledge in many languages | Jun 2024 | | Dropped |
| K-MetBench Models answer 1,774 questions from the Korean National Meteorological Engineer Examination under a zero-shot protocol. The benchmark reports answer accuracy across five subjects and separate visual and Korean-specific subsets. Expert rationales support an additional rubric score for reasoning. | Science questions | Apr 2026 | | Not in cards |
| MentalBench (Psychiatric Diagnosis) Models select psychiatric diagnoses for 24,750 synthetic cases. Exact matching requires both diagnoses when the case lacks the evidence needed to distinguish them. | Medicine | Feb 2026 | 62.7% | Not in cards |
| WearableQA Models answer 4,084 ten-option questions using longitudinal wearable measurements, blood biomarkers, and demographics from 200 users. Questions test computations and physiological interpretation across single or multiple signals. Accuracy compares the selected option with the answer key. | Medicine | Sep 2026 | | Not in cards |
| ProgramDistill Agents inspect working web applications and restore missing features in editable copies. Replayed browser traces check whether patches recover the reference behavior. The paper evaluates partial repairs and full application reconstruction. | Web and app front ends | Sep 2026 | 84.3% | Not in cards |
| FinRED Financial Safety Models answer prompts about financial risks. A finance-specific judge grades each response and reports the percentage that assist harmful conduct. | Deception and alignment | Jun 2026 | 0.15% | Not in cards |
| AnyAppBench Agents repeat the same task in different Android apps, with and without suggested steps. Each app has a verifier that checks the same success condition. | Phones | Sep 2026 | | Not in cards |
| B-MoCA Agents use Android apps such as Instagram on phones with different languages, font sizes, and icon positions. Task-specific checks decide whether each attempt succeeded. | Phones | Apr 2024 | 47.7% | Not in cards |
| EHRNoteQA Models read a patient's accumulated MIMIC-IV discharge summaries and answer clinical questions about them. The set has 962 question and answer pairs, each for a different patient. | Medicine | Feb 2024 | 96.2% | Not in cards |
| ReTrace Computer agents recover from injected errors in desktop tasks. Replaying the remaining actions checks whether the recovered state produces the original successful outcome. | Desktop apps | Oct 2026 | 63.6% | Not in cards |
| PerMemBench Memory systems for long-running agents decide what to store from years of interactions across many domains and user personas. Later questions check whether the system kept the context that matters for each user. The benchmark tests whether memory policies can adapt to the person instead of storing everything the same way. | Learning from the context | May 2026 | | Not in cards |
| AssurAI Models respond to Korean prompts across 35 risk categories. Model judges rate the safety of text, image, audio, and video outputs from 1 to 5. | Deception and alignment | Nov 2025 | Not in cards | |
| SoCRATES Models mediate disputes in 600 scenarios across eight domains. A model judge rates topic agreement to measure consensus gain, intervention timeliness, and intervention effectiveness. | Multi-turn conversation | Jun 2026 | Not in cards |