HAE-RAE
We track 7 benchmarks that HAE-RAE made or helped make.
0 in use 1 with public code 5 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| KMMLU Models answer 35,030 Korean multiple-choice questions across 45 subjects, collected from original Korean exams. Scoring is accuracy. | Knowledge in many languages | Feb 2024 | | Fading |
| HRM8K Models solve 8,011 math problems given as English and Korean parallel pairs. It measures how much math reasoning drops when problems are in Korean. | Word problems | Jan 2025 | | Fading |
| HAE-RAE Bench Models answer Korean questions on vocabulary, history, general knowledge and reading comprehension. It tests cultural and contextual knowledge that translated benchmarks miss. | Knowledge in many languages | Sep 2023 | | Fading |
| HRM8K-KSM Models solve the 1,428 problems of the Korean School Math subset of HRM8K. Problems come from Korean olympiad and competition exams. | Competition problems with answers | Jan 2025 | | Fading |
| KMMMU Models answer 3,466 college and exam level questions that combine images with Korean text. Questions come from Korean exam sources, some with several images. | General visual questions | Apr 2026 | | Fading |
| KMMLU-Hard Models answer about 5,000 Korean multiple-choice questions from KMMLU that most models got wrong, across many expert subjects. | Knowledge in many languages | Jan 2024 | 30.5% | Not in cards |
| CSATQA Models read Korean exam passages and choose among five answers. The harness aggregates results across six question groups. | Reading in many languages | Jul 2023 | Not in cards |