Benchmark Almanac

Seoul National University

We track 8 benchmarks that Seoul National University made or helped make.

0 in use 5 with public code 6 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
KoBALT Models answer 700 expert-written multiple-choice questions on 24 Korean linguistic phenomena in syntax, semantics, pragmatics, phonology and morphology. Reasoning in many languages May 2025 61% Fading
K-Viscuit Models answer multiple-choice visual questions about Korean culture. Questions were written through human and VLM collaboration. General visual questions Jun 2024 89.5% Fading
Rule2DRC Models write KLayout scripts from chip design rules. The evaluator checks whether each script flags the correct violations on every test layout. Electronics May 2026 67.9% Not in cards
TutorCaseBench Tutors respond to students whose behavior interferes with learning. Case-specific rubrics grade the responses, and a separate track tests selection among possible interventions. Other professions Oct 2026 59.4% Not in cards
PReMind-Bench Models inspect a scene and decide whether to retrieve a personal memory before answering. Judges check whether the answer addresses the hazard and uses the right person’s information. Wellbeing and sycophancy Nov 2026 87.3% Not in cards
HumanoidToolBench Humanoid robot policies pick a suitable tool and use it, walking when needed, across 18 tasks in simulation and on a real Unitree G1. Real robots Oct 2026 49.4% Not in cards
ExFunTube Models explain why a moment in a short video is funny. The authors compare explanations with human references and ask viewers to rate them. Video captioning Jul 2023 Not in cards
Ref-YouTube-VOS Models segment and track the object that a sentence describes in YouTube-VOS videos. Scored by J&F on a server-graded validation split. Video understanding Aug 2020 Not in cards