Benchmark Almanac

Upstage

We track 9 benchmarks that Upstage made or helped make. Its models are on its lab page.

1 in use 0 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
KBank-MMLU (Upstage) Models answer Korean finance questions in a multiple-choice format. Upstage built the set in house. Finance Jul 2025 80.8% Fading
Ko-AIME 2025 (Upstage) Models solve AIME 2025 competition problems in Korean. Upstage built the set in house. Competition problems with answers Jan 2026 97.7% Fading
Ko-GDPval Agents complete 170 Korean office tasks from 58 occupations in 11 industries and produce deliverable files. Rubrics grade data, content, structure, quality and format. Tasks from real jobs Jul 2026 87.3% In use
Ko-Arena-Hard v2 (Upstage) Models answer hard user prompts in Korean, judged against a baseline in the style of Arena-Hard v2. Upstage built it in house. Chat and instructions in many languages Dec 2025 79.9% Fading
Ko-IFEval (Upstage) Models follow verifiable instructions written in Korean. Upstage adapted IFEval to Korean for the Open Ko-LLM Leaderboard2. Chat and instructions in many languages Oct 2024 64% System cards only
Ko-AIME 2024 (Upstage) Models solve AIME 2024 competition problems translated into Korean. Upstage built the set in house. Competition problems with answers Dec 2025 80.3% System cards only
Open Ko-LLM Leaderboard 2 Models complete nine Korean evaluations covering reasoning, instructions, social values, safety, and other abilities. The organizers score submitted models on private test data. Reasoning in many languages Aug 2024 49.6% Not in cards
Open Ko-LLM Leaderboard Models answer Korean questions about knowledge, truthfulness, and commonsense reasoning. The leaderboard uses translated tasks alongside a Korean commonsense benchmark. Knowledge in many languages Sep 2023 53% Not in cards
DP-Bench Document parsers and vision models convert PDFs and scanned pages into structured HTML or Markdown. Output is compared with reference structure. Documents, charts and text in images Oct 2024 Not in cards