Benchmark Almanac

LLM-jp

We track 4 benchmarks that LLM-jp made or helped make.

0 in use 3 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Open Japanese LLM Leaderboard Models perform Japanese language tasks drawn from more than 20 datasets. Knowledge in many languages Nov 2024 0.67 Not in cards
JGPQA Models answer graduate-level science questions translated into Japanese from GPQA. Knowledge in many languages Sep 2025 Not in cards
JMultiWOZ-TC Models use Japanese dialogue history and past tool results to predict which tools to call next. Function calling Dec 2025 Not in cards
llm-jp-eval Models answer Japanese language understanding questions, translate passages, and follow instructions. The harness reports scores for individual datasets and groups of tasks. Reasoning in many languages Oct 2023 Not in cards