Benchmark Almanac

SuperCLUE team

We track 9 benchmarks that SuperCLUE team made or helped make.

0 in use 9 with public code 8 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
SuperCLUE-Fin Models answer Chinese finance questions across 25 tasks in six categories. The evaluation covers financial knowledge and its application in practice. Finance Apr 2024 88.7 Not in cards
SuperCLUE-Code3 Models write executable Python functions from Chinese specifications. Unit tests check functionality, and the leaderboard reports results by difficulty. Function-level problems Feb 2024 68% Not in cards
SuperCLUE-Role Models portray characters in Chinese conversations. A judging model rates aspects of the portrayal against reference answers. Chat and instructions in many languages Apr 2024 75.2 Not in cards
Math24o Models solve Chinese high school mathematics olympiad preliminary problems from 2024. A program compares their final answers with reference answers. Competition problems with answers Mar 2025 85.7% Not in cards
SuperCLUE-Industry Models answer multi-turn questions about industrial work in Chinese. A judging model applies task-specific criteria. Tasks from real jobs Mar 2024 74.8 Not in cards
SuperCLUE-Video Models generate videos from Chinese prompts. A judging model examines video frames and scores them against criteria for the prompt. Video generation Feb 2024 54.3 Not in cards
SuperCLUE-Auto Models answer Chinese automotive questions through multiple turns. A judging model scores each answer against criteria for its task. Other professions Dec 2023 76.6 Not in cards
SuperCLUE-Open Models answer Chinese open-domain questions over multiple turns. The authors compare models across skill categories. Chat and instructions in many languages Jul 2023 94.6 Not in cards
SuperCLUE-CPIFOpen Models answer Chinese prompts with precise instructions. The published evaluation code checks their responses. Following precise instructions Jul 2025 Not in cards