Benchmark Almanac

SuperCLUE

We track 65 benchmarks and 7 arenas that SuperCLUE made or helped make.

0 in use 3 with public code 45 with a leaderboard here

Arena leaderboards

Top five in each of its 4 arenas with three or more rated models, the most rated first. We don't have a leaderboard yet for 3 others, listed below.

Benchmarks

Name Kind Released Best score Status
SuperCLUE-Math6 Models solve thousands of multi-step Chinese math word problems. Scores combine reasoning-step quality and accuracy into a comprehensive score and a reasoning level. Word problems Jan 2024 83.3 System cards only
MathCLUE Models solve problems from Chinese high school math competitions. Competition problems with answers 92.5 Not in cards
SuperCLUE-Terminal Agents complete tasks in a terminal. Terminal tasks 64.7 Not in cards
SuperCLUE-Project Models generate code for complete software projects. Building whole projects 82.1 Not in cards
SuperCLUE-Science Models solve Chinese science reasoning problems. Science questions 80.8 Not in cards
SuperCLUE-SWE Agents complete software engineering tasks in Chinese. Fixing real repositories 73.3 Not in cards
SuperCLUE-CoT Models solve Chinese reasoning problems that require several steps. Language reasoning 77.7 Not in cards
SuperCLUE-Math6o Models solve Chinese elementary school math olympiad problems. Competition problems with answers 82.2 Not in cards
SuperCLUE-VLM Vision language models interpret images and answer questions in Chinese. General visual questions 89.0 Not in cards
SuperCLUE-Hard Models follow precise instructions, solve reasoning problems and plan tool calls in Chinese. The report gives separate scores for each ability. Reasoning in many languages Dec 2024 76.7 Not in cards
SuperCLUE-LongContext Models retrieve information across multiple turns with long Chinese context. Finding facts in long inputs 51.4 Not in cards
SuperCLUE-WebCoding Models implement web interfaces from specifications. Web and app front ends 75.7 Not in cards
SuperCLUE-CPIF Models follow precise instructions written in Chinese. Following precise instructions 83.6 Not in cards
SuperCLUE-SWE with Claude Code Models solve Chinese software engineering tasks in the Claude Code harness. Fixing real repositories 75.8 Not in cards
SuperCLUE-Writing Models write original Chinese prose. Creative writing 88.0 Not in cards
AgentCLUE Agents execute tasks in Chinese. Apps and MCP servers 46.4 Not in cards
SuperCLUE-Image Models generate images from Chinese prompts. The benchmark scores text rendering, visual quality, prompt adherence and scene composition. Image generation and editing Jun 2024 94.3 Not in cards
SuperCLUE-Law Models respond to Chinese legal questions. Law 92.1 Not in cards
AgentCLUE-Mobile (offline) Agents select actions in an offline mobile interface. Phones 88.4 Not in cards
SuperCLUE-OnDevice Phone-based models respond to requests in Chinese. Overall chat quality 93.6 Not in cards
SuperCLUE-DeepSearch Agents search for information to answer Chinese research questions. Questions that need search 74.3 Not in cards
SuperCLUE-Faith Models produce answers that remain faithful to a supplied source. Faithful to sources 93.2 Not in cards
SuperCLUE-Mkt Models complete tasks in advertising and marketing. Running a business 85.7 Not in cards
SuperCLUE-OpenClaw Models complete coding, memory, and writing tasks in the OpenClaw environment. General assistants 92.3 Not in cards
SuperCLUE-SWE with mini-swe-agent Models solve Chinese software engineering tasks with the mini-swe-agent harness. Fixing real repositories 60.0 Not in cards
SuperCLUE-AdsVU Models interpret video advertisements. Video understanding 89.1 Not in cards
SuperCLUE-Edit Models edit an input image according to a Chinese instruction. The evaluation measures whether the requested change preserves relevant details. Image generation and editing Oct 2025 82.5 Not in cards
SuperCLUE-Fact Models answer short Chinese questions and identify false statements involving knowledge, common sense, misleading prompts and context. Short factual questions Apr 2025 86.0 Not in cards
SuperCLUE-I2V Models generate videos from still images. Video generation 71.5 Not in cards
AgentCLUE-Task Agents carry out interactive tasks in Chinese. Apps and MCP servers 49.5 Not in cards
SuperCLUE-V Models answer Chinese questions about visual inputs. General visual questions 75.1 Not in cards
EmbodiedCLUE-World World models predict outcomes in embodied settings. Interactive world models May 2026 76.2 Not in cards
SuperCLUE-Agent Scenario Agents complete tasks drawn from Chinese application scenarios. General assistants 96.6 Not in cards
SuperCLUE-Code Scenario Models write code for application scenarios. Building whole projects 86.7 Not in cards
EmbodiedCLUE-VLA (planning) Models reason about actions in physical environments. Embodied reasoning 86.8 Not in cards
SuperCLUE-DSPSafeBench Models respond to Chinese prompts that test content safety. Deception and alignment 78.6 Not in cards
SuperCLUE-TTS Models read Chinese text aloud. The benchmark rates basic speech synthesis and delivery in situations such as public announcements. Audio understanding and voice Dec 2024 69.7 Not in cards
SuperCLUE-VLR Models solve problems that require visual reasoning. General visual questions 72.1 Not in cards
SuperCLUE-World World models generate visual sequences. Video world models 74.6 Not in cards
SuperCLUE-ASR Models transcribe Chinese speech under different acoustic conditions. Speech recognition and translation 12.1% Not in cards
SuperCLUE-ICabin Models respond to requests for an automotive cabin assistant. General assistants 82.2 Not in cards
SuperCLUE-VideoEdit Models edit videos in response to Chinese instructions. Video generation 74.2 Not in cards
EmbodiedCLUE-VLA (tabletop tasks) Robot policies carry out tasks on a tabletop. Real robots 49.3 Not in cards
SuperCLUE-DeepSWE Agents complete extended software engineering tasks in Chinese. Fixing real repositories 84.3 Not in cards
SuperCLUE-R2V Models generate videos conditioned on reference material. Video generation 64.9 Not in cards
AgentCLUE-General Agents complete general purpose tasks in Chinese. General assistants 4.62 Not in cards
EmbodiedCLUE-World 2.0 World models predict outcomes in embodied settings under the second evaluation protocol. Interactive world models Jul 2026 41.8 Not in cards
SuperCLUE-200K Models retrieve information from long Chinese inputs. Finding facts in long inputs 9.98 Not in cards
SuperCLUE-ComicShorts Models generate short animated comics. Video generation 38.5 Not in cards
SuperCLUE-Long Models answer questions about long Chinese texts. Reasoning over long documents 68.4 Not in cards
SuperCLUE-Voice (speech interaction) Models interact with users through speech in real time. Speech recognition and translation 73.0 Not in cards
AgentCLUE-Mobile Mobile assistants complete tasks on a phone. Phones Not in cards
Chinese Language Generation Evaluation (CLGE) Models generate Chinese paper titles from scientific abstracts. The evaluation compares the generated titles with references using ROUGE and BLEU. Reading in many languages Dec 2019 Not in cards
EmbodiedCLUE-VLA RoboTwin Evaluation Robot policies perform ten two-handed manipulation tasks under randomized RoboTwin 2.0 Hard conditions. The score averages task success rates over twenty episodes per task. Simulated robots Aug 2026 Not in cards
pCLUE Models follow Chinese prompts for classification, inference, generation and reading comprehension. The leaderboard combines exact match and accuracy across tasks. Reading in many languages Oct 2022 Not in cards
SuperCLUE-AISearch Search systems answer questions in Chinese. Questions that need search Not in cards
SuperCLUE-Coder Coding assistants complete programming tasks in Chinese. Building whole projects Not in cards
SuperCLUE-DeepResearch Agents research questions and assemble reports in Chinese. Deep research Not in cards
SuperCLUE-LarkClaw Claw assistants complete tasks through Feishu. Apps and MCP servers Not in cards
SuperCLUE-MedAssist Medical assistants respond to Chinese healthcare questions. Medicine Not in cards
SuperCLUE-Office Agents process documents, analyze data, and write office materials. Documents and spreadsheets Not in cards
SuperCLUE-R1 Search DeepSeek R1 services answer questions that require search and reasoning. Questions that need search Not in cards
SuperCLUE-Voice (audiovisual interaction) Models respond to audio and video inputs in real time. Audio understanding and voice Not in cards
SuperCLUE-WeClaw Claw assistants complete tasks through WeChat. Apps and MCP servers Not in cards
SuperCLUE-XClaw Claw assistant products complete user tasks in Chinese. General assistants Not in cards

Arenas without a leaderboard here

Name Kind Released Best score Status
SuperCLUE Arena Image to Video Models animate images. The arena ranks systems from pairwise comparisons. Video generation Not in cards
SuperCLUE Arena Reference to Video Models generate videos from references. The arena ranks systems from pairwise comparisons. Video generation Not in cards
SuperCLUE Arena Text to Video Models generate videos from text. The arena ranks systems from pairwise comparisons. Video generation Not in cards