Benchmark Almanac

CLUE benchmark team

We track 13 benchmarks that CLUE benchmark team made or helped make.

0 in use 13 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
CLUEWSC Models decide whether a pronoun in a Chinese sentence refers to a given noun phrase, a Winograd-style task. Scored by accuracy. Reasoning in many languages Mar 2020 92.8% Fading
SuperCLUE The CLUE team tests Chinese and foreign language models on periodic sets of Chinese questions covering language, professional knowledge, reasoning, agents and safety. Open answers are graded by a strong LLM judge, and results feed an overall ranking. Composite indices May 2023 73.6 Not in cards
SuperCLUE-Agent Chinese language models are tested on agent skills in Chinese tasks. These include tool use with several API calls, task planning and memory over long dialogues. Function calling Oct 2023 78.4 Not in cards
SuperCLUE-Safety Chinese language models answer 4,912 open-ended safety questions over more than 20 sub-dimensions in multi-round adversarial dialogues. Follow-up turns push harder than the first prompt. Jailbreaks and prompt injection Sep 2023 89.3 Not in cards
SuperCLUE-RAG Chinese models answer questions twice, first without documents and then with supplied documents. A judge model compares the answers to rate how well the system uses retrieved text. Faithful to sources Mar 2024 66.7 Not in cards
ZeroCLUE Models perform Chinese language tasks without training on labeled examples from the target task. Reading in many languages 30.9 Not in cards
CLUE Models are fine-tuned or prompted on nine Chinese language understanding tasks, including sentence classification, sentence-pair inference and machine reading comprehension. The score averages the task results. Reading in many languages Apr 2020 Not in cards
CLUENER2020 Models identify named entities in Chinese text across ten categories. Reading in many languages Not in cards
DataCLUE Systems improve the training data for Chinese text classification tasks. Machine learning engineering Not in cards
EPRSTMT Models classify Chinese product reviews as positive or negative after learning from a small labeled sample. Accuracy measures agreement with the sentiment labels. Reading in many languages May 2021 Not in cards
KgCLUE Models answer Chinese questions using a knowledge graph. Questions that need search Not in cards
LGEB Models generate summaries and questions from source passages. Reading comprehension Not in cards
QBQTC Models judge how relevant a page title is to a Chinese search query. Questions that need search Not in cards