CLUE benchmark team
We track 13 benchmarks that CLUE benchmark team made or helped make.
0 in use 13 with public code 4 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| CLUEWSC Models decide whether a pronoun in a Chinese sentence refers to a given noun phrase, a Winograd-style task. Scored by accuracy. | Reasoning in many languages | Mar 2020 | | Fading |
| SuperCLUE The CLUE team tests Chinese and foreign language models on periodic sets of Chinese questions covering language, professional knowledge, reasoning, agents and safety. Open answers are graded by a strong LLM judge, and results feed an overall ranking. | Composite indices | May 2023 | | Not in cards |
| SuperCLUE-Agent Chinese language models are tested on agent skills in Chinese tasks. These include tool use with several API calls, task planning and memory over long dialogues. | Function calling | Oct 2023 | 78.4 | Not in cards |
| SuperCLUE-Safety Chinese language models answer 4,912 open-ended safety questions over more than 20 sub-dimensions in multi-round adversarial dialogues. Follow-up turns push harder than the first prompt. | Jailbreaks and prompt injection | Sep 2023 | | Not in cards |
| SuperCLUE-RAG Chinese models answer questions twice, first without documents and then with supplied documents. A judge model compares the answers to rate how well the system uses retrieved text. | Faithful to sources | Mar 2024 | 66.7 | Not in cards |
| ZeroCLUE Models perform Chinese language tasks without training on labeled examples from the target task. | Reading in many languages | 30.9 | Not in cards | |
| CLUE Models are fine-tuned or prompted on nine Chinese language understanding tasks, including sentence classification, sentence-pair inference and machine reading comprehension. The score averages the task results. | Reading in many languages | Apr 2020 | Not in cards | |
| CLUENER2020 Models identify named entities in Chinese text across ten categories. | Reading in many languages | Not in cards | ||
| DataCLUE Systems improve the training data for Chinese text classification tasks. | Machine learning engineering | Not in cards | ||
| EPRSTMT Models classify Chinese product reviews as positive or negative after learning from a small labeled sample. Accuracy measures agreement with the sentiment labels. | Reading in many languages | May 2021 | Not in cards | |
| KgCLUE Models answer Chinese questions using a knowledge graph. | Questions that need search | Not in cards | ||
| LGEB Models generate summaries and questions from source passages. | Reading comprehension | Not in cards | ||
| QBQTC Models judge how relevant a page title is to a Chinese search query. | Questions that need search | Not in cards |