THUDM
We track 4 benchmarks that THUDM made or helped make.
0 in use 4 with public code 0 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| LongBench Models work through 21 English and Chinese long-input datasets covering QA, summarization, few-shot learning, synthetic tasks, and code. Each dataset has its own metric. | Reasoning over long documents | Aug 2023 | 81.1% | Dropped |
| LongBench English Models answer questions and complete other long-context tasks in the original English headline table. The score is its published average on a percentage scale. | Reasoning over long documents | Aug 2023 | | System cards only |
| HumanEval-X Models generate programs or translate solutions among Python, C++, Java, JavaScript, and Go. The 820 human-written language variants include executable tests, and scoring uses pass@k. | Function-level problems | Mar 2023 | 9.78% | Not in cards |
| LongBench Chinese Models answer questions and complete other long-context tasks in the original Chinese headline table. The score is its published average on a percentage scale. | Reasoning over long documents | Aug 2023 | Not in cards |