Peking University
We track 44 benchmarks that Peking University made or helped make.
2 in use 39 with public code 28 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| TempCompass Video models answer questions about temporal aspects of short clips, such as action order, speed, direction and attribute change, in several question formats. | Video understanding | Mar 2024 | | Fading |
| NOVA-63 Models answer questions written by native speakers in 14 languages across 63 academic disciplines. Scoring is accuracy. | Knowledge in many languages | Nov 2025 | | Fading |
| SysBench Models follow 500 system messages across five user turns each. A model verifier checks constraints, instructions and whole sessions. | Following precise instructions | Aug 2024 | 81.2% | Fading |
| Omni-MATH Models solve 4,428 Olympiad-level competition math problems across 33 subdomains and 10 difficulty levels. A GPT-4 or Omni-Judge grader checks answers. | Competition problems with answers | Oct 2024 | 60.5% | Dropped |
| CFBench Models follow 1,000 real-world instructions with embedded constraints, across more than 200 scenarios and 50 NLP tasks. A judge checks each constraint and the overall requirement. | Following precise instructions | Aug 2024 | 71.8% | Dropped |
| SaaS-Bench 106 long-horizon GUI workflows across 23 locally deployed real SaaS apps. Scored from app databases with weighted checkpoints and full-resolution rate. | Web browsing | May 2026 | 31.1% | In use |
| WISE Text-to-image models draw 1,000 prompts that need world knowledge in culture, space, time and natural science. A vision model judge gives the WiScore. | Image generation and editing | Mar 2025 | 0.82 | System cards only |
| ImgEdit-Bench Image editing models carry out edit instructions such as add, remove, replace, style transfer and background change. A GPT-4.1 judge scores each result from 1 to 5. | Image generation and editing | May 2025 | | System cards only |
| SuperMatBench Models answer materials chemistry questions in a QA part and solve materials tasks with tools in an Agent part. | Research science | Oct 2026 | | In use |
| VideoHallucer Paired basic and hallucinated questions about videos detect intrinsic and extrinsic hallucinations. Scored by pair accuracy. | Video understanding | Jun 2024 | 59.6% | Fading |
| Echo General AI Prediction Leaderboard Models forecast unresolved events drawn from prediction markets, synthesized news questions and expert-written questions. Forecasts made at aligned times are compared in pairwise battles, and an Elo fit ranks the models. | Real-world events | Mar 2026 | | Not in cards |
| Accelerate Bench Coding agents speed up 32 real scientific applications, such as GROMACS and OpenFOAM, on a fixed CPU allocation within 8 hours. A hidden verifier checks correctness, and Score@1 averages the share of runtime saved, with incorrect programs scoring zero. | Speed and optimization | Nov 2026 | 67.4% | Not in cards |
| HWE-Bench v1.0 (RTL bug repair) Agents fix bugs in six open-source hardware projects. The benchmark reports the share of fixes that pass repository tests in simulation. | Electronics | Apr 2026 | 79.9% | Not in cards |
| HelloBench Models generate long answers, summaries, conversations, text completions, and texts from writing instructions. Model judges apply task-specific checklists to the responses. | Overall chat quality | Sep 2024 | 48.5 | Not in cards |
| WISE Verified Models generate images from the revised WISE prompts. The verified evaluation changes about 200 prompts and the judge, so original scores remain separate. | Image generation and editing | Apr 2026 | 0.82 | Not in cards |
| OSWorld-MCP Computer-use agents complete OSWorld desktop tasks while they can also call 158 MCP tools across 7 common applications. The benchmark checks whether agents pick a tool call or GUI actions at the right time. It reports task accuracy, tool invocation rate and average completion steps. | Desktop apps | Oct 2025 | 56.6% | Not in cards |
| HWE-Bench v1.1 (RTL bug repair) Agents fix hardware bugs in 417 repository tasks. The revised tests check whether each fix restores the required behavior. | Electronics | Nov 2026 | 99% | Not in cards |
| OmniEchoBench Models answer 2,972 spatial questions from recorded audio and video or navigate toward sound sources in 900 episodes. Rules score extracted spatial answers and multiple-choice letters. Navigation metrics measure success within one meter and path efficiency. | Embodied reasoning | Sep 2026 | | Not in cards |
| RoughCut-Bench Models select and order video clips or mark speech to remove from a transcript. The evaluation compares cut plans with professional editors’ annotations. | Video understanding | Jul 2026 | | Not in cards |
| VibeVideoBench Agents write code to render a finished video with sound, with no image or video generation model, in their native harness within one hour. Program checks, task rubrics and a judge panel score the result. | Video generation | Oct 2026 | 4.11 | Not in cards |
| DataPrep-Bench Models prepare training data across six domains, and a base model is fine-tuned on their output. Downstream results of that model give the score. | Machine learning engineering | May 2026 | | Not in cards |
| K12-Bench Models select curriculum concepts, prerequisite relationships, related knowledge, experimental evidence, and textbook locations. The 23,640 multi-select questions come from a textbook knowledge graph. Exact match and instance-level F1 compare selected options with graph-derived answers. | Language reasoning | May 2026 | | Not in cards |
| PCA-Bench Models interpret visual scenes, explain their reasoning, and select actions for embodied tasks. GPT-4-0125 compares perception concepts, reasoning, and actions with annotated references. | Embodied reasoning | Feb 2024 | 0.70 | Not in cards |
| ProOPF-B Models translate operational requirements into executable optimal power flow programs. They change grid parameters, objectives, and constraints. The evaluator checks implementations against expert references. | Scientific code | Feb 2026 | | Not in cards |
| DataPrep-Bench Finance Models prepare training data for the finance domain. The score is the published downstream average after backbone training. | Machine learning engineering | May 2026 | 59.3 | Not in cards |
| DataPrep-Bench General Models prepare training data for the general domain. The score is the published downstream average after backbone training. | Machine learning engineering | May 2026 | 78.2 | Not in cards |
| DataPrep-Bench Law Models prepare training data for the law domain. The score is the published downstream average after backbone training. | Machine learning engineering | May 2026 | 76.4 | Not in cards |
| DataPrep-Bench Math Models prepare training data for the math domain. The score is the published downstream average after backbone training. | Machine learning engineering | May 2026 | 25.5 | Not in cards |
| DataPrep-Bench Medical Models prepare training data for the medical domain. The score is the published downstream average after backbone training. | Machine learning engineering | May 2026 | 40.5 | Not in cards |
| DataPrep-Bench Science Models prepare training data for the science domain. The score is the published downstream average after backbone training. | Machine learning engineering | May 2026 | | Not in cards |
| CMI-RewardBench Models judge musical quality and how well recordings follow requests. Some requests combine text, lyrics, and reference audio. Scores measure agreement with human listeners. | Audio understanding and voice | Feb 2026 | | Not in cards |
| AndroidArena Agents use apps such as Gmail and Google Maps, sometimes switching apps or following extra constraints. GPT-4 judges success from the recorded actions and observations. | Phones | Feb 2024 | | Not in cards |
| ContinualSkillBench Agents work through five domains of 100 linked subtasks of rising difficulty, testing whether skill libraries they build in context actually help later tasks. | Learning from the context | Aug 2026 | 64.1% | Not in cards |
| Duplex-MPE Full-duplex speech models join conversations of three or four people and must decide when to answer, stay silent or stop speaking. The set has 2,000 scenarios. | Speech recognition and translation | Sep 2026 | | Not in cards |
| EmbodiedCLBench Self-evolving embodied agents learn tasks in sequence in simulation. The test measures whether they keep past experience and transfer it to new tasks. | Simulated robots | Oct 2026 | 52% | Not in cards |
| LongAV-Compass Models generate minute-long videos with sound from text, an image or a video. Scoring covers identity consistency, narrative coherence and audio-visual alignment over time. | Video generation | May 2026 | | Not in cards |
| RMBench Policies run 9 simulated bimanual manipulation tasks on RoboTwin 2.0 that need memory of earlier observations, at several levels of memory complexity. Success rate. | Simulated robots | Mar 2026 | 10.4% | Not in cards |
| ChemCoTBench Models interpret and edit molecular structures, optimize molecular properties and predict reaction outcomes. Each task has its own scoring rule. | Science questions | May 2025 | Not in cards | |
| Chinese Literature Named Entity Recognition and Relation Extraction Models mark named entities in Chinese literary passages and identify relationships between them. The authors score both tasks with F1. | Reading in many languages | Nov 2017 | Not in cards | |
| ChronoMagic-Bench Video models generate time-lapse scenes from 1,649 prompts covering biological, human-made, weather, and physical changes. Automatic metrics measure the amount of change and temporal coherence, with a separate GPT-4o assessment of change. | Video generation | Jun 2024 | Not in cards | |
| MindWorldBench Models animate a scene to show how a person would act given their beliefs or intentions. Paired prompts change mental states while holding the scene fixed. | Video generation | Sep 2026 | Not in cards | |
| PhysCapBench Models caption videos of physical interactions. The benchmark checks which motions, contacts, and changes of state the captions describe across 1,500 clips. | Video captioning | Nov 2026 | Not in cards | |
| TraceSpatial-Bench Models plan an object-centric 3D trace, an ordered list of metric 3D waypoints, from language instructions in real cluttered indoor scenes with depth. | Embodied reasoning | Dec 2025 | Not in cards | |
| TriWorldBench World models generate synchronized robot-view videos from head and wrist cameras. Nineteen signals measure cross-view agreement, task alignment, physical coherence, motion, temporal stability, and visual quality. | Video world models | Sep 2026 | Not in cards |