Benchmark Almanac

Chinese Academy of Sciences

We track 7 benchmarks that Chinese Academy of Sciences made or helped make.

0 in use 7 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
LongDocURL Models answer 2,325 questions over long documents with more than 33,000 pages in total. Its 20 subtasks cover understanding, numerical reasoning and locating document elements. Documents, charts and text in images Dec 2024 64.5% System cards only
RGB Models answer English and Chinese news questions from retrieved passages, testing noise robustness, negative rejection, information integration, and counterfactual robustness. Scored by accuracy. Faithful to sources Sep 2023 Dropped
Doc2Harness Models build a coding-agent harness from an interface skeleton and a development document. The frozen harness then runs 328 scripted test cases in separate containers. AI research tasks Oct 2026 53% Not in cards
RealBench (Verilog) Models write Verilog modules or complete systems from design specifications. Testbenches check behavior, and a formal checker compares the result with the reference implementation. Electronics Jul 2025 6.65% Not in cards
USIM underwater robot tasks Agents follow instructions to navigate, grasp, transport, and track objects underwater. The simulator measures task completion across 700 trials. Simulated robots Oct 2025 37.6% Not in cards
ChipGPTV Models write Verilog from circuit diagrams and descriptions with different levels of detail. Testbenches check whether the generated circuits behave correctly. Electronics Jul 2024 71.8% Not in cards
MER-UniBench Models describe emotions from audio, video, and text across nine datasets. Label normalization supports hit rate, weighted F1, and emotion-wheel F1. The reported mean averages the primary metric from each dataset. Audio understanding and voice Jan 2025 Not in cards