Benchmark Almanac

iFlytek

We track 7 benchmarks that iFlytek made or helped make. Its models are on its lab page.

3 in use 4 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
CMRC Models extract answer spans from Chinese Wikipedia paragraphs for about 20,000 human-written questions. Scoring is exact match and F1. Reading in many languages Oct 2018 76.3% Dropped
Gaokao Math 2026 Models sit the five 2026 Chinese Gaokao mathematics papers (National I, National II, Beijing, Shanghai and Tianjin). Each paper is graded out of 150 points. Competition problems with answers Sep 2026 134.9 In use
PAgentBench Agents act as personal assistants for task management, file operations and system interactions. iFlytek built it in house. General assistants Oct 2026 74.1% In use
SWebBench Agents browse the web to retrieve information and do research tasks. iFlytek built it in house. Deep research Oct 2026 71.7% In use
Chinese Machine Reading Comprehension 2017 Models fill missing words or answer questions about Chinese children’s reading passages. Each track has 3,000 test questions and uses answer accuracy. Reading in many languages Apr 2017 Not in cards
Chinese Machine Reading Comprehension 2019 Models restore missing sentences in Chinese stories. Some options come from elsewhere in the same story. The organizers score individual blanks and entirely correct passages. Reading in many languages May 2019 Not in cards
People’s Daily and Children’s Fairy Tale Models fill a missing word using a Chinese news passage or a fairy tale. The fairy tales test whether a reader trained on news can transfer. Reading in many languages Jul 2016 Not in cards