Benchmark Almanac

Tencent

We track 99 benchmarks and 1 arena that Tencent made or helped make. Its models are on its lab page.

18 in use 55 with public code 46 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
ArtifactsBench Models generate interactive visual code artifacts for 1,825 tasks. An MLLM judge scores rendered screenshots and code against a per-task checklist. Web and app front ends Jul 2025 66.8% In use
CL-bench Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. Learning from the context Feb 2026 27.9% In use
Video-Holmes Models answer 1,837 questions about 270 suspense short films across seven reasoning tasks. Each question needs several visual clues from different parts of the film. Video understanding May 2025 68.2% Fading
C3 Models answer 19,577 multiple-choice questions on Chinese dialogues and texts from Chinese-as-a-second-language exams. Scoring is accuracy. Reading in many languages Apr 2019 97.8% Dropped
SEED-Bench Models answer 19,000 human-annotated multiple-choice questions over 12 dimensions of image and video understanding. Scored by accuracy against the answer key. General visual questions Jul 2023 70.4% Fading
DPG-Bench Text-to-image models draw 1,000 dense prompts with many objects, attributes and relations. A VQA model checks each element of the prompt graph in the image. Image generation and editing Mar 2024 84.2% Fading
SEED-Bench-2-Plus Multimodal models answer multiple-choice questions about text-rich images such as charts, maps and web pages. Documents, charts and text in images Apr 2024 73.8% System cards only
Hy-SWE Max Coding agents solve 300 Tencent in-house software tasks in Claude Code with a 200-turn budget and no web access. Hidden verifier tests grade each task. Fixing real repositories Apr 2026 64.2% In use
Math23K Models solve Chinese elementary-school math word problems from a set of about 23,000. Scored by final-answer accuracy. Word problems Sep 2017 88.5% System cards only
C3-Bench Agents complete multi-task tool use dialogues built to stress tool dependencies, long context dependencies and frequent switches between policy types. Function calling May 2025 12.5% Dropped
CL-bench Life Models solve 405 tasks that require learning from messy real-life context such as group chats. A model judge checks rubrics to give a solve rate. Learning from the context Apr 2026 22.2% Fading
E-Bench (Tencent) Agents complete Tencent in-house work tasks in the working agent group. Task and scoring details are not published. Documents and spreadsheets Jul 2026 80.6% In use
ComplexNLU (Tencent) Models answer Tencent in-house natural language understanding tasks with complex inputs. Task and scoring details are not published. Reading comprehension Jun 2025 61.2% Dropped
Hy-Backend 2.0 Coding agents complete Tencent in-house backend development tasks in the Claude Code scaffold. Task and scoring details are not published. Fixing real repositories Jul 2026 35.2% In use
InsCtrl (Tencent) Models write text under detailed instructions in a Tencent in-house text creation test. Task and scoring details are not published. Following precise instructions Jun 2025 71.9% Dropped
LengthCtrl (Tencent) Models write text that must meet a requested length, in a Tencent in-house test. Task and scoring details are not published. Creative writing Jun 2025 55.4% Dropped
Mandarin-Minority translation (Tencent) Models translate between Mandarin and Chinese minority languages in a Tencent test set. XCOMET-XXL scores each translation. Translation Sep 2025 60.8% Dropped
Word-Task (Tencent) Models solve Tencent in-house word level language tasks. Task and scoring details are not published. Reading comprehension Jun 2025 62.9% Dropped
UniGenBench Text-to-image models draw images for 600 prompts across 5 themes and 20 subthemes, in English and Chinese and in short and long forms. A multimodal model judge checks each image on 10 primary and 27 secondary semantic dimensions, such as attributes, actions, relationships, layout, text and logical reasoning. Image generation and editing Aug 2025 95.8% System cards only
ChartMimic Multimodal models write plotting code that reproduces 1,000 charts from scientific papers. Scores compare the rendered chart with the original. Web and app front ends Jun 2024 60.3 System cards only
E-Bench-Code (Tencent) Agents complete Tencent in-house work tasks that need code. Task and scoring details are not published. Documents and spreadsheets Aug 2026 83.3% In use
InFoBench Models follow 500 instructions that are split into 2,250 yes or no requirement questions. A judge answers each question, and the score is the share met. Following precise instructions Jan 2024 89.4% System cards only
CHAOS-Bench Models read deliberately corrupted words in 500 academic page images. Scores measure faithful recognition of the visible strings instead of correction to plausible words. Documents, charts and text in images Jul 2026 29.8% In use
PenguinScrolls Models answer user-style questions over long natural documents such as books, financial reports and legal papers, up to 128K tokens. Tasks cover extraction, localization, qualitative analysis and numerical reasoning. Reasoning over long documents Nov 2024 87.7% System cards only
DSBench Data science agents solve 466 data analysis tasks and 74 end-to-end data modeling tasks drawn from Eloquence and Kaggle competitions. Analysis is scored by accuracy and modeling by relative performance gap. Data science and SQL Sep 2024 89.9% Dropped
Hunyuan-MT Challenge Testset (Tencent) Models translate between Chinese and English across everyday and specialist domains. Expert raters score each translation from 0 to 4 on marked error-prone points. Translation Sep 2025 3.19 System cards only
Hy-Backend Coding agents complete Tencent in-house backend development tasks. Task and scoring details are not published. Fixing real repositories Apr 2026 54.7% Fading
Hy-BrowseComp-Pro2 Search agents answer hard browsing questions in a Tencent in-house set named after BrowseComp. Task and scoring details are not published. Deep research Aug 2026 56.1% In use
Hy-CompanyBench Coding agents complete Tencent in-house company engineering tasks. Task and scoring details are not published. Building whole projects Jul 2026 41.7% In use
Hy-CompanyBench V2 Coding agents complete a second version of Tencent in-house company engineering tasks. Task and scoring details are not published. Building whole projects Aug 2026 62.4% In use
Hy-Euler pro Models with tools solve Tencent in-house STEM problems. Task and scoring details are not published. Research mathematics Jul 2026 24.2% In use
Hy-FinAgentBench Agents complete Tencent in-house finance analyst tasks. Task and scoring details are not published. Finance Aug 2026 79.7% In use
Hy-FinModelBench Agents build financial models for Tencent in-house finance tasks. Task and scoring details are not published. Finance Jul 2026 69% In use
Hy-FinmodelBench v2 Agents build financial models for a second version of Tencent in-house finance tasks. Task and scoring details are not published. Finance Aug 2026 57% In use
Hy-LifeSearch Search agents answer Tencent in-house everyday life questions with web search. Task and scoring details are not published. Deep research Aug 2026 49.2% In use
Hy-Math Models solve Tencent in-house math problems without tools. Task and scoring details are not published. Competition problems with answers Jul 2026 60.9% In use
Hy-SkillsWorld Agents use skills to complete Tencent in-house working tasks. Task and scoring details are not published. General assistants Jul 2026 45.8% In use
Hy-Vibe Bench Coding agents complete development workflows drawn from real Tencent users. Task and scoring details are not published. Building whole projects Apr 2026 50.5% Fading
ProdBench (Tencent) Agents complete Tencent in-house productivity tasks in the OpenClaw harness. Scored by pass^3. General assistants Jul 2026 23% In use
GameHorizon Models are tested on gameplay over several time horizons. Tasks combine visual understanding, instruction decomposition, planning and precise action control, using offline data instead of noisy rollouts. Video games Sep 2026 80.2 Not in cards
AutoCodeBench Models solve automatically generated hard coding problems across many programming languages. A sandbox runs generated tests on each solution. Function-level problems Aug 2025 52.4% Not in cards
ElephantBench Models answer 1,094 closed-book questions about facts with multiple verified but conflicting accounts. A model judge assigns complete, partial, or failed recall. Short factual questions Aug 2026 52.4% Not in cards
ArtifactsBench (Gemini 2.5 Pro 0506 judge) Models generate interactive visual code artifacts for 1,825 tasks. An MLLM judge scores rendered screenshots and code against a per-task checklist. Web and app front ends Jul 2025 57% Not in cards
Long-Horizon-Terminal-Bench (LHTB) Terminal-Bench-style container tasks needing hundreds of steps. Subtasks graded deterministically for dense partial-credit reward. Solved at reward >=0.95. Long terminal sessions Jul 2026 60.2% Not in cards
PatternEval Models answer 2,415 image prompts in thinking and non-thinking modes. A model judge flags exposed draft reasoning, repetition, contradictions, and unsupported reasoning. The Trigger rate measures responses with any flagged failure. Overall chat quality Aug 2026 0.78% Not in cards
A.S.E (AICGSecEval) Models generate code inside real GitHub projects at spots taken from CVE patches. Static analysis then checks whether the generated code is secure. Secure code and review Jul 2025 63.0 Not in cards
Accelerate Bench Coding agents speed up 32 real scientific applications, such as GROMACS and OpenFOAM, on a fixed CPU allocation within 8 hours. A hidden verifier checks correctness, and Score@1 averages the share of runtime saved, with incorrect programs scoring zero. Speed and optimization Nov 2026 67.4% Not in cards
GameXpert-Bench Agents build games from 97 briefs, repair bugs in 100 tasks, and improve 17 games over six requests each. Each track reports its own scores. Building whole projects Aug 2026 70.9 Not in cards
Long-Horizon-Terminal-Bench 2.0 Agents work for up to 3 hours per task on 77 container tasks, including 30 tasks that rebuild a scientific tool from its outputs. Hidden verifiers give partial-credit rewards. Long terminal sessions Oct 2026 69.2% Not in cards
LLM Uncertainty Bench Models answer questions across five language tasks. The original benchmark measures accuracy and uncertainty, while the port checks only the selected answer. Language reasoning Jan 2024 81.5% Not in cards
Tencent WorkBuddy Bench Coding agents do tasks in four work domains: code, web, office and security. Each task is rebuilt from a real commit or business scenario as a short request that resists web search. Fixing real repositories Jul 2026 81.6% Not in cards
WorkBuddyBench-Office Agents edit office files and reconcile records across a workspace. One task requires updating an eleven-slide presentation from a spreadsheet while preserving unrelated slides. Documents and spreadsheets Jul 2026 86.4% Not in cards
DeskCraft Desktop agents complete 538 Ubuntu tasks, including 152 tasks with later user clarifications or interruptions. Programmatic checks inspect final application state, saved files, and exported artifacts. Desktop apps Jun 2026 29.8% Not in cards
PlanningBench Models solve generated planning problems that coordinate goals, constraints and resources. Instances are scalable and come with verifiers. Language reasoning May 2026 63.2% Not in cards
IFMTBench Models translate text while following explicit instructions, such as keeping JSON or HTML structure, using a glossary, using provided context and matching a register. It covers seven languages with 4,506 single-constraint and 2,838 multi-constraint items. Deterministic checkers verify some constraints, and a rubric-based model judge scores the rest. Translation May 2026 81.1% Not in cards
MirageBench Models personalize six kinds of responses from three disclosed facts about each of 150 personas. A model judge classifies claims by evidential support. The primary over-inference rate counts stereotyped and fabricated claims. Faithful to sources Aug 2026 35.1% Not in cards
ExplorationBench Agents explore verifiable alien worlds with hidden laws, framing hypotheses and running experiments. The worlds are built so that recalled pretraining knowledge does not help. Research science Sep 2026 89% Not in cards
FinHopper Agents research financial facts through up to six dependent searches. Intermediate answer keys show where an incorrect company, date, or indicator derails the search. Finance Oct 2026 67.3% Not in cards
AD-Bench Agents answer advertising analytics requests with production-platform tools. A model judge checks final answers against refreshed reference answers and measures coverage of expert tool-call trajectories. Results report success over one or three attempts. Tasks from real jobs Feb 2026 83% Not in cards
PPTGen-Bench Agents create slide decks from text requests. Tencent renders the decks to PDF with LibreOffice. A judging agent assesses appearance and applies penalties for serious faults. Documents and spreadsheets Nov 2026 67.7 Not in cards
ExcelFixBench Agents repair formulas, data, and formatting in an existing workbook. The task rubric checks the repair and whether unrelated content survived. Documents and spreadsheets Nov 2026 27.8 Not in cards
KnowU-Bench Agents use Android apps such as Mattermost and Taodian. They infer preferences, ask for missing details, and decide when to act or stay silent. Phones Apr 2026 60.4% Not in cards
VWE-Bench Multimodal agents build interactive 3D worlds from a user request. They infer intent, plan the scene layout, call 3D tools for asset retrieval, editing and rendering, and revise from rendered feedback. The benchmark has 2,616 3D assets, 323 human-annotated seed worlds and 6,828 queries. Verified queries have ground truth, and the rest are scored with rubrics. CAD models Aug 2026 57.3% Not in cards
WordEditBench Agents edit Word documents. A rule-based checker reads the document XML, rewards requested changes, and penalizes changes to unrelated content. Documents and spreadsheets Nov 2026 57.2 Not in cards
ArtEdit-Bench-Lr Models retouch photos from English or Chinese instructions. GPT-4o scores instruction following and visual quality. Pixel errors measure fidelity to reference edits. Apps and MCP servers Nov 2025 8.62 Not in cards
OmniVBench Video generators make videos that follow many kinds of reference inputs at once. Scoring checks that each reference factor is kept, separated and placed correctly. Video generation Sep 2026 72.7 Not in cards
SEED-Bench-2 Multimodal models answer multiple-choice questions that cover comprehension of interleaved image and text inputs and image generation, in hierarchical capability levels. General visual questions Nov 2023 71.9% Not in cards
ORAV Bench Models combine reference images, sounds, and videos into a requested audiovisual scene. A judge compares pairs of outputs against each reference's assigned role. Video generation Sep 2026 0.70 Not in cards
WeGenEditBench Models edit images from instructions that can include text or multiple references. A shared checklist scores each requested change and the details that should stay intact. Image generation and editing Oct 2026 4.14 Not in cards
WorkBuddy User Blind Evaluation Agents complete everyday work requests. WorkBuddy users rate their satisfaction on a four-point scale without seeing the model identity. Documents and spreadsheets Nov 2026 3 Not in cards
EvalVerse Video models generate cinematic clips from prompts and reference material derived from professional videos. Film experts compare outputs across visual design, acting, cinematography, aesthetics, emotion, shot sequences, and sound. A calibrated vision-language evaluator also estimates these judgments. Video generation May 2026 0.73 Not in cards
Honor of Kings Bug AI Root Cause Analysis Agents use the Honor of Kings Bug AI platform to identify the cause of code defects. The team measures how often agents identify the correct cause. Tasks from real jobs Nov 2026 73.8% Not in cards
WeRef-Bench Models locate objects described in English or Chinese and return no boxes when the image contains no match. Detection and captioning Oct 2026 50.7% Not in cards
ArtEdit-Bench-Eval Models rate photo edits against English instructions. Spearman and Pearson correlations measure agreement with human ratings on 200 cases. Image generation and editing Nov 2025 0.62 Not in cards
PhoneHarness Bench Agents complete Android workflows through graphical controls, device commands, and structured tools. Checks inspect files, system settings, application state, and unintended side effects. Phones Jun 2026 62.1% Not in cards
WMT 2023 Discourse-Level Literary Translation Models translate chapters of Chinese web novels into English. Professional translators mark errors and rank systems within each submission group. Translation Apr 2023 54.8 Not in cards
A.S.E 2.0 Models and agentic coding tools write code in real projects covering 29 CWE types. A mix of static analysis, tests and vulnerability proofs of concept checks security. Secure code and review Nov 2025 Not in cards
AppAgent Test Set Agents follow 45 instructions across nine Android apps, including scheduling email and changing music playback. Tasks time out after ten steps. Phones Dec 2023 Not in cards
AudioVideoBench Agents produce and edit audio and video in containers. The tasks include compositing, mixing, repairing media, reframing shots, and transcribing recordings. Tasks from real jobs Nov 2026 Not in cards
CSL Models generate paper titles and keywords or classify Chinese scientific papers by field. Scoring uses ROUGE-L for titles, Bpref for keywords, and accuracy for classification. Reading in many languages Sep 2022 Not in cards
EvalCrafter Text-to-video models generate clips for 700 prompts drawn from real user requests. The clips are scored with 17 automatic metrics for visual quality, content quality, motion and text alignment. Weights fitted to human ratings combine the metrics into one score. Video generation Oct 2023 Not in cards
GroupAssistBench Assistants use memories from group conversations to answer questions and complete real-world group assistance tasks. Model judges score grounded helpfulness. Learning from the context Nov 2026 Not in cards
Honor of Kings Game Configuration Agents configure game data with the Honor of Kings spreadsheet platform. The team measures configuration accuracy before and after the agent checks and repairs its work. Tasks from real jobs Nov 2026 Not in cards
Hy-Nomas-Bench Interactive video world models follow text and camera controls across multiple turns and retain spatial memory. Interactive world models Oct 2026 Not in cards
HY3D-Bench Models generate textured 3D meshes from text and images, with semantic and geometric quality evaluated across categories. Image generation and editing Feb 2026 Not in cards
Joi-Bench-Task Voice agents use tools while a simulated caller changes a request or interrupts a reply. The evaluator tracks completion, responsiveness, and omitted information. Speech recognition and translation Nov 2026 Not in cards
MdCoEditBench Agents edit selected Markdown in the WorkBuddy editor alongside a user. Tasks include restoring a version, resolving an edit conflict, and continuing existing text. Documents and spreadsheets Nov 2026 Not in cards
OSWorkerBench Desktop agents complete long-horizon office tasks across business applications with executable evaluators and optional demonstrations. Desktop apps Aug 2026 Not in cards
PPTCoEditBench Agents edit selected content in PowerPoint slides. Tasks include changing a title or table while preserving surrounding shapes and animations. Documents and spreadsheets Nov 2026 Not in cards
PPTFixBench Agents inspect a PowerPoint file and repair its formatting and content. Each of the 50 files contains more than 25 issues. Documents and spreadsheets Nov 2026 Not in cards
RVMS-Bench Agents search open-domain videos and localize moments using global impression, key moment, temporal context and auditory memory cues. Web browsing Mar 2026 Not in cards
SongBench Models generate songs from lyrics and a style prompt. A trained music judge scores seven aspects, including the melody, arrangement, vocals, and mix. Audio understanding and voice Mar 2026 Not in cards
TabFact Models decide whether a statement is entailed or refuted by a Wikipedia table. Classification accuracy measures agreement with human labels. Faithful to sources Sep 2019 Not in cards
TaxoBench Agents find research papers and organize them into a taxonomy. The evaluator compares their groupings and hierarchy with taxonomies from 72 published surveys. Deep research Jan 2026 Not in cards
VCB-Bench Audio language models answer spoken prompts across instruction following, empathy, safety, simulation and vocalization tasks. Judges score the responses. Audio understanding and voice Oct 2025 Not in cards
VulnGym White-box agents hunt vulnerabilities in real repositories at vulnerable commits, including business-logic bugs that need cross-module reasoning. Each sample has a human-reviewed entry point and trace. Finding and exploiting real bugs Apr 2026 Not in cards
WeChat Pay Physical Client Evaluation Agents operate the WeChat Pay client on physical phones. The team scores execution paths, use of supplied knowledge, acceptance checks, and account creation accuracy. Phones Nov 2026 Not in cards
WMT 2024 Discourse-Level Literary Translation Models translate Chinese web fiction into English, German, or Russian. Translators rate English outputs for general quality and for consistency across the passage. Translation May 2024 Not in cards
WorkIndex Agents complete everyday WorkBuddy tasks. Tencent combines eight tests covering office files, audio and video work, and visual questions. Composite work indices Nov 2026 Not in cards

Arenas

Name Kind Released Best score Status
3DArena-Modeling Agents create 3D models for 120 tasks covering precision and appearance. Tencent converts Elo ratings from blind comparisons into percentages. Spatial reasoning and counting Nov 2026 Not in cards