Benchmark Almanac

Moonshot AI

We track 41 benchmarks and 2 arenas that Moonshot AI made or helped make. Its models are on its lab page.

10 in use 5 with public code 9 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
WorldVQA Model names visual entities in 3,500 image-question pairs across nine categories, isolating memorized visual world knowledge. Exact-answer accuracy. General visual questions Jan 2026 61.1% In use
PerceptionBench (Moonshot) Moonshot in-house benchmark of atomic visual perception skills. Scoring details unpublished. General visual questions Jul 2026 59.7% In use
Kimi Code Bench v2 Coding agents complete Moonshot's realistic tasks from internal engineering work, production incidents, and open-source projects in over ten languages. Fixing real repositories Jun 2026 77.1% In use
CombiBench Provers solve 100 combinatorics problems formalized in Lean 4, from middle school to IMO and university level. Lean checks each proof. Formal proofs May 2025 48% Fading
Swarm Bench Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Multi-agent games Feb 2026 76.3% System cards only
Kimi Code Bench 3.0 Long-horizon Coding agents carry out long multi-step engineering tasks with tools across several engineering domains. Moonshot reports a percentage score. Building whole projects Oct 2026 66.5% In use
Kimi Hallu Bench Models answer hard prompts drawn from practical work tasks. Fact-checking agents score the share of claims that are hallucinated. Hallucination rates Oct 2026 4.20% In use
xdev-bench-hard Agents get vulnerable source code, a short bug hint and the binary, and must write an exploit that reaches code execution in a sandbox. Success is checked by retrieving a secret string. Finding and exploiting real bugs Oct 2026 100% In use
Kimi Claw 24/7 Bench Agents work on 17 persistent multi-day professional scenarios through OpenClaw, with 610 checkpoints. The score is the mean pass rate over three runs. Tasks from real jobs Jun 2026 46.9% In use
Kimi-Audio speech conversation evaluation People rate spoken replies on speed, accent and emotion control, empathy and style control. Scored on a 1 to 5 scale and averaged. Speech recognition and translation Apr 2025 3.90 System cards only
Business Operation (Moonshot) Agents manage a simulated business over a long horizon and make operating decisions. Moonshot reports a percentage score. Running a business Oct 2026 37.8% In use
24/7 ClawBench 2.0 Agents do always-on assistant work through OpenClaw. Tasks span several days, events arrive at the same time, and interruptions are routine. Tasks from real jobs Jul 2026 48.3% System cards only
Agent Behavior Bench Agents are scored on process quality, including tool-use behavior, efficiency and discipline, together with task completion. General assistants Jul 2026 65% System cards only
Agentic Vision Bench (Moonshot) Agents must notice and correctly use key visual facts while they execute tasks. General visual questions Jul 2026 78.3% System cards only
Chat All-in-One Bench Models handle conversational scenarios built around real online user needs at every stage of product usage. Overall chat quality Jul 2026 85.2% System cards only
CLIF Bench Models learn from a provided context while they follow instructions that combine several complex skills. Learning from the context Jul 2026 52.4% System cards only
Coding Experience (Moonshot) Coding agents work in real development workflows. The score reflects the practical experience of working with the agent, such as communication and stability. Fixing real repositories Jul 2026 59.9% System cards only
DECK Bench Models produce presentation decks from task descriptions drawn from real usage. The score measures deck quality. Documents and spreadsheets Jul 2026 73.5% System cards only
DECK-Arena Models build presentations for real user queries with python-pptx, pptxjs or a Kimi slide language, with search and image generation. An automated grader rates them as Elo scores. Documents and spreadsheets Oct 2026 1,456 In use
Deep Research Bench (Moonshot) Agents answer deep-research queries curated by domain experts. Expert-aligned rubrics grade the reports. Deep research Jul 2026 90% System cards only
Design Bench (Moonshot) Models make graphic designs from direct prompts or few-shot reference images. An automated grader rates the results as Elo scores. Taste and style Oct 2026 957 In use
Faithfulness (Moonshot) A fact checker verifies each model response for factual hallucinations. The score is one minus the hallucination rate. Hallucination rates Jul 2026 85.5% System cards only
Finance Bench (Moonshot) Agents complete realistic financial workflows end to end, from source materials to deliverables that reviewers can check. Finance Jul 2026 62.6% System cards only
KAET (Kimi Autonomous Execution Tasks) Agents execute long-horizon tasks that simulate real user requests and enterprise system operations. General assistants Jul 2026 83.5% System cards only
Kimi exploit development suite Agents turn real CVEs into working exploits in 16 user-space tasks and 20 Linux kernel privilege-escalation tasks. Experts verified every task solvable. Finding and exploiting real bugs Jul 2026 38.9% System cards only
KWV Bench Models answer atomic visual questions extracted from tasks distilled from real knowledge-work scenarios. Documents, charts and text in images Jul 2026 64.7% System cards only
Longform Writing Models write long-form text without tools, capped at 32k output tokens. Moonshot AI has not published the tasks or the scoring method. Creative writing Nov 2025 73.8% Dropped
MIRA Bench Agents carry out long multi-role enterprise tasks across several systems and decide when to delegate to subagents. Simulated companies Jul 2026 64.1% System cards only
Online Experience (Moonshot) Agents produce the deliverable file types that real online users request most often, following the distribution of real usage. Documents and spreadsheets Jul 2026 77.9% System cards only
Kimi Promptfoo red-team safety evaluation Promptfoo generates attack prompts for harmful, criminal, misinformation, privacy and security plugins under five strategies. Human reviewers judge the responses. Jailbreaks and prompt injection Jul 2025 System cards only
Kimi vulnerability discovery (Tier 1) Agents search current versions of widely deployed software for new bugs and show that they reproduce. Humans review the findings. Finding and exploiting real bugs Jul 2026 System cards only
Kimi-ASR Internal Testset Models transcribe speech from two internal Moonshot test subsets. Scored by word error rate, lower is better. Speech recognition and translation Apr 2025 System cards only
TCSExam-Lite Agents solve 20 theoretical computer science tasks, five per track. The default solver protocol gives hidden feedback only after submission. AI research tasks Nov 2026 72.3 Not in cards
TCSExam Agents prove theorems, write solvers, build mathematical constructions and address open conjectures in theoretical computer science. AI research tasks Nov 2026 71.9 Not in cards
Kimi-ASR Internal Testset Subset 1 Models transcribe speech from Moonshot internal subset 1. The score is the published word error rate. Speech recognition and translation Apr 2025 1.42% Not in cards
Kimi-ASR Internal Testset Subset 2 Models transcribe speech from Moonshot internal subset 2. The score is the published word error rate. Speech recognition and translation Apr 2025 2.44% Not in cards
AI Office Benchmark (Moonshot) Agents complete end-to-end office tasks and produce documents, spreadsheets, PDFs and slide decks. Experts rate output quality. Documents and spreadsheets Jan 2026 Not in cards
General Agent Benchmark (Moonshot) Agents complete multi-step production workflows. Experts compare their work with human expert performance. Tasks from real jobs Jan 2026 Not in cards
Internal Finance Modeling (Moonshot) Agents complete financial modeling tasks with linked formulas, financial logic and final workbook quality. Experts rate the work. Finance Oct 2026 Not in cards
Internal Financial Research (Moonshot) Agents produce auditable financial research under evidence and cutoff constraints. Experts rate the resulting research. Finance Oct 2026 Not in cards
Kimi Code Bench Moonshot in-house coding-agent tasks from internal engineering cases, production incidents and open-source projects across 10+ languages. 80 tasks. Fixing real repositories Not in cards

Arenas

Name Kind Released Best score Status
Kimi Webdev Bench Models build games, 3D scenes and website clones from real prompts. Blind experts compare two outputs and report win, tie and loss rates. Web and app front ends Jul 2026 58.6% System cards only
Kimi Chinese in-house open-ended benchmark Models answer held-out Chinese prompts drawn from real user queries. Outputs are compared pairwise against another model to give win, tie and loss rates. Overall chat quality Jul 2025 System cards only