Benchmark Almanac

StepFun

We track 21 benchmarks that StepFun made or helped make. Its models are on its lab page.

2 in use 10 with public code 10 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
AndroidDaily Agents complete 350 daily tasks in 94 real closed-source Android apps. A process-aware reviewer model checks each trajectory against written guidelines. Phones Nov 2025 61.9% Fading
GEdit-Bench Image editing models follow real-world user edit instructions. A model judge scores semantic consistency, perceptual quality and an overall score. Image generation and editing Apr 2025 6.52 Dropped
KRIS-Bench Image editing models follow 1,267 instructions that need factual, conceptual or procedural knowledge across 22 tasks. A GPT-4o judge scores the edits. Image generation and editing May 2025 66.2 System cards only
CF-Div2-Stepfun Models solve 53 Codeforces Div. 2 problems from September 2024 to February 2025. An offline judge with rebuilt test cases scores C++, Python and Java submissions. Competitive programming Feb 2026 86.1% System cards only
StepCodeBench Models do software engineering tasks in 553 repositories, across 9 task types, 20 application domains and 33 languages. Task types include feature changes, bug repair, refactoring and environment setup. Fixing real repositories Sep 2026 63.9% In use
Consulting and Recommendations Benchmark (StepFun) Models answer 500 advice queries taken from Reddit, Stack Exchange and forums. Model judges score usefulness, logic, instruction following and tone against an expert reference set at 88. Overall chat quality Feb 2026 70.5% System cards only
Data Analysis Benchmark (StepFun) Agents in Claude Code solve 50 business data-analysis tasks with CSV data. A model grader scores outputs against expert rubrics averaging 26.9 items a task. Data science and SQL Feb 2026 39.6% System cards only
FinStepBench Models do three kinds of finance work: retrieving and checking live financial facts, building company valuations, and writing full research reports. Graders check accuracy, sourcing and reproducible calculations. Finance Sep 2026 63.6% In use
Step-SWE-Bench Models fix software issues inside several coding agent harnesses, such as Claude Code, OpenCode and Hermes Agent. The score is the average over harnesses. Fixing real repositories May 2026 67.1% Fading
WebVR Models recreate a webpage from a screen recording and supplied assets. A model judge scores the rendered page against visual and interaction criteria. Web and app front ends Mar 2026 79.1 Not in cards
GEBench Image generation models create interactive GUI states from prompts. A judge scores goal, logic, consistency, interface and quality across five task types. General visual questions Feb 2026 63.2 Not in cards
Panda-CVL Models accept correct image-grounded responses or locate and replace their first erroneous text span. The test expands 652 conversations into 2,126 instances. Its overall F1 combines correct acceptance with exact correction of faulty responses. General visual questions Sep 2026 17.1% Not in cards
FinStepBench CorporateValuation Models value companies using financial evidence and calculations. The score is the published CorporateValuation percentage. Finance Sep 2026 77.3% Not in cards
FinStepBench DeepResearch Models write finance research reports. The score is the published DeepResearch percentage. Finance Sep 2026 59.1% Not in cards
FinStepBench LiveSearch Models retrieve and verify live financial facts. The score is the published LiveSearch percentage. Finance Sep 2026 76.7% Not in cards
AutoVend Agents buy stock and schedule deliveries for 20 simulated vending machines over 180 days. They share one warehouse and pay for each site visit. Running a business Oct 2026 309.4 Not in cards
ViStoryBench Models turn stories and character references into coherent image sequences. Model judgments and image features measure prompt alignment, character consistency, style, and image quality. Image generation and editing May 2025 2.00 Not in cards
Step-Caption Audio models describe speakers' vocal characteristics across language, age, rate, rhythm, pitch, timbre, emotion and related attributes. A judge compares the description with expert labels. Audio understanding and voice Apr 2026 Not in cards
Step-DU Audio models answer targeted questions about a speaker's vocal characteristics in dialogue. A judge scores answers against reference labels. Audio understanding and voice Apr 2026 Not in cards
Step-SPQA Audio models answer paralinguistic questions about age, emotion, events, gender, pitch, rhythm, scene, speed, voice sounds and style. Audio understanding and voice Apr 2026 Not in cards
StepEval-Audio-360 Speech models answer multi-turn Chinese prompts about instruction following, voice understanding, reasoning, role play, creativity and speech control. Audio understanding and voice Feb 2025 Not in cards