ByteDance Seed
We track 78 benchmarks that ByteDance Seed made or helped make. Its models are on its lab page.
13 in use 27 with public code 41 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| SuperGPQA 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. | Expert-written questions | Feb 2025 | | In use |
| NL2Repo-Bench From one natural-language requirements document and an empty workspace, agent builds an installable Python library. Scored by original project's test pass rate. | Building whole projects | Dec 2025 | | In use |
| WideSearch 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. | Questions that need search | Aug 2025 | | In use |
| Beyond AIME Models answer 100 original competition-style math problems at or above the hardest AIME level. Each answer is a positive integer checked by exact match. | Competition problems with answers | Apr 2025 | | Fading |
| Multi-SWE-bench Agents resolve 1,632 GitHub issues in Java, TypeScript, JavaScript, Go, Rust, C, and C++ repos. Patches must pass the repos' tests. | Fixing real repositories | Apr 2025 | | Fading |
| FullStackBench Models write code for 3,374 problems in 16 programming languages across domains like data analysis and web development. Unit tests score each answer. | Function-level problems | Nov 2024 | | Fading |
| FinSearchComp Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. | Finance | Sep 2025 | | In use |
| Seed-TTS-eval Models clone a voice from a short prompt and speak new text in English and Mandarin. Scored by word error rate from an ASR model and by speaker similarity. | Speech recognition and translation | Jun 2024 | | Fading |
| MTVQA-Test Models answer 6,778 expert-written questions about text in 2,116 images across 9 languages, such as Arabic, Japanese, and Thai. Scoring is accuracy. | Documents, charts and text in images | May 2024 | | Fading |
| MM-BrowseComp Browsing agents answer 400 hand-written questions whose evidence sits in web images and videos. Each question has a checklist for analysis of the reasoning path. | Deep research | Aug 2025 | 24.3% | Fading |
| Inverse IFEval Models follow 1,012 Chinese and English instructions that conflict with habits learned in fine-tuning, such as writing code without comments. A model judge scores compliance. | Following precise instructions | Sep 2025 | 76.1% | Fading |
| VATEX Models write English or Chinese captions for short video clips from a set of over 41,000 videos. Scored by CIDEr. | Video captioning | Apr 2019 | | Fading |
| AetherCode Models write solutions to problems from premier programming contests such as IOI and ICPC. Expert-validated test suites judge each solution. | Competitive programming | Aug 2025 | 35.5% | Fading |
| MARS-Bench Models handle long multi-turn dialogues built from play-by-play sports commentary. Tasks test ultra multi-turn, interactive multi-turn and cross-turn reasoning. | Multi-turn conversation | May 2025 | 62.3% | Fading |
| DiscoX Models translate 200 expert texts between Chinese and English at discourse level, averaging over 1,700 tokens each. Metric-S, a reference-free judge, scores accuracy, fluency and appropriateness, and the overall score is their sum. | Translation | Nov 2025 | 76.7 | Fading |
| WorkSpaceBench Agent completes 388 tasks in realistic worker file workspaces (20k files) needing cross-file dependency reasoning. Scored against 7,399 rubrics. | Documents and spreadsheets | May 2026 | | In use |
| XpertBench Models complete expert tasks from more than 140 task types in law, finance, education, humanities and science. Over 200 domain experts wrote the prompts and weighted rubrics. | Tasks from real jobs | Dec 2025 | | Fading |
| LPFQA Models answer 430 long-tail professional questions drawn from real forum discussions in 7 academic and industrial domains. | Expert-written questions | Nov 2025 | | Fading |
| MME-CC Models answer 1,173 expert-annotated vision-grounded questions in 11 task types grouped as spatial, geometric and visual knowledge reasoning. | Spatial reasoning and counting | Nov 2025 | 40.3% | Fading |
| BABE Models reason from experimental results in peer-reviewed biology papers to draw conclusions, with causal and cross-scale inference. | Genomics and life sciences | Feb 2026 | 52.3% | Fading |
| WorldTravel Agents plan trips for 150 real-world scenarios across 5 cities with more than 15 tightly coupled time and logic constraints each. A web environment supplies multimodal travel data. | General assistants | Dec 2025 | 32.7% | Fading |
| AInstein-SWE-Bench Agents fix issues and add features in production scientific code repositories, such as quantum chemistry and astrophysics codes. Fail-to-pass tests in containers check each change. | Scientific code | Dec 2025 | | Fading |
| BIOBench (ByteDance Seed) Models answer 150 olympiad-level biology questions, 80% of them with images. Domain experts annotated and reviewed each question. | Genomics and life sciences | Dec 2025 | | Fading |
| Complex Workflow (ByteDance Seed in-house) Models carry out step-by-step tasks that follow a given standard operating procedure, such as multi-turn after-sales service in support, sales and healthcare. | Following precise instructions | Dec 2025 | | Fading |
| Customer Support Q&A (ByteDance Seed in-house) Models answer user questions from noisy enterprise knowledge base results, such as policy questions in e-commerce, government services and telecom. | Customer service | Dec 2025 | | Fading |
| Education (ByteDance Seed in-house) Models solve, grade, explain and write K to 12 problems across core subjects in teaching scenarios. | Other professions | Dec 2025 | | Fading |
| Information Extraction (ByteDance Seed in-house) Models extract words, sentences or fields from meeting records, legal texts, contracts, product descriptions and knowledge bases. | Documents and spreadsheets | Dec 2025 | | Fading |
| Encyclo-K Models judge questions composed at test time from knowledge statements extracted from authoritative textbooks. Random composition is meant to resist contamination and test several facts at once. | Expert-written questions | Dec 2025 | 62.1% | Fading |
| DREAM-1K Video models write fine-grained descriptions of 1,000 videos from varied sources. An automatic method scores how many described events match the reference. | Video captioning | Jul 2024 | 34.4% | Dropped |
| Multi-SWE-bench mini Agents fix real GitHub issues in 8 programming languages, a 400-instance subset of Multi-SWE-bench made for cheaper runs. Scored by tests passing. | Fixing real repositories | Apr 2025 | | Dropped |
| DeR2 Bench Models reason over supplied scientific documents in a sandbox that separates evidence access from reasoning. Four regimes range from instructions only to relevant documents plus distractors. | Deep research | Jan 2026 | 83.8% | Fading |
| Agent Startup Bench Agents complete workflows taken from market-validated AI-native startup products and their users. Domain experts wrote the tasks, attachments and rubrics after user interviews. | Running a business | Jun 2026 | | In use |
| SeedClawBench Agents complete 100 OpenClaw-style user tasks built from Ark usage logs and crowdsourced tasks, such as data analysis and office work. An agent judge checks rubric items against the trace and deliverables. | General assistants | Jun 2026 | | In use |
| CreativeWork Agents complete productivity tasks in Notion, Canva and Figma by mixing GUI actions with MCP tool calls. | Desktop apps | Jun 2026 | | In use |
| Doubao Multi-Turn Bench Models answer the turn of a real Doubao conversation where the user was dissatisfied, with the earlier turns as context. Rubrics grade each answer. | Multi-turn conversation | Jun 2026 | | In use |
| Image2FloorPlan Models study several real photos of a home and draw its floor plan. Scored by an average quality score. | Spatial reasoning and counting | Jun 2026 | | In use |
| xDailyBench Models answer open-ended requests grounded in real user needs across more than 30 daily-life, study and office scenarios. Rubrics grade completion, correctness, coverage and usefulness. | Overall chat quality | Jun 2026 | | In use |
| IntelligentBench Image editing models complete 350 free-form edits that need reasoning and world knowledge. GPT-4o scores each output from 0 to 2 against a reference image, normalized to 100. | Video generation | May 2025 | | Dropped |
| ZeroVideo Models answer real-world video reasoning questions that need high frame rate motion perception or multi-hop reasoning over long videos. Videos average 1,672 seconds. | Long videos | Dec 2025 | | System cards only |
| Information Processing (ByteDance Seed in-house) Models analyze large collections of user-generated content, for example summarizing viewpoints and judging overall sentiment. | Documents and spreadsheets | Dec 2025 | | Fading |
| Intention Recognition (ByteDance Seed in-house) Models classify intent from dialogues, meeting transcripts, social posts and articles, such as purchase intent in customer service chats. | Running a business | Dec 2025 | | Fading |
| LVIS-MG Models ground many objects at once by outputting boxes for named categories in LVIS images. | Detection and captioning | May 2025 | | Dropped |
| MSQA Models answer culture-specific knowledge questions in 11 major languages. | Knowledge in many languages | Jun 2026 | | In use |
| U-Artifacts Models build interactive artifacts such as small apps and pages for 199 cases across 13 topics. A GUI agent simulates a non-expert user to grade interactivity, looks and fit to the need. | Web and app front ends | Dec 2025 | | Fading |
| ViVerBench Models judge whether visual outcomes in multimodal reasoning are correct, across 16 task categories such as boxes, points, counts, mazes, GUIs, charts and LaTeX. | General visual questions | Oct 2025 | 74.4% | System cards only |
| Multi-SWE-bench flash Agents fix real GitHub issues in several languages, a 300-instance subset of Multi-SWE-bench chosen for fast evaluation and agent rollouts. Scored by tests passing. | Fixing real repositories | Jul 2025 | | Dropped |
| PatchEval Models write patches for 1,000 real CVE vulnerabilities from 2015 to 2025 in Go, JavaScript and Python. Containers run exploit and unit tests to validate each patch. | Finding and exploiting real bugs | Nov 2025 | 83.9% | In use |
| Chinese Complex Instruction Following (ByteDance Seed in-house) Models follow complex Chinese production prompts with constraints on format, conditions, content, phrasing, tone, emoji, few-shot style and length. It has 912 cases over 17 weighted dimensions. | Following precise instructions | Feb 2026 | | System cards only |
| Compositional Tasks (ByteDance Seed in-house) Models identify several elements of a business text in one inference, such as user intent and slot values. | Running a business | Feb 2026 | | System cards only |
| FinSearchComp Global Agents search for financial information about global assets across three task types. The score is the published average accuracy for the Global subset. | Finance | Sep 2025 | | Dropped |
| GDPVal-Diamond Models complete a ByteDance Seed subset of GDPval occupational tasks. Rubrics grade each deliverable automatically. | Tasks from real jobs | Feb 2026 | | System cards only |
| Past IMO (Seed-Prover formal set) Provers write Lean proofs for 155 past IMO problems, mostly adapted from Compfiles and MiniF2F. | Formal proofs | Jul 2025 | | Dropped |
| Seed1.5-VL internal benchmark (ByteDance Seed in-house) Models answer more than 12,000 internal image questions over 100 tasks, from perception, OCR and STEM to agents, code and out-of-distribution puzzles. LLM judges grade answers, and the overall is a weighted average. | General visual questions | May 2025 | | System cards only |
| Text Classification (ByteDance Seed in-house) Models analyze business text and return labels or structured outputs, for example sentiment or the main viewpoints in user content. | Running a business | Feb 2026 | | System cards only |
| UI-TARS 15 Games Collection GUI agents play 15 browser games such as 2048, Infinity-Loop and Snake-solver from screenshots. Each game score is divided by a human score, and the mean is reported. | Video games | Sep 2025 | | Dropped |
| SeedKernelBench Models write or optimize CUDA style kernels for operator tasks mined from FlashInfer, SGLang, FlashAttention, DeepGEMM, CUTLASS and cuDNN. Scored as average speedup over reference code. | GPU kernels | Jun 2026 | System cards only | |
| Trae Agent Bench Coding agents complete tasks drawn from real use of the Trae coding product, such as web development, bug fixing, code generation, artifacts and environment setup. | Fixing real repositories | Jun 2026 | System cards only | |
| AncientDoc Models transcribe historical Chinese pages, translate passages into modern Chinese, and answer questions about meaning, knowledge, and literary style. | Documents, charts and text in images | Aug 2025 | 5.77 | Not in cards |
| ReSpaceBench Models recognize rooms across separate visits and track objects that move between visits. The authors test 32 models on synthetic and real homes. | Embodied reasoning | Nov 2026 | 50.5% | Not in cards |
| SafePyramid Models identify rules violated by a conversation under a supplied policy. Rule matching rate averages four thresholds for false positive and false negative counts. | Refusing harmful requests | Jun 2026 | 54.2% | Not in cards |
| Workspace-Bench-Lite Agents complete 100 workspace tasks. The leaderboard reports the percentage of 1,856 rubric checks passed, with separate runs for each model and harness. | Documents and spreadsheets | May 2026 | 66.7% | Not in cards |
| AInsteinBench Coding agents do development tasks taken from maintainer pull requests in six widely used scientific codebases, such as quantum chemistry and molecular simulation packages. | Scientific code | Dec 2025 | 44% | Not in cards |
| FutureX Agents predict upcoming events collected daily from 195 websites in 11 domains, within a one-week window. Answers are scored once real results are crawled, with harder tiers weighted more. | Real-world events | Aug 2025 | 27.9 | Not in cards |
| RoughCut-Bench Models select and order video clips or mark speech to remove from a transcript. The evaluation compares cut plans with professional editors’ annotations. | Video understanding | Jul 2026 | | Not in cards |
| DAComp Data agents do 210 enterprise data tasks: building multi-stage SQL pipelines on industrial schemas and analyzing tables into decision-oriented insights. | Data science and SQL | Dec 2025 | 47% | Not in cards |
| HarnessDev Agents build a complete agent harness from a minimal seed, then revise their own harness using downstream execution feedback. The harness's task results are the score. | AI research tasks | Sep 2026 | 67.8% | Not in cards |
| EdgeBench Agents iterate for up to 12 hours on 134 executable real-world tasks with multi-level feedback. Scores are tracked at 2 hour steps to measure learning from the environment. | Long terminal sessions | Jul 2026 | 51.3 | Not in cards |
| SWE-Bench ProMax Agents do 170 expert-curated, large refactoring tasks from real commits in Python, Java, TypeScript, Go, C, C++ and Rust, with rewritten issue descriptions. | Fixing real repositories | Aug 2026 | 41.2% | Not in cards |
| PatchEval-Verified Models write patches for 1,000 real CVE vulnerabilities from 2015 to 2025 in Go, JavaScript and Python. Containers run exploit and unit tests to validate each patch. | Finding and exploiting real bugs | Jul 2026 | 83.9% | Not in cards |
| ReportBench Agents write literature surveys. The evaluator compares their references with published surveys and checks whether cited sources support their claims. | Deep research | Aug 2025 | 0.39 | Not in cards |
| Aspire Agent gets only a vague capability goal and must choose data, training and self-checks to evolve weights or harness. Scored on 520 hidden items. | AI research tasks | Aug 2026 | | Not in cards |
| ScribbleEdit Models edit images using a scribble as the only instruction. The evaluator checks whether the change matches the intended operation and location. | Image generation and editing | Oct 2026 | | Not in cards |
| BM-Bench Image editing models perform instruction-guided non-rigid image manipulations on the ByteMorph benchmark dataset. | Image generation and editing | Jun 2025 | Not in cards | |
| EvaLearn Models solve 648 problems in 182 sequences to measure learning from feedback across six task types. | Task length | Jun 2025 | Not in cards | |
| FinSearchComp Greater China Agents search for financial information about Greater China assets across three task types. The score is the published average accuracy for the Greater China subset. | Finance | Sep 2025 | Not in cards | |
| M3-Bench Multimodal agents answer questions about long videos while using long-term memory and multimodal retrieval. | Long videos | Aug 2025 | Not in cards | |
| RivaBench Models answer questions about stand-up routines and academic talks using speech and visual cues. They also classify clips as real or synthetic. | Video understanding | Feb 2025 | Not in cards | |
| S3Gym Models try to improve themselves through self-testing, self-judging and self-improvement in seven text-based games with executable verifiers. Held-out evaluation is kept strict. | AI research tasks | Aug 2026 | Not in cards |