xbench
We track 10 benchmarks that xbench made or helped make.
2 in use 6 with public code 8 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| BabyVision Model solves 388 perception-heavy visual puzzles (discrimination, tracking, spatial, patterns) that young children solve. Accuracy versus 94% human baseline. | Spatial reasoning and counting | Jan 2026 | | In use |
| xbench-DeepSearch Agents search the web to answer 100 Chinese questions that need multi-step retrieval. Scored by accuracy against reference answers. | Deep research | May 2025 | | Fading |
| $OneMillion-Bench Agent solves 400 expert-curated tasks in law, finance, industry, healthcare and science. Rubric grading of accuracy, logic, feasibility and compliance. | Tasks from real jobs | Mar 2026 | 42.1% | In use |
| xbench-DeepSearch-2510 Agents answer 100 new Chinese deep search questions that need many web searches and cross-checks. Answers are matched to a key. | Deep research | Oct 2025 | | Fading |
| xbench-ScienceQA Models answer postgraduate-level science questions written by experts, in Chinese, as part of the xbench AGI Tracking series. | Science questions | May 2025 | 76.4% | Not in cards |
| Monthly-SWEBench Coding agents resolve 100 fresh GitHub tasks each month, half bug fixes and half features, from newly merged pull requests. Harbor verifiers score pass@1. | Fixing real repositories | Mar 2026 | 76% | Not in cards |
| BiomniBench Agents redo data analysis tasks from 100 published biomedical studies in a Harbor environment. An LLM judge grades the whole trajectory against rubrics written with the paper authors, so method quality counts, not just the final answer. | Genomics and life sciences | May 2026 | 63.9 | Not in cards |
| xbench-Recruitment Agents do 50 headhunting tasks from live business cases: company mapping, finding facts about people and finding people who fit a profile. A judge scores answers 1 to 5. | Running a business | Jun 2025 | 78.5 | Not in cards |
| xbench-Marketing Agents search for influencers who fit 50 real advertiser briefs, scored against a curated pool of 836 influencers. | Running a business | Jun 2025 | 50.8 | Not in cards |
| AgentIF-OneDay Agents complete 104 tasks that stand for a full day of human work, life and study. Tasks include tedious workflows, inferring rules from examples and changing requirements. | General assistants | Dec 2025 | 62.6% | Not in cards |