Benchmark Almanac

Ant Group

We track 7 benchmarks that Ant Group made or helped make. Its models are on its lab page.

1 in use 6 with public code 3 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
SuperMatBench Models answer materials chemistry questions in a QA part and solve materials tasks with tools in an Agent part. Research science Oct 2026 51.2% In use
VideoGAIA Agents inspect selected video segments, search the web, and read pages before answering questions. Model judges check answers, and the harness reports accuracy by category and difficulty. Video understanding Aug 2026 58.3% Not in cards
RetailBench Agents operate a simulated supermarket for up to 180 days. They choose suppliers, order stock, and set prices as goods age and customers respond. Running a business Mar 2026 $24,351 Not in cards
ConceptEdit-Bench Models edit images according to 1,000 instructions across six categories. A vision-language judge scores instruction following, visual quality, and consistency in unedited regions from 0 to 10. The per-case score averages those three dimensions. Image generation and editing Aug 2026 66.2 Not in cards
VenusBench-Mobile Agents use Android apps such as VLC, Markor, and Pro Expense. A model judges 90 primary tasks. The other tasks use programmatic checks. Phones Feb 2026 24.8% Not in cards
Fin-Eva Models answer 13,106 Chinese multiple-choice questions about financial knowledge, calculations, analysis, and compliance. The evaluator checks the chosen option against the answer key. Finance Sep 2023 Not in cards
Ming-Freeform-Audio-Edit Models edit Chinese and English speech from written instructions. The evaluator checks edits to the spoken words and changes to voice characteristics. Audio understanding and voice Sep 2025 Not in cards