Meituan
We track 19 benchmarks that Meituan made or helped make. Its models are on its lab page.
1 in use 12 with public code 10 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| VITA-Bench Agents handle multi-turn requests in simulated food delivery, in-store, and travel services with 66 tools. A rubric-based judge scores task success. | Customer service | Sep 2025 | | Fading |
| AMO-Bench Models solve 50 original math problems at or above IMO difficulty that need only a final answer. Answers are graded automatically for accuracy. | Competition problems with answers | Oct 2025 | 52.4% | Fading |
| LongCat safety evaluation (Meituan) Models answer private test queries in four risk categories: harmful content, criminal activity, misinformation and privacy. The score is the share of safe responses in each category. | Refusing harmful requests | Sep 2025 | | Dropped |
| Meeseeks Models follow complex instructions over several turns and get feedback on failed requirements after each turn. Scored by how many requirements they meet. | Multi-turn conversation | Apr 2025 | | Dropped |
| UNO-Bench Models answer 1,250 human-written omni-modal questions and 2,480 uni-modal questions across 44 task types and 5 modality mixes. Multiple choice and multi-step open questions are scored, the open ones by a scoring model. | Audio understanding and voice | Oct 2025 | | Dropped |
| Random Complex Tasks (LongCat) Agents solve 100 tasks sampled from a generator that builds random executable tool environments across more than four domains. Scored by Avg@4, averaged over three runs. | Apps and MCP servers | Jan 2026 | | Fading |
| tau2-Noise (LongCat) Agents run tau2-bench tasks after a pipeline injects instruction noise and tool failures into the environment. The score is the average over several noisy copies. | Customer service | Jan 2026 | | Fading |
| VitaBench-Noise (LongCat) Agents run VitaBench life-service tasks after a pipeline injects instruction noise and tool failures into the environment. The score is the average over several noisy copies. | Apps and MCP servers | Jan 2026 | | Fading |
| Math-VR Models answer 5,000 English and Chinese math questions whose solutions need visual reasoning, such as drawing auxiliary lines or plotting functions. Answers are checked against ground truth. | Visual math and diagrams | Oct 2025 | | System cards only |
| VitaBench 2.0 Agents serve one simulated user across a sequence of life-service subtasks over days or months. They must infer, use and update the user's preferences, judged by rubrics. | Customer service | May 2026 | 50.3% | In use |
| Real-time Audio-Visual Interaction Evaluation Trained conversationalists hold three minute live video calls with a model. Real users rate each session for naturalness and fluency on a 0 to 3 scale. | Speech recognition and translation | Oct 2025 | | System cards only |
| General365 Models solve 365 hand-written reasoning problems and 1,095 variants in eight categories that need only K-12 knowledge. Scoring mixes rule-based and model-based checks. | Abstract puzzles and logic | Apr 2026 | | Not in cards |
| LocalSearchBench Agents search merchant listings to answer 900 questions about local services across nine Chinese cities. A model judges each answer against a reference. | Questions that need search | Dec 2025 | | Not in cards |
| Meeseeks Utility Rate Models follow complex instructions with feedback for up to 20 turns. The score measures fully usable responses as a percentage. | Multi-turn conversation | Apr 2025 | 90.5% | Not in cards |
| WBench Interactive video world models run 289 multi-turn cases with navigation, subject actions, event edits and perspective switches. 22 metrics cover quality, adherence, consistency and physics. | Interactive world models | May 2026 | | Not in cards |
| PRISM-Bench (audio and video generation) Models generate speech, music, or environmental sounds with video. A judge compares outputs with reference clips across 35 criteria, including synchronization and prompt following. | Video generation | May 2026 | | Not in cards |
| LARYBench Models answer expert questions in the LARYBench dataset released by Meituan LongCat. | Expert-written questions | Mar 2026 | Not in cards | |
| OutboundEval Agents conduct outbound calls with simulated users. The benchmark scores compliance with the professional workflow and the quality of the interaction. | Multi-turn conversation | Oct 2025 | Not in cards | |
| PosterBench Agents turn research papers into posters. Programmatic checks and visual model judgments score seven content and design dimensions, with ceilings for serious rendering or layout failures. | Documents and spreadsheets | Aug 2026 | Not in cards |