Huawei Noah's Ark Lab
We track 3 benchmarks that Huawei Noah's Ark Lab made or helped make.
0 in use 3 with public code 2 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| DEEPSYNTH Agents search multiple websites and combine evidence to answer 120 research tasks across seven domains. They return structured JSON answers scored with precision, recall, F1, and exact match. A separate model judge assesses answers beyond exact matching. | Deep research | Feb 2026 | 8.97% | Not in cards |
| FollowBench Models answer 820 English and Chinese instructions where each level adds one more constraint of content, situation, style, format or example type. Rules and an LLM judge check each constraint. | Following precise instructions | Oct 2023 | 73.4% | Not in cards |
| SPA-Bench Agents use Android apps such as Airbnb, Spotify, and WeChat. The grader matches screenshots against task requirements and uses a vision model to resolve uncertain cases. | Phones | Oct 2024 | 1.35% | Not in cards |