Benchmark Almanac

Beijing Jiaotong University

We track 3 benchmarks that Beijing Jiaotong University made or helped make.

0 in use 2 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
AndroidWorld-Knowledge Agents attempt 48 AndroidWorld tasks across five apps. Each app provides guidance for its simplest task, and harder tasks test whether the agent can reuse it. Phones May 2025 Not in cards
Mobile-Eval Agents follow 33 instructions on Android. Some requests specify the app. Others describe a goal and leave the choice of app to the agent. Phones Jan 2024 Not in cards
Mobile-Eval-v2 Agents follow basic and advanced instructions on real phones. The authors give 44 instructions in English on Android and 44 in Chinese on HarmonyOS. Phones Jun 2024 Not in cards