Benchmark Almanac

Android Agent Arena team

We track 2 benchmarks that Android Agent Arena team made or helped make.

0 in use 2 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Android Agent Arena (100 tasks) Agents use 20 Android apps to find information and perform actions. A vision model checks whether the agent reached the required intermediate states and completed the task. Phones Aug 2025 53% Not in cards
Android Agent Arena (201 tasks) Agents find live information and perform actions in Android apps such as YouTube Music and Booking.com. Task functions check app state and grade answers. Phones Jan 2025 10.3% Not in cards