Computer use
Models operate a computer or phone through its screen. We count 149 benchmarks here. A benchmark belongs here when environment includes desktop apps, browser or phone apps, and input includes screens.
How fast it grows
31 were released since July, more than the 30 in the first half of 2026.
How much is still open
No model has passed 85% on 86 of the 104 benchmarks here that score in percent, 83% of them.
Every benchmark in this view
Newest first. Facets come from each benchmark’s own documents, and our methodology says how we tag them.
| Benchmark | Level | Released | Best score | Status |
|---|---|---|---|---|
| Browser Use LinkedIn Agents answer plain-language questions about LinkedIn using recorded page snapshots. The leaderboard reports correctness, cost, and time per task. | Everyday | Nov 2026 | 97.9 | Not in cards |
| FORTE-MM Agents complete multimodal office tasks across 14 professions. A run passes only when the agent's output satisfies every task rubric. | University | Nov 2026 | | Not in cards |
| Kaeru Flip (paprika3d) Agents play an 86-note rhythm game from browser pixels. They must finish the song with at most one miss and leave autoplay off. | Everyday | Nov 2026 | Not in cards | |
| WeChat Pay Physical Client Evaluation Agents operate the WeChat Pay client on physical phones. The team scores execution paths, use of supplied knowledge, acceptance checks, and account creation accuracy. | University | Nov 2026 | Not in cards | |
| ArenaWebdevCorrectness Judges inspect generated applications and predict whether their features work. Human labels provide the reference answers. | University | Oct 2026 | 81.6% | Not in cards |
| BrowseSafe-Adapt Browser agents face adaptive prompt injections across 1,024 tasks. The score is attack success within 64 attempts. | University | Oct 2026 | 30.7% | Not in cards |
| CLASHarness Computer-use agents work in multi-agent setups where peers report stale states, give contradictory instructions or compete for files. Agents must check claims against the environment. | Everyday | Oct 2026 | 28.7% | Not in cards |
| Gym-Anything-RT Vision-language agents play 100 real-time browser minigames by screenshots and mouse and keyboard input. Each game also has a paused, non-real-time counterpart, so the gap shows the cost of acting in real time. | Everyday | Oct 2026 | 9.00% | Not in cards |
| Personal Superintelligence Index (Meta) Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO. | Expert | Oct 2026 | In use | |
| Realtime-Bench Agents act in 50 live game and web tasks where a late action fails: 30 web tasks (travel, shopping, services and security) and 20 games. The environment keeps moving while the model thinks. Three clocks separate speed from skill: Wall Clock runs on real time, Paused Clock stops while the model thinks, and Token Clock advances with each generated token. Scores are task success rates over three runs. | Everyday | Oct 2026 | 42.7% | Not in cards |
| ReTrace Computer agents recover from injected errors in desktop tasks. Replaying the remaining actions checks whether the recovered state produces the original successful outcome. | Everyday | Oct 2026 | 63.6% | Not in cards |
| Web Clone Bench Agent gets a running reference app (Gmail, Jira, Figma, etc.) and 8 hours to build a clone. Expert-written workflows are replayed as end-to-end tests. | Research frontier | Oct 2026 | 62% | Not in cards |
| WildAWM validation Agents complete 33 held-out computer-use tasks in generated software environments. Harbor packages the validation split separately from the training tasks. | Expert | Oct 2026 | | Not in cards |
| AnyAppBench Agents repeat the same task in different Android apps, with and without suggested steps. Each app has a verifier that checks the same success condition. | Everyday | Sep 2026 | | Not in cards |
| BU Bench V2.1 Browser agents do 200 real web tasks. A findings judge checks tool evidence, answers and files, and deterministic integrity checks run as well. | Everyday | Sep 2026 | 81.3 | Not in cards |
| Internal computer use safety benchmark (OpenAI) Models complete computer-use tasks while avoiding unsafe actions. | Research frontier | Sep 2026 | 2.40% | System cards only |
| LongPuzzleBench GUI agents solve 114 visual puzzle levels across six games through native interface actions. | Everyday | Sep 2026 | Not in cards | |
| MatToolBench Agents use materials science software on Windows to operate interfaces, write scripts, and query databases. The evaluator awards partial credit for completed steps. | University | Sep 2026 | 51.5% | Not in cards |
| Misaligned Behavior in Realistic Work Environments Agents do computer and browser tasks in simulated workplace apps with ambiguous instructions. Scored on the rate of unintended outcomes such as data disclosure. | Expert | Sep 2026 | 1.40% | System cards only |
| OSWorld 2.1 Agents complete 108 long-horizon workflows across 31 self-hosted websites and desktop apps, averaging 318 tool calls. Scored by binary completion and checkpoint partial credit. | University | Sep 2026 | 44.3% | In use |
| OSWorld 2.1 (offline subset) Agents complete the 82 OSWorld 2.1 tasks on the official offline list, with no internet access. Scored by checkpoint partial credit and strict pass rate. | University | Sep 2026 | 89.3% | In use |
| OSWorld-Science Computer-use agents do 146 scientific software tasks, such as molecular drawing, across several domains. Produced artifacts are checked to score each task. | Expert | Sep 2026 | 73.7% | Not in cards |
| PhoneShell iOS Agent Benchmark Agents operate apps on a physical iPhone to set timers, find books, and complete other everyday tasks. Automated checks read the resulting device state. | Everyday | Sep 2026 | 96.6% | Not in cards |
| ProgramDistill Agents inspect working web applications and restore missing features in editable copies. Replayed browser traces check whether patches recover the reference behavior. The paper evaluates partial repairs and full application reconstruction. | University | Sep 2026 | 84.3% | Not in cards |
| RecreationBench Agents explore a running reference app and rebuild it, across 250 tasks on five platforms. Programmatic tests and a VLM judge score the result. | Expert | Sep 2026 | 58.1% | In use |
| SpeedrunBench Agents try to finish goals in 10 video games as fast as possible. Score is time to goal in frames or seconds, lower is better. | Everyday | Sep 2026 | Not in cards | |
| ADeptS-Bench Desktop Agents choose safe interface actions on the Desktop test set. The published safety score combines task success and attack resistance. | Expert | Aug 2026 | | Not in cards |
| CommerceAgentBench Agents finish real e-commerce work across procurement, listings, operations, fulfillment and after-sales through CLI, browser, file and API tools. A task passes only if every verifier check on the end state succeeds. | Expert | Aug 2026 | Not in cards | |
| OSWorkerBench Desktop agents complete long-horizon office tasks across business applications with executable evaluators and optional demonstrations. | Expert | Aug 2026 | Not in cards | |
| OSWorld 2.0 (Aug 2026) Agents complete 108 long-horizon workflows across 31 self-hosted websites and desktop apps, averaging 318 tool calls. Scored by binary completion and checkpoint partial credit. | University | Aug 2026 | 77.9% | System cards only |
| Composite-Bench Agents solve generated browser tasks in planning, rule extraction, deduplication, routing, and information gathering. Success requires matching a certified optimal solution. A separate number-partitioning task measures solution quality rather than contributing to the success rate. | Expert | Jul 2026 | 74% | Not in cards |
| ChainWorld Agents complete sequences of desktop tasks while preserving the state needed for later steps. OSWorld checks each step. | Everyday | Jun 2026 | 31% | Not in cards |
| CreativeWork Agents complete productivity tasks in Notion, Canva and Figma by mixing GUI actions with MCP tool calls. | Everyday | Jun 2026 | | In use |
| Cua-Bench EDA (KiCad v1) Agents build or edit KiCad schematics through the desktop interface. The grader checks the resulting netlist and reports full passes, partial passes, and mean reward. | Expert | Jun 2026 | 24% | Not in cards |
| DeskCraft Desktop agents complete 538 Ubuntu tasks, including 152 tasks with later user clarifications or interruptions. Programmatic checks inspect final application state, saved files, and exported artifacts. | Expert | Jun 2026 | 29.8% | Not in cards |
| iOSWorld Phone agents work in a native iOS simulator across 26 purpose-built apps that share one persistent user's data. Tasks span single apps, multi-app flows and memory or personalization from the user's history. | Everyday | Jun 2026 | 51.9% | Not in cards |
| K-BrowseComp Web browsing agents answer 400 hard search problems grounded in Korean contexts. A 300-problem Verified subset was built and checked by native speakers. | Expert | Jun 2026 | 45.7% | Not in cards |
| MyPCBench Computer-use agents work in a personalized Linux desktop with 17 simulated logged-in web apps and a user's history, across 184 tasks. | Everyday | Jun 2026 | 62% | Not in cards |
| OSWorld 2.0 Agents complete 108 long-horizon workflows across 31 self-hosted websites and desktop apps, averaging 318 tool calls. Scored by binary completion and checkpoint partial credit. | University | Jun 2026 | 20.6% | In use |
| PhoneHarness Bench Agents complete Android workflows through graphical controls, device commands, and structured tools. Checks inspect files, system settings, application state, and unintended side effects. | Everyday | Jun 2026 | | Not in cards |
| SentinelBench Agents monitor changing web environments and respond when scripted events make progress possible. Scores measure completion, reaction time and resource use. | Expert | Jun 2026 | Not in cards | |
| Turing Agentic Environments Agents use business and consumer apps through graphical interfaces or tools. Turing reports average pass rate across the apps each model attempted. | University | Jun 2026 | 60.2% | Not in cards |
| WeaveBench Computer-use agents do 114 long tasks in 8 work domains that mix GUI actions with command-line and code operations in one trajectory. Results are checked against verifiable artifacts. | Expert | Jun 2026 | 41.2% | Not in cards |
| ClawBench V2 Browser agents do 130 new everyday tasks on live websites under six supported harnesses. Request interception and a more lenient LLM judge score each run. | Everyday | May 2026 | 44.6% | Not in cards |
| CutVerse Agents complete editing tasks in seven applications, including Premiere Pro, Photoshop, and DaVinci Resolve. Vision models grade progress against milestones in the recorded workflow. | University | May 2026 | 68.3% | Not in cards |
| ERP-Bench Agents complete 300 procurement and manufacturing workflows in Odoo 19. A constraint solver generates the tasks and optimal solutions, and the verifier checks final business state. | Expert | May 2026 | 43.4% | Not in cards |
| MobileGym Mobile GUI agents use 28 simulated apps in a browser-hosted phone, across 416 parameterized task templates. Deterministic judges check the app state. | Everyday | May 2026 | | Not in cards |
| OpenComputer Computer-use agents complete 1,000 desktop tasks across 33 real applications, including browsers, office tools, creative software, development environments and messaging apps. App-specific verifiers inspect the final application state, and the harness gives partial credit from the recorded trajectory. | Everyday | May 2026 | 68.3% | Not in cards |
| SaaS-Bench 106 long-horizon GUI workflows across 23 locally deployed real SaaS apps. Scored from app databases with weighted checkpoints and full-resolution rate. | University | May 2026 | 31.1% | In use |
| UniGUI-Bench Agents interpret and operate graphical interfaces. The benchmark tests 11 capabilities and includes separate system evaluations. | Everyday | May 2026 | 80.9% | Not in cards |
| WebHarbor Agents navigate local copies of 153 websites to search, sign in, book services, and change stored state. The suite contains 3,518 tasks, including 306 verified test tasks and 2,288 training tasks. The test score measures the percentage of tasks passed by scripted verifiers. | Everyday | May 2026 | 86.7% | Not in cards |
| ClawBench Browser agents do 153 everyday tasks on live websites, such as booking travel, ordering food and applying for jobs. Request interception stops real side effects, and an LLM judge scores the five-layer traces. | Everyday | Apr 2026 | 61.4% | Not in cards |
| CUA-World-Long Computer-use agents do one long task in each of 200 real applications, such as 3D Slicer, GIMP and OpenEMR, often needing 500 or more steps. Checklist verifiers with privileged state score each task. | Expert | Apr 2026 | 27.5% | Not in cards |
| GameWorld Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. | Everyday | Apr 2026 | 76.8% | System cards only |
| KnowU-Bench Agents use Android apps such as Mattermost and Taodian. They infer preferences, ask for missing details, and decide when to act or stay silent. | Everyday | Apr 2026 | 60.4% | Not in cards |
| GUI-CEval Models answer questions about Chinese app screens, locate controls, and choose actions. Agents also complete tasks on physical devices, where the grader checks interface elements. | Everyday | Mar 2026 | | Not in cards |
| AutoElicit An optimizer perturbs benign instructions to elicit unsafe behavior from computer-use agents. Reports elicitation success rate on transferred prompts. | Expert | Feb 2026 | | System cards only |
| MemGUI-Bench Agents remember details while working across Android apps. Tasks include recording audio and creating calendar events. A model checks the actions and results across repeated attempts. | Everyday | Feb 2026 | | Not in cards |
| Over-eager behavior on GUI computer use tasks (Anthropic) Computer-use agents face impossible tasks with an unwanted shortcut available. Score is the rate of unapproved workarounds instead of checking with the user. | Expert | Feb 2026 | 9.40% | System cards only |
| VenusBench-Mobile Agents use Android apps such as VLC, Markor, and Pro Expense. A model judges 90 primary tasks. The other tasks use programmatic checks. | Everyday | Feb 2026 | 24.8% | Not in cards |
| BU Bench V2 Browser agents do real web tasks with the Browser Use library. A judge scores each run against a rubric. | Everyday | Jan 2026 | 80.6 | Not in cards |
| MobileWorld Agents complete 201 tasks across 20 mobile apps, including tasks that need user clarification or MCP tools. Backend checks verify success. | Everyday | Dec 2025 | 55.3% | In use |
| WebArena-Verified Agents complete 812 audited WebArena tasks on self-hosted websites. Deterministic evaluators check the agent's answer and captured network traces. | Everyday | Dec 2025 | | In use |
| WorldTravel Agents plan trips for 150 real-world scenarios across 5 cities with more than 15 tightly coupled time and logic constraints each. A web environment supplies multimodal travel data. | Everyday | Dec 2025 | 32.7% | Fading |
| AndroidDaily Agents complete 350 daily tasks in 94 real closed-source Android apps. A process-aware reviewer model checks each trajectory against written guidelines. | Everyday | Nov 2025 | | Fading |
| OSGym Infrastructure running 1,000+ parallel OS sandboxes for computer-use agent training and evaluation. Tasks span browsing, documents, SWE and multi-app work. | Everyday | Nov 2025 | Not in cards | |
| Prompt injection robustness against adaptive attackers, browser use (Anthropic) Browser agents face indirect prompt injections in web environments from an internal adaptive attacker. Score is attack success rate. | Expert | Nov 2025 | 0% | System cards only |
| Prompt injection robustness against adaptive attackers, computer use (Anthropic) Computer-use agents face indirect prompt injections crafted by an adaptive attacker. Score is attack success rate. | Expert | Nov 2025 | 0% | System cards only |
| WebTailBench Computer-use agents complete 609 hand-verified tasks on live websites across 11 categories, such as bookings, real estate and job search. A companion set of 111 harmful tasks tests refusals. | Everyday | Nov 2025 | 57.4% | Fading |
| WebTailBench-Refusals Computer-use agents receive 111 harmful browser tasks in seven policy categories, such as illegal activity and misinformation. Scored as the share of tasks safely refused. | Everyday | Nov 2025 | | System cards only |
| OSWorld-MCP Computer-use agents complete OSWorld desktop tasks while they can also call 158 MCP tools across 7 common applications. The benchmark checks whether agents pick a tool call or GUI actions at the right time. It reports task accuracy, tool invocation rate and average completion steps. | Everyday | Oct 2025 | 56.6% | Not in cards |
| WARC-Bench Screenshot-only agents do short web subtasks, such as date pickers and scroll containers, on real sites replayed from Web Archive snapshots. A program checks the end state. | Everyday | Oct 2025 | 64.8% | Not in cards |
| Automotive-ENV Agents operate climate controls in an Android Automotive OS simulator. A validator checks the screen and vehicle signals after requests such as turning on the defroster. | Everyday | Sep 2025 | 49.3% | Not in cards |
| DeepResearch Arena Deep research agents get research tasks generated from academic seminar transcripts, which reflect expert discussion and reduce data leakage. | Expert | Sep 2025 | 3.96 | Not in cards |
| SCUBA Agents work in a Salesforce sandbox as administrators, sales representatives, or service agents. Evaluators check milestones such as creating accounts and routing support cases. | University | Sep 2025 | 53.9% | Not in cards |
| UI-TARS 15 Games Collection GUI agents play 15 browser games such as 2048, Infinity-Loop and Snake-solver from screenshots. Each game score is divided by a human score, and the mean is reported. | Everyday | Sep 2025 | | Dropped |
| Android Agent Arena (100 tasks) Agents use 20 Android apps to find information and perform actions. A vision model checks whether the agent reached the required intermediate states and completed the task. | Everyday | Aug 2025 | | Not in cards |
| Browsing prompt injections Agents answer web research questions while malicious instructions are injected in the browsing context. Scored on resisting the injections. | Expert | Aug 2025 | 99% | System cards only |
| WideSearch 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. | Everyday | Aug 2025 | | In use |
| Active data exfiltration, visual browser Agents using a visual browser meet attacks that require them to fetch sensitive information before exfiltrating it. Scored on resisting them. | Expert | Jul 2025 | 67% | System cards only |
| In-context data exfiltration, visual browser Agents using a visual browser meet attacks that try to exfiltrate data already present in the conversation context. Scored on resisting them. | Expert | Jul 2025 | 78% | System cards only |
| Irrelevant instructions, visual browser Agents using a visual browser meet irrelevant instructions or data exfiltration attempts on web pages, based on red teaming scenarios. Scored on ignoring them. | Expert | Jul 2025 | 95% | System cards only |
| MMBench-GUI GUI agents are tested at four levels on Windows, macOS, Linux, iOS, Android and the web: content understanding, element grounding, task automation and task collaboration. | Everyday | Jul 2025 | | System cards only |
| OSWorld-Verified 369 open-ended tasks in real desktop apps. Agent acts through screenshots, mouse and keyboard. Execution-based checkers score success. | Everyday | Jul 2025 | 86% | In use |
| DeepShop Web agents answer complex shopping queries on live sites with product attributes, search filters and sorting preferences. An LLM judge checks whether results meet each requirement. | Everyday | Jun 2025 | | Fading |
| Mind2Web 2 Agents browse the live web for 130 long tasks and return answers with citations. Task-specific judge agents built from rubric trees grade correctness and attribution. | Expert | Jun 2025 | 28% | Not in cards |
| ScreenSuite Models use screenshots to locate controls, select actions and complete computer tasks. The suite measures perception, single actions and complete interactions. | Everyday | Jun 2025 | | Not in cards |
| AndroidWorld-Knowledge Agents attempt 48 AndroidWorld tasks across five apps. Each app provides guidance for its simplest task, and harder tasks test whether the agent can reuse it. | Everyday | May 2025 | Not in cards | |
| Deep Research Bench (FutureSearch) Web research agents work on multi-step research tasks against RetroSearch, a frozen copy of 10,000 to 100,000 web pages per task. Answers are compared with answers worked out by people. | Expert | May 2025 | 0.55 | Not in cards |
| RiOSWorld Computer-use agents work in an Ubuntu virtual machine through risky scenarios such as phishing emails and pop-ups. Scores track whether agents notice and avoid the risk. | Everyday | May 2025 | | Not in cards |
| ScienceBoard Computer-use agents do scientific workflows in desktop software such as ChimeraX, Celestia, GRASS GIS and Lean 4. The apps are modified to expose state for checking. | Expert | May 2025 | 68% | Not in cards |
| VideoGameBench Lite Agents play video games while the environment pauses during inference. The headline is the official Overall Score. | Expert | May 2025 | 1.60% | Not in cards |
| WebBench (Halluminate) Browser agents complete 2,454 tasks on 452 live websites from the global top 1,000 by traffic. Tasks either read information or write data: creating, updating and deleting records, logging in and downloading files. The board compares browser agent products. | Everyday | May 2025 | 76.5% | Not in cards |
| Online-Mind2Web Web agents do 300 tasks on 136 live websites. An LLM judge reads the agent's trajectory and decides whether the task succeeded. | Everyday | Apr 2025 | | Fading |
| REAL Agents complete 112 tasks on replicas of 11 websites. State checks grade actions, while a model judge checks retrieved answers against references. | Everyday | Apr 2025 | 41.1% | Not in cards |
| AI Agent Privacy Web agents complete tasks while exposed to private information. The benchmark measures privacy leakage and unsafe disclosure during autonomous operation. | Expert | Mar 2025 | Not in cards | |
| Malicious computer use (Anthropic) Computer-use agents receive harmful tasks such as surveillance or fraud in GUI environments. Score is the refusal rate. | Everyday | Feb 2025 | 99.4% | System cards only |
| WorldGUI Desktop GUI agents do 611 tasks that start from varied initial states, such as the target app not being open. Tasks come with instructional videos and project files. | Everyday | Feb 2025 | 45.8% | Not in cards |
| Agentic harmful task refusals Browser agents receive harmful tasks such as illicit purchases, prohibited financial activity and searches for sensitive personal data. Scored as refusal rate per category. | Everyday | Jan 2025 | 97% | System cards only |
| Android Agent Arena (201 tasks) Agents find live information and perform actions in Android apps such as YouTube Music and Booking.com. Task functions check app state and grade answers. | Everyday | Jan 2025 | | Not in cards |
| Autonomy-10 Agents complete browser, desktop and tool-use tasks across nine domains that measure skill, flexibility and intelligence. | Expert | Jan 2025 | Not in cards | |
| Mobile-Eval-E Agents complete 25 smartphone tasks across 15 apps. Human reviewers score the share of task requirements each agent meets. | Everyday | Jan 2025 | Not in cards | |
| Operator prompt injection scenarios Browser agents meet 31 automatically checkable prompt injection scenarios that fooled earlier model versions. Scored as susceptibility rate. | Expert | Jan 2025 | 23% | System cards only |
| Proactive refusals of high-risk tasks Browser agents receive high-risk tasks such as banking transfers and high-stakes decisions that policy says they must refuse. Scored on refusal recall. | Everyday | Jan 2025 | 94% | System cards only |
| User Confirmations During Computer Use Agents doing computer use tasks reach financial, high-stakes communication and other actions. Scored on asking the user for confirmation when policy requires it. | Everyday | Jan 2025 | 92% | System cards only |
| TheAgentCompany Agents do work tasks in a simulated software company, using a browser, code, and chat with simulated coworkers. Scored by full and partial checkpoint completion. | University | Dec 2024 | Dropped | |
| AndroidLab Mobile agents complete 138 tasks across nine apps on predefined Android virtual devices, in text-only and multimodal modes. Checks on the final device state score each task. | University | Oct 2024 | 31.2% | Not in cards |
| BrowserART Browser agents receive 100 harmful requests on synthetic and real websites. The test measures how often the agent attempts the harmful behavior that its backbone model would refuse in chat. | Expert | Oct 2024 | 11% | Not in cards |
| SPA-Bench Agents use Android apps such as Airbnb, Spotify, and WeChat. The grader matches screenshots against task requirements and uses a vision model to resolve uncertain cases. | Everyday | Oct 2024 | 1.35% | Not in cards |
| Biotool and Wet Lab Actions Agents perform wet lab assistant tasks such as finding a genome file or ordering DNA through a computer or browser. Scored as pass@10 per task. | Expert | Sep 2024 | System cards only | |
| Windows Agent Arena Agents complete more than 150 tasks in a real Windows OS with its apps, tools and browsers. Scripts adapted from OSWorld check the final state. | Everyday | Sep 2024 | | Dropped |
| VisualAgentBench Large multimodal models act as agents in embodied, GUI and visual design environments, including mobile, WebArena-based web and Minecraft tasks. | Everyday | Aug 2024 | 36.2% | Not in cards |
| AssistantBench Web agents do 214 realistic, time-consuming tasks on the open web, such as monitoring real-estate markets. Answers are checked automatically against gold answers. | Everyday | Jul 2024 | Not in cards | |
| CRAB Multimodal agents do tasks that span an Ubuntu desktop and an Android phone at once. Graph-based evaluators give credit for intermediate steps. | Everyday | Jul 2024 | 38% | Not in cards |
| Spider2-V Multimodal agents do data science and engineering workflows in real enterprise apps such as BigQuery, dbt and Airbyte, on a desktop computer. Execution-based checks score each task. | Expert | Jul 2024 | 14% | Not in cards |
| WorkArena++ Web agents complete 682 compositional knowledge-work tasks in ServiceNow, which need planning, reasoning and memory across many steps. | Expert | Jul 2024 | 1.90% | Not in cards |
| GUI Odyssey Mobile GUI agents carry out tasks that cross several apps, such as sharing a post from one social app to another. The dataset has 8,334 episodes averaging 15.3 steps, over 212 apps, 1,357 app combinations and six devices. Agents are scored by how often each predicted action matches the recorded step. | Everyday | Jun 2024 | 55% | Not in cards |
| Mobile-Eval-v2 Agents follow basic and advanced instructions on real phones. The authors give 44 instructions in English on Android and 44 in Chinese on HarmonyOS. | Everyday | Jun 2024 | Not in cards | |
| MobileAgentBench Agents complete 100 tasks across ten Android apps. The grader checks app state and accessibility events, so agents can take different paths. | Everyday | Jun 2024 | 8.00% | Not in cards |
| WebCanvas Web agents do tasks on live websites, scored on key intermediate states rather than one fixed action path, so the test survives site changes. | Everyday | Jun 2024 | 28.8% | Not in cards |
| AndroidWorld Agents complete 116 tasks across 20 real Android apps in an emulator. Each task checks the device's system state to judge success. | Everyday | May 2024 | 100% | In use |
| MobileMiniWoB++ Agents complete MiniWoB++ web tasks inside the Android emulator of AndroidWorld. Scored by task success rate. | Everyday | May 2024 | | Dropped |
| B-MoCA Agents use Android apps such as Instagram on phones with different languages, font sizes, and icon positions. Task-specific checks decide whether each attempt succeeded. | Everyday | Apr 2024 | 47.7% | Not in cards |
| LlamaTouch Agents carry out requests in Android apps. The evaluator checks whether each run visited the required app or system states. | Everyday | Apr 2024 | Not in cards | |
| OSWorld Agents complete 369 real computer tasks in Ubuntu desktop apps through screenshots and mouse and keyboard actions. Execution-based scripts check the final state. | Everyday | Apr 2024 | | Fading |
| WorkArena Web agents do routine knowledge-work tasks in a live ServiceNow instance, such as filling forms and filtering lists. WorkArena++ adds 682 compositional tasks at levels 2 and 3. | University | Mar 2024 | Not in cards | |
| AndroidArena Agents use apps such as Gmail and Google Maps, sometimes switching apps or following extra constraints. GPT-4 judges success from the recorded actions and observations. | Everyday | Feb 2024 | | Not in cards |
| OmniACT Models generate executable scripts from a screenshot and a user request. The evaluation compares the actions needed to complete desktop and web tasks. | University | Feb 2024 | Not in cards | |
| WebLINX Agents follow a user's multi-turn chat instructions to navigate real websites. 100,000 interactions from 2,300 expert demonstrations give turn-level metrics. | Everyday | Feb 2024 | 10.4 | Not in cards |
| Mobile-Eval Agents follow 33 instructions on Android. Some requests specify the app. Others describe a goal and leave the choice of app to the agent. | Everyday | Jan 2024 | Not in cards | |
| VisualWebArena Multimodal web agents complete 910 visually grounded tasks on self-hosted classifieds, shopping and Reddit-style sites. Execution-based checks score each task. | University | Jan 2024 | 16.4% | Not in cards |
| WebVoyager Agents complete user tasks on 15 popular real websites by browsing from screenshots and page text. A GPT-4V judge grades whether each task succeeded. | Everyday | Jan 2024 | | Fading |
| AppAgent Test Set Agents follow 45 instructions across nine Android apps, including scheduling email and changing music playback. Tasks time out after ten steps. | Everyday | Dec 2023 | Not in cards | |
| DroidTask Agents select actions for 158 Android tasks across 13 apps. The original completion score requires every action to match the recorded sequence. | Everyday | Aug 2023 | Not in cards | |
| WebArena Agents complete 812 realistic tasks on self-hosted shopping, forum, GitLab, CMS and map websites. Functional graders check the outcome. | Everyday | Jul 2023 | 68% | Fading |
| WebShop Agents search and click through a simulated shopping site with 1.18 million real products to buy an item that matches a text instruction. Reward reflects how well the purchase matches. | Everyday | Jul 2022 | Not in cards | |
| MiniWoB++ Agents operate over 100 small synthetic web pages, such as forms, menus and date pickers, from a natural-language instruction. Each task returns a success reward. | Everyday | Feb 2018 | Not in cards | |
| AgentCLUE-Mobile Mobile assistants complete tasks on a phone. | Everyday | Not in cards | ||
| AgentCLUE-Mobile (offline) Agents select actions in an offline mobile interface. | Everyday | 88.4 | Not in cards | |
| BrowserInject v1 Browser agents do benign tasks on 30 retail, health, support and travel sites seeded with prompt injections. Reports attack success rate. | Expert | | System cards only | |
| Child Safety Action Testing Computer use agents get requests for actions that facilitate child exploitation, endangerment or illegal access. Reports the refusal score. | Everyday | | System cards only | |
| Computer use manual red teaming Google's generative AI red team probes computer use agents in sandboxes for tool misuse, skipped confirmations and prohibited actions. | Everyday | | System cards only | |
| Computer use Prohibited Action Evaluation Computer use agents get single-turn adversarial requests for actions that break Google policy. Reports the safe action rate. | Everyday | | System cards only | |
| Computer use prompt injection evaluation (Anthropic) Agents face about 600 prompt injection scenarios in coding platforms, browsers and email workflows. Score is the attack prevention rate. | Expert | 78% | System cards only | |
| Computer use prompt injection robustness Computer use agents meet injected text and live DOM attacks across 27 web scenarios. A multimodal judge records breaches. Reports survival after 100 attempts. | Expert | | System cards only | |
| OdysseysMega Agents complete tasks in a web browser. A judge checks each rubric criterion against the recorded actions and screenshots. | Expert | 41% | Not in cards | |
| Personal-device assistant prompt injection eval OS assistant agents in 44 stateful scenarios ingest untrusted content that tries to trigger unauthorized actions. Reports 5-step adaptive survival and task utility. | Expert | | System cards only | |
| SheetBench Agents solve finance and accounting spreadsheet tasks through a browser in a fresh Google Sheet. The score is the share of expected answer cells that match exactly. | Expert | 64.7% | Not in cards | |
| WebsiteBench Agents explore reference websites and rebuild their appearance, behavior, and persistent state. Hidden browser tests require functional correctness and screenshot similarity, then report the fraction passed. | Expert | 26.2% | Not in cards |