MemGUI-Bench
Agents remember details while working across Android apps. Tasks include recording audio and creating calendar events. A model checks the actions and results across repeated attempts.
The leaderboard scores 29 systems on 128 tasks across 26 apps. Ant Group independently reports first-attempt results for two UI-Venus-2 models.
- Made by
- Zhejiang University, Nankai University, Chinese University of Hong Kong, Shanghai Jiao Tong University, and vivo AI Lab
- Released
- February 2026
- Kind
- Phones
- Level
- Everyday
- Scored
- Graded by a model
- Answers
- Agent task
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Agentic
- Environment
- Phone apps
- Items from
- Written by experts
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Computer use, Agents
- Strongest model tested
- GPT-5.5 #21 on the capability index
- Last model added
- August 2026
- Scores refreshed
- November 20th, 2026
Qwen-UI-Agent tops the leaderboard with 77.3% Gemini 3.6 Flash leads with 73.4% among models released by August 2026 Gemini 3.6 Flash leads with 73.4% in a standard harness
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran on its own with several settings, we keep its best. Agents other companies built around a model get a pale shade of that model's lab color. Gray marks blends and entries not in our model list, such as fine-tunes and older models. None of these count toward leaders. The best model we track is Gemini 3.6 Flash, at 73.4%. Agents built around a model, such as Agent-S2 + Gemini 2.5 Pro, are their own entries. We don't count them as the model's score. This view shows only models we track, each at its best. This view keeps only runs in a standard harness, a harness no model’s maker built. A maker’s own agent, such as Claude Code with a Claude model, is left out.
The newest model with a score here came out in August 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 77.3 Qwen-UI-Agent
- 73.4
Gemini 3.6 Flash
- 71.9
Qwen 3.8 Max
- 70.3 UI-Venus-2-27B
- 61.7 UI-Venus-2-9B
- 57.0 GPT-5.5
- 43.8
Gemini 3.1 Pro
- 42.2 GPT-5.6 Luna
- 39.1 Kimi K2.6
- 37.5 MemGUI-Agent-235B
- 32.8
Gemini 2.5 Pro
- 31.2
Seed2.0 Pro
- 27.3 Agent-S2 + Gemini 2.5 Pro
- 24.2
Qwen3-VL-235B-A22B
- 23.4
Qwen3-VL-235B-A22B-Instruct
- 23.4 MemGUI-8B-SFT
- 20.3
Qwen3-VL-32B-Instruct
- 11.7 GUI-Owl-1.5-8B-Instruct
- 10.9 GUI-Owl-1.5-32B-Instruct
- 9.40
Qwen3-VL-8B-Instruct
- 6.20
GUI-Owl-7B
- 5.50 Mobile-Agent-E + Gemini 2.5 Pro
- 5.50 UI-Venus-7B
- 3.10 AppAgent + Gemini 2.5 Pro
- 3.10 Mobile-Agent-V2 + Gemini 2.5 Pro
- 3.10
UI-TARS-1.5
- 2.30 SeeAct + Gemini 2.5 Pro
- 0.0 CogAgent
- 1 Qwen-UI-Agent 77.3%
- 2
Gemini 3.6 Flash 73.4%
- 3
Qwen 3.8 Max 71.9%
- 4 UI-Venus-2-27B 70.3%
- 5 UI-Venus-2-9B 61.7%
- 6 GPT-5.5 57%
- 7
Gemini 3.1 Pro 43.8%
- 8 GPT-5.6 Luna 42.2%
- 9 Kimi K2.6 39.1%
- 10 MemGUI-Agent-235B 37.5%
- 11
Gemini 2.5 Pro Mar 2025 32.8%
- 12
Seed2.0 Pro 31.2%
- 13 Agent-S2 + Gemini 2.5 Pro Mar 2025 27.3%
- 14
Qwen3-VL-235B-A22B thinking 24.2%
- 15
Qwen3-VL-235B-A22B-Instruct 23.4%
- 16 MemGUI-8B-SFT 23.4%
- 17
Qwen3-VL-32B-Instruct 20.3%
- 18 GUI-Owl-1.5-8B-Instruct 11.7%
- 19 GUI-Owl-1.5-32B-Instruct 10.9%
- 20
Qwen3-VL-8B-Instruct 9.40%
- 21
GUI-Owl-7B 6.20%
- 22 Mobile-Agent-E + Gemini 2.5 Pro Mar 2025 5.50%
- 23 UI-Venus-7B 5.50%
- 24 AppAgent + Gemini 2.5 Pro Mar 2025 3.10%
- 25 Mobile-Agent-V2 + Gemini 2.5 Pro Mar 2025 3.10%
- 26
UI-TARS-1.5 3.10%
- 27 SeeAct + Gemini 2.5 Pro Mar 2025 2.30%
- 28 CogAgent 0%
Source: Official MemGUI-Bench leaderboard
- 1
Gemini 3.6 Flash 73.4%
- 2
Qwen 3.8 Max 71.9%
- 3 GPT-5.5 57%
- 4
Gemini 3.1 Pro 43.8%
- 5 GPT-5.6 Luna 42.2%
- 6 Kimi K2.6 39.1%
- 7
Gemini 2.5 Pro Mar 2025 32.8%
- 8
Seed2.0 Pro 31.2%
- 9
Qwen3-VL-235B-A22B thinking 24.2%
- 10
Qwen3-VL-235B-A22B-Instruct 23.4%
- 11
Qwen3-VL-32B-Instruct 20.3%
- 12
Qwen3-VL-8B-Instruct 9.40%
- 13
GUI-Owl-7B 6.20%
- 14
UI-TARS-1.5 3.10%
- 15 CogAgent 0%
Source: Official MemGUI-Bench leaderboard
- 1
Gemini 3.6 Flash 73.4%
- 2
Qwen 3.8 Max 71.9%
- 3 GPT-5.5 57%
- 4
Gemini 3.1 Pro 43.8%
- 5 GPT-5.6 Luna 42.2%
- 6 Kimi K2.6 39.1%
- 7
Seed2.0 Pro 31.2%
- 8
Qwen3-VL-235B-A22B thinking 24.2%
- 9
Qwen3-VL-235B-A22B-Instruct 23.4%
- 10
Qwen3-VL-32B-Instruct 20.3%
- 11
Qwen3-VL-8B-Instruct 9.40%
- 12
GUI-Owl-7B 6.20%
- 13
UI-TARS-1.5 3.10%
- 14 CogAgent 0%
Source: Official MemGUI-Bench leaderboard
| # | Entry | Score | Setting | Agent by | Released | Run on | Link | Source |
|---|---|---|---|---|---|---|---|---|
| 1 | Qwen-UI-Agent | 77.3% | Alibaba | Jul 2026 | arxiv.org | Official leaderboard | ||
| 2 | | 73.4% | Jul 2026 | Jul 2026 | blog.google | Official leaderboard | ||
| 3 | | 71.9% | Alibaba | Aug 2026 | Aug 2026 | qwen.ai | Official leaderboard | |
| 4 | UI-Venus-2-27B | 70.3% | InclusionAI | Sep 2026 | github.com | Official leaderboard | ||
| 5 | UI-Venus-2-9B | 61.7% | InclusionAI | Sep 2026 | github.com | Official leaderboard | ||
| 6 | GPT-5.5 | 57% | OpenAI | Apr 2026 | Apr 2026 | openai.com | Official leaderboard | |
| 7 | | 43.8% | Feb 2026 | Feb 2026 | deepmind.google | Official leaderboard | ||
| 8 | GPT-5.6 Luna | 42.2% | OpenAI | Jul 2026 | Jul 2026 | openai.com | Official leaderboard | |
| 9 | Kimi K2.6 | 39.1% | Alibaba | Apr 2026 | Apr 2026 | kimi-k2.org | Official leaderboard | |
| 10 | MemGUI-Agent-235B | 37.5% | Kuaishou | Jun 2026 | arxiv.org | Official leaderboard | ||
| 11 | | 32.8% | M3A | Google DeepMind | Mar 2025 | May 2024 | arxiv.org | Official leaderboard |
| 12 | | 31.2% | ByteDance Seed | Feb 2026 | Feb 2026 | seed.bytedance.com | Official leaderboard | |
| 13 | Agent-S2 + Gemini 2.5 Pro Mar 2025 | 27.3% | UCSB | Mar 2025 | Apr 2025 | arxiv.org | Official leaderboard | |
| 14 | | 24.2% | thinking | Alibaba | Sep 2025 | Feb 2026 | arxiv.org | Official leaderboard |
| 15 | | 23.4% | Alibaba | Sep 2025 | Feb 2026 | arxiv.org | Official leaderboard | |
| 16 | MemGUI-8B-SFT | 23.4% | Kuaishou | Jun 2026 | arxiv.org | Official leaderboard | ||
| 17 | | 20.3% | Alibaba | Oct 2025 | Feb 2026 | arxiv.org | Official leaderboard | |
| 18 | GUI-Owl-1.5-8B-Instruct | 11.7% | Alibaba | Feb 2026 | arxiv.org | Official leaderboard | ||
| 19 | GUI-Owl-1.5-32B-Instruct | 10.9% | Alibaba | Feb 2026 | arxiv.org | Official leaderboard | ||
| 20 | | 9.40% | Alibaba | Oct 2025 | Nov 2025 | arxiv.org | Official leaderboard | |
| 21 | | 6.20% | Alibaba | Aug 2025 | Aug 2025 | Official leaderboard | ||
| 22 | Mobile-Agent-E + Gemini 2.5 Pro Mar 2025 | 5.50% | UIUC & Alibaba | Mar 2025 | Jan 2025 | arxiv.org | Official leaderboard | |
| 23 | UI-Venus-7B | 5.50% | InclusionAI | Aug 2025 | arxiv.org | Official leaderboard | ||
| 24 | AppAgent + Gemini 2.5 Pro Mar 2025 | 3.10% | Tencent | Mar 2025 | Dec 2023 | arxiv.org | Official leaderboard | |
| 25 | Mobile-Agent-V2 + Gemini 2.5 Pro Mar 2025 | 3.10% | Alibaba | Mar 2025 | Jun 2024 | arxiv.org | Official leaderboard | |
| 26 | | 3.10% | ByteDance Seed | Apr 2025 | Jan 2025 | arxiv.org | Official leaderboard | |
| 27 | SeeAct + Gemini 2.5 Pro Mar 2025 | 2.30% | OSU NLP | Mar 2025 | Jan 2024 | arxiv.org | Official leaderboard | |
| 28 | CogAgent | 0% | Tsinghua & Zhipu | Dec 2023 | Dec 2023 | arxiv.org | Official leaderboard |
| # | Model | Score | Setting | Agent by | Released | Run on | Link | Source |
|---|---|---|---|---|---|---|---|---|
| 1 | | 73.4% | Jul 2026 | Jul 2026 | blog.google | Official leaderboard | ||
| 2 | | 71.9% | Alibaba | Aug 2026 | Aug 2026 | qwen.ai | Official leaderboard | |
| 3 | GPT-5.5 | 57% | OpenAI | Apr 2026 | Apr 2026 | openai.com | Official leaderboard | |
| 4 | | 43.8% | Feb 2026 | Feb 2026 | deepmind.google | Official leaderboard | ||
| 5 | GPT-5.6 Luna | 42.2% | OpenAI | Jul 2026 | Jul 2026 | openai.com | Official leaderboard | |
| 6 | Kimi K2.6 | 39.1% | Alibaba | Apr 2026 | Apr 2026 | kimi-k2.org | Official leaderboard | |
| 7 | | 32.8% | M3A | Google DeepMind | Mar 2025 | May 2024 | arxiv.org | Official leaderboard |
| 8 | | 31.2% | ByteDance Seed | Feb 2026 | Feb 2026 | seed.bytedance.com | Official leaderboard | |
| 9 | | 24.2% | thinking | Alibaba | Sep 2025 | Feb 2026 | arxiv.org | Official leaderboard |
| 10 | | 23.4% | Alibaba | Sep 2025 | Feb 2026 | arxiv.org | Official leaderboard | |
| 11 | | 20.3% | Alibaba | Oct 2025 | Feb 2026 | arxiv.org | Official leaderboard | |
| 12 | | 9.40% | Alibaba | Oct 2025 | Nov 2025 | arxiv.org | Official leaderboard | |
| 13 | | 6.20% | Alibaba | Aug 2025 | Aug 2025 | Official leaderboard | ||
| 14 | | 3.10% | ByteDance Seed | Apr 2025 | Jan 2025 | arxiv.org | Official leaderboard | |
| 15 | CogAgent | 0% | Tsinghua & Zhipu | Dec 2023 | Dec 2023 | arxiv.org | Official leaderboard |
| # | Model | Score | Setting | Agent by | Released | Run on | Link | Source |
|---|---|---|---|---|---|---|---|---|
| 1 | | 73.4% | Jul 2026 | Jul 2026 | blog.google | Official leaderboard | ||
| 2 | | 71.9% | Alibaba | Aug 2026 | Aug 2026 | qwen.ai | Official leaderboard | |
| 3 | GPT-5.5 | 57% | OpenAI | Apr 2026 | Apr 2026 | openai.com | Official leaderboard | |
| 4 | | 43.8% | Feb 2026 | Feb 2026 | deepmind.google | Official leaderboard | ||
| 5 | GPT-5.6 Luna | 42.2% | OpenAI | Jul 2026 | Jul 2026 | openai.com | Official leaderboard | |
| 6 | Kimi K2.6 | 39.1% | Alibaba | Apr 2026 | Apr 2026 | kimi-k2.org | Official leaderboard | |
| 7 | | 31.2% | ByteDance Seed | Feb 2026 | Feb 2026 | seed.bytedance.com | Official leaderboard | |
| 8 | | 24.2% | thinking | Alibaba | Sep 2025 | Feb 2026 | arxiv.org | Official leaderboard |
| 9 | | 23.4% | Alibaba | Sep 2025 | Feb 2026 | arxiv.org | Official leaderboard | |
| 10 | | 20.3% | Alibaba | Oct 2025 | Feb 2026 | arxiv.org | Official leaderboard | |
| 11 | | 9.40% | Alibaba | Oct 2025 | Nov 2025 | arxiv.org | Official leaderboard | |
| 12 | | 6.20% | Alibaba | Aug 2025 | Aug 2025 | Official leaderboard | ||
| 13 | | 3.10% | ByteDance Seed | Apr 2025 | Jan 2025 | arxiv.org | Official leaderboard | |
| 14 | CogAgent | 0% | Tsinghua & Zhipu | Dec 2023 | Dec 2023 | arxiv.org | Official leaderboard |
Rankings by part
We read 7 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as easy, medium and hard. Gemini 3.6 Flash leads 4 of them, and Qwen 3.8 Max leads the other 3. Arrows show how far a model moved from its overall rank among the models we track.
easy
15 models we track have a score here.
- 1
Qwen 3.8 Max ↑1 68.8%
- 2
Gemini 3.6 Flash ↓1 66.7%
- 3 GPT-5.5 52.1%
- 4
Gemini 3.1 Pro 47.9%
- 5
Gemini 2.5 Pro Mar 2025 ↑2 39.6%
- 6 Kimi K2.6 39.6%
- 7
Qwen3-VL-32B-Instruct ↑4 35.4%
- 8
Seed2.0 Pro 35.4%
- 9 GPT-5.6 Luna ↓4 35.4%
- 10
Qwen3-VL-235B-A22B-Instruct 27.1%
medium
15 models we track have a score here.
- 1
Gemini 3.6 Flash 85.7%
- 2
Qwen 3.8 Max 83.3%
- 3 GPT-5.5 64.3%
- 4 GPT-5.6 Luna ↑1 54.8%
- 5
Gemini 3.1 Pro ↓1 47.6%
- 6 Kimi K2.6 47.6%
- 7
Gemini 2.5 Pro Mar 2025 35.7%
- 8
Qwen3-VL-235B-A22B ↑1 28.6%
- 9
Seed2.0 Pro ↓1 28.6%
- 10
Qwen3-VL-235B-A22B-Instruct 21.4%
hard
15 models we track have a score here.
- 1
Gemini 3.6 Flash 68.4%
- 2
Qwen 3.8 Max 63.2%
- 3 GPT-5.5 55.3%
- 4 GPT-5.6 Luna ↑1 36.8%
- 5
Gemini 3.1 Pro ↓1 34.2%
- 6
Seed2.0 Pro ↑2 28.9%
- 7 Kimi K2.6 ↓1 28.9%
- 8
Qwen3-VL-235B-A22B ↑1 26.3%
- 9
Gemini 2.5 Pro Mar 2025 ↓2 21.1%
- 10
Qwen3-VL-235B-A22B-Instruct 21.1%
app1
15 models we track have a score here.
- 1
Qwen 3.8 Max ↑1 85.7%
- 2
Gemini 3.1 Pro ↑2 71.4%
- 3 GPT-5.5 71.4%
- 4
Gemini 3.6 Flash ↓3 67.9%
- 5 GPT-5.6 Luna 53.6%
- 6
Seed2.0 Pro ↑2 50%
- 7
Gemini 2.5 Pro Mar 2025 46.4%
- 8
Qwen3-VL-32B-Instruct ↑3 46.4%
- 9 Kimi K2.6 ↓3 39.3%
- 10
Qwen3-VL-235B-A22B-Instruct 35.7%
app2
15 models we track have a score here.
- 1
Gemini 3.6 Flash 78.6%
- 2
Qwen 3.8 Max 73.2%
- 3 GPT-5.5 50%
- 4 GPT-5.6 Luna ↑1 42.9%
- 5 Kimi K2.6 ↑1 41.1%
- 6
Gemini 3.1 Pro ↓2 32.1%
- 7
Gemini 2.5 Pro Mar 2025 28.6%
- 8
Qwen3-VL-235B-A22B ↑1 23.2%
- 9
Seed2.0 Pro ↓1 23.2%
- 10
Qwen3-VL-235B-A22B-Instruct 21.4%
app3
15 models we track have a score here.
- 1
Gemini 3.6 Flash 76.5%
- 2
Qwen 3.8 Max 61.8%
- 3 GPT-5.5 58.8%
- 4
Gemini 3.1 Pro 50%
- 5 Kimi K2.6 ↑1 38.2%
- 6 GPT-5.6 Luna ↓1 38.2%
- 7
Seed2.0 Pro ↑1 32.4%
- 8
Gemini 2.5 Pro Mar 2025 ↓1 29.4%
- 9
Qwen3-VL-235B-A22B 23.5%
- 10
Qwen3-VL-235B-A22B-Instruct 20.6%
app4
15 models we track have a score here.
- 1
Qwen 3.8 Max ↑1 60%
- 2 GPT-5.5 ↑1 50%
- 3
Gemini 3.6 Flash ↓2 50%
- 4
Gemini 2.5 Pro Mar 2025 ↑3 30%
- 5 Kimi K2.6 ↑1 30%
- 6
Seed2.0 Pro ↑2 20%
- 7 GPT-5.6 Luna ↓2 20%
- 8
Qwen3-VL-235B-A22B ↑1 10%
- 9
Qwen3-VL-235B-A22B-Instruct ↑1 10%
- 10
Qwen3-VL-32B-Instruct ↑1 10%
4 models have held number one on the official leaderboard since its release in February 2026
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official MemGUI-Bench leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official MemGUI-Bench leaderboard
Source: Official MemGUI-Bench leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Computer and browser use
- BrowseComp In use Hard-to-find facts that need persistent web browsing. Short verifiable answers graded for exact correctness. Claude Fable 5 93.3%
- ScreenSpot-Pro In use GUI grounding on high-resolution professional-software screenshots. The model must locate the target UI element. GPT-6 Astra 92.7%
- AndroidWorld In use Agents complete 116 tasks across 20 real Android apps in an emulator. Each task checks the device's system state to judge success. GPT-6 Astra 100%
- OSWorld 2.1 In use Agents complete 108 long-horizon workflows across 31 self-hosted websites and desktop apps, averaging 318 tool calls. Scored by binary completion and checkpoint partial credit. Claude Opus 5 44.3%
- WebArena-Verified In use Agents complete 812 audited WebArena tasks on self-hosted websites. Deterministic evaluators check the agent's answer and captured network traces.
Muse Spark 1.1 69%
- OSWorld 2.1 (offline subset) In use Agents complete the 82 OSWorld 2.1 tasks on the official offline list, with no internet access. Scored by checkpoint partial credit and strict pass rate. Claude Fable 5.2 89.3%
Data audit
All 10 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (28 entries).
- Pass Models scored 15 models we track, plus 13 other entries on its leaderboard.
- Pass Newest score The newest is from August 2026, for Qwen 3.8 Max.
- Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5.5 (#21).
- Pass Headroom The best is 73.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on memgui-bench.github.io: 21 rows for models we track and 8 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.