Benchmark Almanac

MemGUI-Bench

Qwen-UI-Agent 77.3% tops its leaderboard Gemini 3.6 Flash 73.4% leads the models we track Gemini 3.6 Flash 73.4% leads in a standard harness
Not in model cards

Agents remember details while working across Android apps. Tasks include recording audio and creating calendar events. A model checks the actions and results across repeated attempts.

The leaderboard scores 29 systems on 128 tasks across 26 apps. Ant Group independently reports first-attempt results for two UI-Venus-2 models.

Released
February 2026
Kind
Phones
Level
Everyday
Scored
Graded by a model
Answers
Agent task
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Screens → Actions
Interaction
Agentic
Environment
Phone apps
Items from
Written by experts
Refresh
One fixed set
Run by
Anyone
In views
Computer use, Agents
Strongest model tested
GPT-5.5 #21 on the capability index
Last model added
August 2026
Scores refreshed
November 20th, 2026
28
Entries on its own leaderboard, 15 models we track

Qwen-UI-Agent tops the leaderboard with 77.3% Gemini 3.6 Flash leads with 73.4% among models released by August 2026 Gemini 3.6 Flash leads with 73.4% in a standard harness

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran on its own with several settings, we keep its best. Agents other companies built around a model get a pale shade of that model's lab color. Gray marks blends and entries not in our model list, such as fine-tunes and older models. None of these count toward leaders. The best model we track is Gemini 3.6 Flash, at 73.4%. Agents built around a model, such as Agent-S2 + Gemini 2.5 Pro, are their own entries. We don't count them as the model's score. This view shows only models we track, each at its best. This view keeps only runs in a standard harness, a harness no model’s maker built. A maker’s own agent, such as Claude Code with a Claude model, is left out.

The newest model with a score here came out in August 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Qwen-UI-Agent 77.3%
  2. 2 Gemini 3.6 Flash 73.4%
  3. 3 Qwen 3.8 Max 71.9%
  4. 4 UI-Venus-2-27B 70.3%
  5. 5 UI-Venus-2-9B 61.7%
  6. 6 GPT-5.5 57%
  7. 7 Gemini 3.1 Pro 43.8%
  8. 8 GPT-5.6 Luna 42.2%
  9. 9 Kimi K2.6 39.1%
  10. 10 MemGUI-Agent-235B 37.5%
  11. 11 Gemini 2.5 Pro Mar 2025 32.8%
  12. 12 Seed2.0 Pro 31.2%
  13. 13 Agent-S2 + Gemini 2.5 Pro Mar 2025 27.3%
  14. 14 Qwen3-VL-235B-A22B thinking 24.2%
  15. 15 Qwen3-VL-235B-A22B-Instruct 23.4%
  16. 16 MemGUI-8B-SFT 23.4%
  17. 17 Qwen3-VL-32B-Instruct 20.3%
  18. 18 GUI-Owl-1.5-8B-Instruct 11.7%
  19. 19 GUI-Owl-1.5-32B-Instruct 10.9%
  20. 20 Qwen3-VL-8B-Instruct 9.40%
  21. 21 GUI-Owl-7B 6.20%
  22. 22 Mobile-Agent-E + Gemini 2.5 Pro Mar 2025 5.50%
  23. 23 UI-Venus-7B 5.50%
  24. 24 AppAgent + Gemini 2.5 Pro Mar 2025 3.10%
  25. 25 Mobile-Agent-V2 + Gemini 2.5 Pro Mar 2025 3.10%
  26. 26 UI-TARS-1.5 3.10%
  27. 27 SeeAct + Gemini 2.5 Pro Mar 2025 2.30%
  28. 28 CogAgent 0%

Source: Official MemGUI-Bench leaderboard

#EntryScoreSettingAgent byReleasedRun onLinkSource
1 Qwen-UI-Agent 77.3% Alibaba Jul 2026 arxiv.org Official leaderboard
2 Gemini 3.6 Flash 73.4% Google Jul 2026 Jul 2026 blog.google Official leaderboard
3 Qwen 3.8 Max 71.9% Alibaba Aug 2026 Aug 2026 qwen.ai Official leaderboard
4 UI-Venus-2-27B 70.3% InclusionAI Sep 2026 github.com Official leaderboard
5 UI-Venus-2-9B 61.7% InclusionAI Sep 2026 github.com Official leaderboard
6 GPT-5.5 57% OpenAI Apr 2026 Apr 2026 openai.com Official leaderboard
7 Gemini 3.1 Pro 43.8% Google Feb 2026 Feb 2026 deepmind.google Official leaderboard
8 GPT-5.6 Luna 42.2% OpenAI Jul 2026 Jul 2026 openai.com Official leaderboard
9 Kimi K2.6 39.1% Alibaba Apr 2026 Apr 2026 kimi-k2.org Official leaderboard
10 MemGUI-Agent-235B 37.5% Kuaishou Jun 2026 arxiv.org Official leaderboard
11 Gemini 2.5 Pro Mar 2025 32.8% M3A Google DeepMind Mar 2025 May 2024 arxiv.org Official leaderboard
12 Seed2.0 Pro 31.2% ByteDance Seed Feb 2026 Feb 2026 seed.bytedance.com Official leaderboard
13 Agent-S2 + Gemini 2.5 Pro Mar 2025 27.3% UCSB Mar 2025 Apr 2025 arxiv.org Official leaderboard
14 Qwen3-VL-235B-A22B 24.2% thinking Alibaba Sep 2025 Feb 2026 arxiv.org Official leaderboard
15 Qwen3-VL-235B-A22B-Instruct 23.4% Alibaba Sep 2025 Feb 2026 arxiv.org Official leaderboard
16 MemGUI-8B-SFT 23.4% Kuaishou Jun 2026 arxiv.org Official leaderboard
17 Qwen3-VL-32B-Instruct 20.3% Alibaba Oct 2025 Feb 2026 arxiv.org Official leaderboard
18 GUI-Owl-1.5-8B-Instruct 11.7% Alibaba Feb 2026 arxiv.org Official leaderboard
19 GUI-Owl-1.5-32B-Instruct 10.9% Alibaba Feb 2026 arxiv.org Official leaderboard
20 Qwen3-VL-8B-Instruct 9.40% Alibaba Oct 2025 Nov 2025 arxiv.org Official leaderboard
21 GUI-Owl-7B 6.20% Alibaba Aug 2025 Aug 2025 Official leaderboard
22 Mobile-Agent-E + Gemini 2.5 Pro Mar 2025 5.50% UIUC & Alibaba Mar 2025 Jan 2025 arxiv.org Official leaderboard
23 UI-Venus-7B 5.50% InclusionAI Aug 2025 arxiv.org Official leaderboard
24 AppAgent + Gemini 2.5 Pro Mar 2025 3.10% Tencent Mar 2025 Dec 2023 arxiv.org Official leaderboard
25 Mobile-Agent-V2 + Gemini 2.5 Pro Mar 2025 3.10% Alibaba Mar 2025 Jun 2024 arxiv.org Official leaderboard
26 UI-TARS-1.5 3.10% ByteDance Seed Apr 2025 Jan 2025 arxiv.org Official leaderboard
27 SeeAct + Gemini 2.5 Pro Mar 2025 2.30% OSU NLP Mar 2025 Jan 2024 arxiv.org Official leaderboard
28 CogAgent 0% Tsinghua & Zhipu Dec 2023 Dec 2023 arxiv.org Official leaderboard
#ModelScoreSettingAgent byReleasedRun onLinkSource
1 Gemini 3.6 Flash 73.4% Google Jul 2026 Jul 2026 blog.google Official leaderboard
2 Qwen 3.8 Max 71.9% Alibaba Aug 2026 Aug 2026 qwen.ai Official leaderboard
3 GPT-5.5 57% OpenAI Apr 2026 Apr 2026 openai.com Official leaderboard
4 Gemini 3.1 Pro 43.8% Google Feb 2026 Feb 2026 deepmind.google Official leaderboard
5 GPT-5.6 Luna 42.2% OpenAI Jul 2026 Jul 2026 openai.com Official leaderboard
6 Kimi K2.6 39.1% Alibaba Apr 2026 Apr 2026 kimi-k2.org Official leaderboard
7 Gemini 2.5 Pro Mar 2025 32.8% M3A Google DeepMind Mar 2025 May 2024 arxiv.org Official leaderboard
8 Seed2.0 Pro 31.2% ByteDance Seed Feb 2026 Feb 2026 seed.bytedance.com Official leaderboard
9 Qwen3-VL-235B-A22B 24.2% thinking Alibaba Sep 2025 Feb 2026 arxiv.org Official leaderboard
10 Qwen3-VL-235B-A22B-Instruct 23.4% Alibaba Sep 2025 Feb 2026 arxiv.org Official leaderboard
11 Qwen3-VL-32B-Instruct 20.3% Alibaba Oct 2025 Feb 2026 arxiv.org Official leaderboard
12 Qwen3-VL-8B-Instruct 9.40% Alibaba Oct 2025 Nov 2025 arxiv.org Official leaderboard
13 GUI-Owl-7B 6.20% Alibaba Aug 2025 Aug 2025 Official leaderboard
14 UI-TARS-1.5 3.10% ByteDance Seed Apr 2025 Jan 2025 arxiv.org Official leaderboard
15 CogAgent 0% Tsinghua & Zhipu Dec 2023 Dec 2023 arxiv.org Official leaderboard
#ModelScoreSettingAgent byReleasedRun onLinkSource
1 Gemini 3.6 Flash 73.4% Google Jul 2026 Jul 2026 blog.google Official leaderboard
2 Qwen 3.8 Max 71.9% Alibaba Aug 2026 Aug 2026 qwen.ai Official leaderboard
3 GPT-5.5 57% OpenAI Apr 2026 Apr 2026 openai.com Official leaderboard
4 Gemini 3.1 Pro 43.8% Google Feb 2026 Feb 2026 deepmind.google Official leaderboard
5 GPT-5.6 Luna 42.2% OpenAI Jul 2026 Jul 2026 openai.com Official leaderboard
6 Kimi K2.6 39.1% Alibaba Apr 2026 Apr 2026 kimi-k2.org Official leaderboard
7 Seed2.0 Pro 31.2% ByteDance Seed Feb 2026 Feb 2026 seed.bytedance.com Official leaderboard
8 Qwen3-VL-235B-A22B 24.2% thinking Alibaba Sep 2025 Feb 2026 arxiv.org Official leaderboard
9 Qwen3-VL-235B-A22B-Instruct 23.4% Alibaba Sep 2025 Feb 2026 arxiv.org Official leaderboard
10 Qwen3-VL-32B-Instruct 20.3% Alibaba Oct 2025 Feb 2026 arxiv.org Official leaderboard
11 Qwen3-VL-8B-Instruct 9.40% Alibaba Oct 2025 Nov 2025 arxiv.org Official leaderboard
12 GUI-Owl-7B 6.20% Alibaba Aug 2025 Aug 2025 Official leaderboard
13 UI-TARS-1.5 3.10% ByteDance Seed Apr 2025 Jan 2025 arxiv.org Official leaderboard
14 CogAgent 0% Tsinghua & Zhipu Dec 2023 Dec 2023 arxiv.org Official leaderboard

Rankings by part

We read 7 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as easy, medium and hard. Gemini 3.6 Flash leads 4 of them, and Qwen 3.8 Max leads the other 3. Arrows show how far a model moved from its overall rank among the models we track.

easy

15 models we track have a score here.

  1. 1 Qwen 3.8 Max ↑1 68.8%
  2. 2 Gemini 3.6 Flash ↓1 66.7%
  3. 3 GPT-5.5 52.1%
  4. 4 Gemini 3.1 Pro 47.9%
  5. 5 Gemini 2.5 Pro Mar 2025 ↑2 39.6%
  6. 6 Kimi K2.6 39.6%
  7. 7 Qwen3-VL-32B-Instruct ↑4 35.4%
  8. 8 Seed2.0 Pro 35.4%
  9. 9 GPT-5.6 Luna ↓4 35.4%
  10. 10 Qwen3-VL-235B-A22B-Instruct 27.1%

4 models have held number one on the official leaderboard since its release in February 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind OpenAI
Own leaderboard Not tested since Aug 2026

Source: Official MemGUI-Bench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemini 3.6 Flash GPT-5.5 Gemini 3.1 Pro Gemini 2.5 Pro CogAgent 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemini 3.6 Flash GPT-5.5

Source: Official MemGUI-Bench leaderboard

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemini 3.6 Flash GPT-5.5 Gemini 3.1 Pro Seed2.0 Pro Qwen3-VL-235B-A22B GUI-Owl-7B CogAgent 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemini 3.6 Flash GPT-5.5

Source: Official MemGUI-Bench leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Computer and browser use

All 119 in Computer and browser use
  • BrowseComp In use Hard-to-find facts that need persistent web browsing. Short verifiable answers graded for exact correctness. Claude Fable 5 93.3%
  • ScreenSpot-Pro In use GUI grounding on high-resolution professional-software screenshots. The model must locate the target UI element. GPT-6 Astra 92.7%
  • AndroidWorld In use Agents complete 116 tasks across 20 real Android apps in an emulator. Each task checks the device's system state to judge success. GPT-6 Astra 100%
  • OSWorld 2.1 In use Agents complete 108 long-horizon workflows across 31 self-hosted websites and desktop apps, averaging 318 tool calls. Scored by binary completion and checkpoint partial credit. Claude Opus 5 44.3%
  • WebArena-Verified In use Agents complete 812 audited WebArena tasks on self-hosted websites. Deterministic evaluators check the agent's answer and captured network traces. Muse Spark 1.1 69%
  • OSWorld 2.1 (offline subset) In use Agents complete the 82 OSWorld 2.1 tasks on the official offline list, with no internet access. Scored by checkpoint partial credit and strict pass rate. Claude Fable 5.2 89.3%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (28 entries).
  • Pass Models scored 15 models we track, plus 13 other entries on its leaderboard.
  • Pass Newest score The newest is from August 2026, for Qwen 3.8 Max.
  • Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5.5 (#21).
  • Pass Headroom The best is 73.4%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on memgui-bench.github.io: 21 rows for models we track and 8 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check