SuperCLUE-OnDevice
Phone-based models respond to requests in Chinese.
SuperCLUE reports scores in a public workbook.
- Made by
- SuperCLUE
- Kind
- Overall chat quality
- Level
- Everyday
- Scored
- Graded by people
- Answers
- Open-ended
- Measures
- Capability
- Language
- Chinese
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- A host
- Strongest model tested
-
Qwen 3.8 Max #30 on the capability index
- Last model added
- August 2026
- Scores refreshed
- November 20th, 2026
Gemini 3.6 Flash leads with 93.6 among models released by August 2026 Gemini 3.6 Flash leads with 93.6 among models released by August 2026
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.
The newest model with a score here came out in August 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 1
Gemini 3.6 Flash 云端 93.6
- 2
Doubao Seed 2.1 Pro 云端 93.0
- 3
Qwen 3.8 Max 云端 92.3
- 4 BlueLM-3.5-Nano-3B 端侧 89.9
- 5
Qwen3.5-9B 端侧 87.8
- 6 GLM-5V-Turbo 云端 87.8
- 7
Qwen3.5-4B 端侧 85.2
- 8 GLM-4.6V-Flash 端侧 81.4
- 9
Gemma 4 E4B 端侧 73.0
- 10
Gemma 4 E2B 端侧 66.5
Source: Official SuperCLUE-OnDevice leaderboard
- 1
Gemini 3.6 Flash 云端 93.6
- 2
Doubao Seed 2.1 Pro 云端 93.0
- 3
Qwen 3.8 Max 云端 92.3
- 4
Qwen3.5-9B 端侧 87.8
- 5 GLM-5V-Turbo 云端 87.8
- 6
Qwen3.5-4B 端侧 85.2
- 7 GLM-4.6V-Flash 端侧 81.4
- 8
Gemma 4 E4B 端侧 73.0
- 9
Gemma 4 E2B 端侧 66.5
Source: Official SuperCLUE-OnDevice leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 93.6 | 云端 | Jul 2026 | Official leaderboard |
| 2 | | 93.0 | 云端 | Jun 2026 | Official leaderboard |
| 3 | | 92.3 | 云端 | Aug 2026 | Official leaderboard |
| 4 | BlueLM-3.5-Nano-3B | 89.9 | 端侧 | Official leaderboard | |
| 5 | | 87.8 | 端侧 | Feb 2026 | Official leaderboard |
| 6 | GLM-5V-Turbo | 87.8 | 云端 | Apr 2026 | Official leaderboard |
| 7 | | 85.2 | 端侧 | Feb 2026 | Official leaderboard |
| 8 | GLM-4.6V-Flash | 81.4 | 端侧 | Dec 2025 | Official leaderboard |
| 9 | | 73.0 | 端侧 | Mar 2026 | Official leaderboard |
| 10 | | 66.5 | 端侧 | Mar 2026 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 93.6 | 云端 | Jul 2026 | Official leaderboard |
| 2 | | 93.0 | 云端 | Jun 2026 | Official leaderboard |
| 3 | | 92.3 | 云端 | Aug 2026 | Official leaderboard |
| 4 | | 87.8 | 端侧 | Feb 2026 | Official leaderboard |
| 5 | GLM-5V-Turbo | 87.8 | 云端 | Apr 2026 | Official leaderboard |
| 6 | | 85.2 | 端侧 | Feb 2026 | Official leaderboard |
| 7 | GLM-4.6V-Flash | 81.4 | 端侧 | Dec 2025 | Official leaderboard |
| 8 | | 73.0 | 端侧 | Mar 2026 | Official leaderboard |
| 9 | | 66.5 | 端侧 | Mar 2026 | Official leaderboard |
Rankings by part
We read 15 parts from the benchmark’s own leaderboard in 3 groups: 文本理解, 图像理解 and 视频理解. Each ranks models on one part of the test. Doubao Seed 2.1 Pro leads 6 of them, and 2 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
分数
9 models we track have a score here.
- 1
Gemini 3.6 Flash 95.2
- 2
Qwen 3.8 Max ↑1 94.6
- 3 GLM-5V-Turbo ↑2 93.0
- 4
Doubao Seed 2.1 Pro ↓2 93.0
- 5
Qwen3.5-9B ↓1 90.0
- 6
Gemma 4 E4B ↑2 88.7
- 7 GLM-4.6V-Flash 87.7
- 8
Qwen3.5-4B ↓2 87.0
- 9
Gemma 4 E2B 81.6
事件抽取
9 models we track have a score here.
- 1
Gemini 3.6 Flash 98.6
- 2 GLM-5V-Turbo ↑3 98.4
- 3
Qwen 3.8 Max 97.2
- 4 GLM-4.6V-Flash ↑3 94.9
- 5
Doubao Seed 2.1 Pro ↓3 94.0
- 6
Qwen3.5-9B ↓2 93.5
- 7
Gemma 4 E2B ↑2 91.0
- 8
Gemma 4 E4B 87.2
- 9
Qwen3.5-4B ↓3 85.1
文档总结
9 models we track have a score here.
- 1
Qwen 3.8 Max ↑2 94.6
- 2
Gemini 3.6 Flash ↓1 93.7
- 3
Doubao Seed 2.1 Pro ↓1 92.4
- 4
Qwen3.5-9B 89.4
- 5 GLM-5V-Turbo 89.2
- 6
Qwen3.5-4B 88.4
- 7
Gemma 4 E4B ↑1 87.7
- 8
Gemma 4 E2B ↑1 86.7
- 9 GLM-4.6V-Flash ↓2 85.6
会话总结
9 models we track have a score here.
- 1
Doubao Seed 2.1 Pro ↑1 98.7
- 2
Qwen 3.8 Max ↑1 98.0
- 3
Gemini 3.6 Flash ↓2 98.0
- 4 GLM-5V-Turbo ↑1 97.0
- 5
Qwen3.5-9B ↓1 90.7
- 6
Gemma 4 E4B ↑2 88.7
- 7
Qwen3.5-4B ↓1 88.1
- 8
Gemma 4 E2B ↑1 84.3
- 9 GLM-4.6V-Flash ↓2 79.9
NER
9 models we track have a score here.
- 1
Gemini 3.6 Flash 86.5
- 2
Qwen 3.8 Max ↑1 83.3
- 3
Gemma 4 E4B ↑5 82.7
- 4 GLM-5V-Turbo ↑1 81.2
- 5 GLM-4.6V-Flash ↑2 81.0
- 6
Doubao Seed 2.1 Pro ↓4 80.3
- 7
Qwen3.5-9B ↓3 78.2
- 8
Qwen3.5-4B ↓2 76.6
- 9
Gemma 4 E2B 75.1
问答
9 models we track have a score here.
- 1
Qwen 3.8 Max ↑2 99.8
- 2
Doubao Seed 2.1 Pro 99.7
- 3 GLM-5V-Turbo ↑2 99.5
- 4
Gemini 3.6 Flash ↓3 99.5
- 5
Qwen3.5-9B ↓1 98.1
- 6
Gemma 4 E4B ↑2 97.4
- 7 GLM-4.6V-Flash 96.9
- 8
Qwen3.5-4B ↓2 96.9
- 9
Gemma 4 E2B 70.9
分数
9 models we track have a score here.
- 1
Doubao Seed 2.1 Pro ↑1 95.9
- 2
Gemini 3.6 Flash ↓1 95.9
- 3
Qwen 3.8 Max 93.4
- 4 GLM-5V-Turbo ↑1 90.7
- 5
Qwen3.5-9B ↓1 89.1
- 6
Qwen3.5-4B 87.5
- 7 GLM-4.6V-Flash 76.1
- 8
Gemma 4 E4B 68.0
- 9
Gemma 4 E2B 58.9
视觉QA
9 models we track have a score here.
- 1
Gemini 3.6 Flash 96.8
- 2
Qwen 3.8 Max ↑1 96.4
- 3
Doubao Seed 2.1 Pro ↓1 94.2
- 4 GLM-5V-Turbo ↑1 90.0
- 5
Qwen3.5-9B ↓1 86.3
- 6 GLM-4.6V-Flash ↑1 85.6
- 7
Qwen3.5-4B ↓1 85.6
- 8
Gemma 4 E4B 79.8
- 9
Gemma 4 E2B 67.9
OCR
9 models we track have a score here.
- 1
Qwen 3.8 Max ↑2 97.0
- 2
Gemini 3.6 Flash ↓1 96.8
- 3
Doubao Seed 2.1 Pro ↓1 96.1
- 4 GLM-5V-Turbo ↑1 91.8
- 5
Qwen3.5-4B ↑1 90.4
- 6
Qwen3.5-9B ↓2 90.0
- 7
Gemma 4 E4B ↑1 73.1
- 8 GLM-4.6V-Flash ↓1 65.9
- 9
Gemma 4 E2B 64.5
信息提取
9 models we track have a score here.
- 1
Doubao Seed 2.1 Pro ↑1 96.4
- 2
Gemini 3.6 Flash ↓1 96.1
- 3
Qwen 3.8 Max 94.3
- 4
Qwen3.5-9B 93.9
- 5
Qwen3.5-4B ↑1 91.4
- 6 GLM-4.6V-Flash ↑1 90.0
- 7 GLM-5V-Turbo ↓2 86.8
- 8
Gemma 4 E4B 68.9
- 9
Gemma 4 E2B 58.9
视觉定位
9 models we track have a score here.
- 1
Doubao Seed 2.1 Pro ↑1 97.3
- 2
Qwen3.5-4B ↑4 96.0
- 3 GLM-5V-Turbo ↑2 95.3
- 4
Gemini 3.6 Flash ↓3 93.3
- 5
Qwen3.5-9B ↓1 92.0
- 6
Qwen 3.8 Max ↓3 82.0
- 7 GLM-4.6V-Flash 70.7
- 8
Gemma 4 E4B 56.7
- 9
Gemma 4 E2B 46.7
表格提取
9 models we track have a score here.
- 1
Qwen 3.8 Max ↑2 97.3
- 2
Gemini 3.6 Flash ↓1 96.3
- 3
Doubao Seed 2.1 Pro ↓1 95.5
- 4 GLM-5V-Turbo ↑1 89.3
- 5
Qwen3.5-9B ↓1 83.4
- 6
Qwen3.5-4B 74.3
- 7 GLM-4.6V-Flash 68.5
- 8
Gemma 4 E4B 61.4
- 9
Gemma 4 E2B 56.7
分数
9 models we track have a score here.
- 1
Doubao Seed 2.1 Pro ↑1 85.5
- 2
Gemini 3.6 Flash ↓1 84.1
- 3
Qwen 3.8 Max 83.5
- 4
Qwen3.5-9B 79.2
- 5 GLM-4.6V-Flash ↑2 79.0
- 6
Qwen3.5-4B 74.6
- 7 GLM-5V-Turbo ↓2 67.2
- 8
Gemma 4 E2B ↑1 47.4
- 9
Gemma 4 E4B ↓1 46.0
视频QA
9 models we track have a score here.
- 1
Qwen 3.8 Max ↑2 96.0
- 2
Doubao Seed 2.1 Pro 94.0
- 3
Gemini 3.6 Flash ↓2 94.0
- 4 GLM-4.6V-Flash ↑3 91.0
- 5
Qwen3.5-9B ↓1 90.0
- 6 GLM-5V-Turbo ↓1 89.0
- 7
Qwen3.5-4B ↓1 86.0
- 8
Gemma 4 E4B 75.0
- 9
Gemma 4 E2B 65.0
视频时序定位
9 models we track have a score here.
- 1
Doubao Seed 2.1 Pro ↑1 77.1
- 2
Gemini 3.6 Flash ↓1 74.2
- 3
Qwen 3.8 Max 71.0
- 4
Qwen3.5-9B 68.3
- 5 GLM-4.6V-Flash ↑2 66.9
- 6
Qwen3.5-4B 63.3
- 7 GLM-5V-Turbo ↓2 45.5
- 8
Gemma 4 E2B ↑1 29.8
- 9
Gemma 4 E4B ↓1 17.0
4 models have held number one on the official leaderboard since 2023
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official SuperCLUE-OnDevice leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official SuperCLUE-OnDevice leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Instructions and chat
- IFEval In use Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy.
Gemma 4 31B IT 98.9%
- IFBench In use 58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose). Flint 89.2%
- MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
- Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
- AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics.
MAI-Thinking-1 85%
- MM-IFEval In use Multimodal models follow image-grounded instructions with exact output constraints. Checks mix rule-based verification and a judge model. GPT-4o 64.6%
Data audit
8 of 9 checks pass, 1 warning. Look first at the Task access check. We don’t know whether the tasks are public. Every benchmark
Catalog record 1 warning 1 pass
- Warning Task access We don’t know whether the tasks are public.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (10 entries).
- Pass Models scored 9 models we track, plus 1 other entry on its leaderboard.
- Pass Newest score The newest is from August 2026, for Qwen 3.8 Max.
- Note Frontier models None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30).
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on superclueai.com: 9 rows for models we track and 1 other entry at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.