OmniScore-Bench
Models read sheet music, locate symbols, and transcribe passages. The evaluator checks their responses against annotations for the relevant part of the score.
The authors compare 12 models on 3,000 questions.
- Made by
- Tsinghua University and collaborators
- Released
- November 2026
- Kind
- Documents, charts and text in images
- Level
- University
- Scored
- Checked automatically
- Answers
- Short answer
- Measures
- Capability
- Subject
- Arts
- Language
- English
- Interaction
- Single turn
- Items from
- Real work and use
- Refresh
- One fixed set
- Run by
- A host
- In views
- Vision
- Strongest model tested
- GPT-6 Astra #4 on the capability index
- Last model added
- October 2026
- Scores refreshed
- November 20th, 2026
GPT-6.1 Sol leads with 91.0
Best score per model from the benchmark's paper, read on November 20th, 2026. A paper’s table doesn’t add newer models. The paper gives no overall score, so the headline is its first measure, localization F1 for noteheads and rests, in percent. Where a model ran with several settings or harnesses, we keep its best.
- 1 GPT-6.1 Sol 91.0
- 2 GPT-6 Astra 89.8
- 3
Gemini 4 Argon 75.1
- 4 Kimi K3.1 Pro 36.4
- 5 Kimi K3 20.9
- 6 Claude Opus 5.5 7.90
- 7
Qwen 3.8 Max 7.50
- 8 Claude Fable 5.1 4.10
- 9
DeepSeek V4.1 Flash 0.40
- 10
Gemini 3.8 Flash 0.20
- 11 GLM-5.3-Flash 0.10
- 12
Qwen3.7-Plus 0.00
Source: The OmniScore-Bench paper
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6.1 Sol | 91.0 | Sep 2026 | Official leaderboard | |
| 2 | GPT-6 Astra | 89.8 | Sep 2026 | Official leaderboard | |
| 3 | | 75.1 | Sep 2026 | Official leaderboard | |
| 4 | Kimi K3.1 Pro | 36.4 | Oct 2026 | Official leaderboard | |
| 5 | Kimi K3 | 20.9 | Jul 2026 | Official leaderboard | |
| 6 | Claude Opus 5.5 | 7.90 | Sep 2026 | Official leaderboard | |
| 7 | | 7.50 | Aug 2026 | Official leaderboard | |
| 8 | Claude Fable 5.1 | 4.10 | Sep 2026 | Official leaderboard | |
| 9 | | 0.40 | Sep 2026 | Official leaderboard | |
| 10 | | 0.20 | Sep 2026 | Official leaderboard | |
| 11 | GLM-5.3-Flash | 0.10 | Aug 2026 | Official leaderboard | |
| 12 | | 0.00 | Jun 2026 | Official leaderboard |
Rankings by part
We read 24 parts from the benchmark’s paper, each ranking models on one part of the test, such as T1.1: Noteheads and rests, T1.2: Clefs and accidentals and T1.3: Rhythmic marks. GPT-6 Astra leads 17 of them, and 4 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
T1.1: Noteheads and rests
12 models we track have a score here.
- 1 GPT-6.1 Sol 91%
- 2 GPT-6 Astra 89.8%
- 3
Gemini 4 Argon 75.1%
- 4 Kimi K3.1 Pro 36.4%
- 5 Kimi K3 20.9%
- 6 Claude Opus 5.5 7.90%
- 7
Qwen 3.8 Max 7.50%
- 8 Claude Fable 5.1 4.10%
- 9
DeepSeek V4.1 Flash 0.40%
- 10
Gemini 3.8 Flash 0.20%
T1.2: Clefs and accidentals
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 94.8%
- 2
Gemini 4 Argon ↑1 92%
- 3 GPT-6.1 Sol ↓2 91.7%
- 4 Kimi K3.1 Pro 55%
- 5
Qwen 3.8 Max ↑2 37%
- 6 Kimi K3 ↓1 29.9%
- 7 Claude Fable 5.1 ↑1 26.6%
- 8 Claude Opus 5.5 ↓2 24%
- 9
Gemini 3.8 Flash ↑1 12.6%
- 10 GLM-5.3-Flash ↑1 3.70%
T1.3: Rhythmic marks
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 76.2%
- 2 GPT-6.1 Sol ↓1 58.9%
- 3
Gemini 4 Argon 43.3%
- 4 Kimi K3.1 Pro 21.5%
- 5 Kimi K3 8.20%
- 6 Claude Opus 5.5 4.70%
- 7
Qwen 3.8 Max 4.10%
- 8 Claude Fable 5.1 3.30%
- 9
Gemini 3.8 Flash ↑1 0.40%
- 10
DeepSeek V4.1 Flash ↓1 0.30%
T1.4: Expression symbols
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 86.1%
- 2 GPT-6.1 Sol ↓1 85.4%
- 3
Gemini 4 Argon 73.4%
- 4 Kimi K3.1 Pro 60.1%
- 5 Kimi K3 48.8%
- 6
Qwen 3.8 Max ↑1 41.3%
- 7 Claude Opus 5.5 ↓1 24.8%
- 8 Claude Fable 5.1 17.7%
- 9
Gemini 3.8 Flash ↑1 16.9%
- 10 GLM-5.3-Flash ↑1 0.90%
T2.1: Systems and staves
12 models we track have a score here.
- 1 GPT-6.1 Sol 98.9%
- 2
Gemini 4 Argon ↑1 98.8%
- 3 GPT-6 Astra ↓1 98.6%
- 4 Kimi K3.1 Pro 94%
- 5 Kimi K3 91.7%
- 6 Claude Fable 5.1 ↑2 88.3%
- 7 Claude Opus 5.5 ↓1 86.7%
- 8
Qwen 3.8 Max ↓1 86.5%
- 9 GLM-5.3-Flash ↑2 44.5%
- 10
DeepSeek V4.1 Flash ↓1 36%
T2.2: Measure regions
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 98.4%
- 2 GPT-6.1 Sol ↓1 98.2%
- 3
Gemini 4 Argon 97.4%
- 4 Claude Opus 5.5 ↑2 94%
- 5 Kimi K3.1 Pro ↓1 91.5%
- 6 Kimi K3 ↓1 86.9%
- 7 Claude Fable 5.1 ↑1 86.6%
- 8
Qwen 3.8 Max ↓1 84.4%
- 9
DeepSeek V4.1 Flash 41.4%
- 10 GLM-5.3-Flash ↑1 31.4%
T2.3: Structural membership
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 95.1%
- 2 GPT-6.1 Sol ↓1 91%
- 3
Gemini 4 Argon 89.3%
- 4 Claude Opus 5.5 ↑2 88.1%
- 5 Kimi K3.1 Pro ↓1 70.3%
- 6 Kimi K3 ↓1 66%
- 7 Claude Fable 5.1 ↑1 63.5%
- 8
DeepSeek V4.1 Flash ↑1 54.1%
- 9
Qwen 3.8 Max ↓2 46.9%
- 10
Gemini 3.8 Flash 17.4%
T2.4: Continuation
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 99.8%
- 2 GPT-6.1 Sol ↓1 99.7%
- 3
Gemini 4 Argon 98%
- 4
Qwen 3.8 Max ↑3 91.1%
- 5 Kimi K3.1 Pro ↓1 89.5%
- 6 Claude Fable 5.1 ↑2 89.2%
- 7 Kimi K3 ↓2 88.3%
- 8 Claude Opus 5.5 ↓2 77%
- 9 GLM-5.3-Flash ↑2 48.1%
- 10
DeepSeek V4.1 Flash ↓1 32.5%
T3.1: Tie and slur endpoints
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 88.7%
- 2 GPT-6.1 Sol ↓1 86.8%
- 3 Claude Opus 5.5 ↑3 66.2%
- 4
Qwen 3.8 Max ↑3 65.4%
- 5
Gemini 4 Argon ↓2 57.7%
- 6 Kimi K3.1 Pro ↓2 57.3%
- 7 Claude Fable 5.1 ↑1 53.2%
- 8 Kimi K3 ↓3 52.4%
- 9 GLM-5.3-Flash ↑2 30.2%
- 10
DeepSeek V4.1 Flash ↓1 27%
T3.2: Chord and beam groups
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 90.4%
- 2 GPT-6.1 Sol ↓1 87.9%
- 3 Claude Opus 5.5 ↑3 76%
- 4
Gemini 4 Argon ↓1 66.7%
- 5 Kimi K3.1 Pro ↓1 66.7%
- 6 Claude Fable 5.1 ↑2 61%
- 7
Qwen 3.8 Max 52.5%
- 8 Kimi K3 ↓3 47.2%
- 9
DeepSeek V4.1 Flash 26.6%
- 10
Gemini 3.8 Flash 22.9%
T3.3: Cross-voice timing
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 93.9%
- 2 GPT-6.1 Sol ↓1 90.2%
- 3
Gemini 4 Argon 78.4%
- 4 Claude Opus 5.5 ↑2 76.1%
- 5 Kimi K3.1 Pro ↓1 70.6%
- 6
Qwen 3.8 Max ↑1 70.2%
- 7 Claude Fable 5.1 ↑1 68.7%
- 8 Kimi K3 ↓3 60.9%
- 9 GLM-5.3-Flash ↑2 42.3%
- 10
DeepSeek V4.1 Flash ↓1 27.5%
T3.4: Additional typed connections
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 93.3%
- 2 GPT-6.1 Sol ↓1 89.7%
- 3
Gemini 4 Argon 65.7%
- 4 Claude Fable 5.1 ↑4 64.7%
- 5 Kimi K3.1 Pro ↓1 61.4%
- 6 Claude Opus 5.5 57.3%
- 7
Qwen 3.8 Max 52.2%
- 8 Kimi K3 ↓3 49.5%
- 9
DeepSeek V4.1 Flash 18.4%
- 10 GLM-5.3-Flash ↑1 14.6%
T4.1: Monophonic excerpts
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 23.5
- 2 GPT-6.1 Sol ↓1 28.0
- 3
Gemini 4 Argon 46.2
- 4 Claude Opus 5.5 ↑2 46.3
- 5 Claude Fable 5.1 ↑3 64.4
- 6 Kimi K3.1 Pro ↓2 64.4
- 7 Kimi K3 ↓2 71.7
- 8
Qwen 3.8 Max ↓1 81.1
- 9
Gemini 3.8 Flash ↑1 84.4
- 10 GLM-5.3-Flash ↑1 89.6
T4.2: Single-staff polyphony
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 21.4
- 2 GPT-6.1 Sol ↓1 23.5
- 3
Gemini 4 Argon 29.0
- 4 Claude Opus 5.5 ↑2 55.1
- 5 Kimi K3.1 Pro ↓1 64.1
- 6 Claude Fable 5.1 ↑2 69.2
- 7 Kimi K3 ↓2 69.3
- 8
Gemini 3.8 Flash ↑2 79.8
- 9
Qwen 3.8 Max ↓2 81.1
- 10
DeepSeek V4.1 Flash ↓1 86.1
T4.3: Single-page transcription
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 26.8
- 2 GPT-6.1 Sol ↓1 30.5
- 3 Claude Opus 5.5 ↑3 43.5
- 4
Gemini 4 Argon ↓1 52.0
- 5 Claude Fable 5.1 ↑3 59.3
- 6 Kimi K3.1 Pro ↓2 62.8
- 7 Kimi K3 ↓2 70.9
- 8
Qwen 3.8 Max ↓1 76.1
- 9 GLM-5.3-Flash ↑2 83.7
- 10
Gemini 3.8 Flash 85.7
T4.4: Multipage transcription
12 models we track have a score here.
- 1 Claude Opus 5.5 ↑5 39.9
- 2 GPT-6 Astra 50.6
- 3 GPT-6.1 Sol ↓2 55.2
- 4 Kimi K3.1 Pro 67.0
- 5 Kimi K3 74.5
- 6
Qwen 3.8 Max ↑1 76.2
- 7 Claude Fable 5.1 ↑1 77.1
- 8
Gemini 4 Argon ↓5 88.3
- 9
DeepSeek V4.1 Flash 89.0
- 10 GLM-5.3-Flash ↑1 93.1
T5.1: Meter and timing
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 89.8%
- 2 Claude Opus 5.5 ↑4 84.6%
- 3 GPT-6.1 Sol ↓2 83.8%
- 4
Gemini 4 Argon ↓1 76.3%
- 5 Kimi K3.1 Pro ↓1 64.8%
- 6 Claude Fable 5.1 ↑2 62.4%
- 7
Qwen 3.8 Max 58.3%
- 8 Kimi K3 ↓3 57.2%
- 9
DeepSeek V4.1 Flash 28.4%
- 10
Gemini 3.8 Flash 27.4%
T5.2: Written and sounding pitch
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 81.7%
- 2 GPT-6.1 Sol ↓1 75.7%
- 3 Claude Opus 5.5 ↑3 56%
- 4
Gemini 4 Argon ↓1 55.3%
- 5 Kimi K3.1 Pro ↓1 53.2%
- 6 Claude Fable 5.1 ↑2 41.7%
- 7
Qwen 3.8 Max 37.1%
- 8 Kimi K3 ↓3 23.8%
- 9
DeepSeek V4.1 Flash 5.60%
- 10
Gemini 3.8 Flash 3.80%
T5.3: Sounding events
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 77.2%
- 2 GPT-6.1 Sol ↓1 66.2%
- 3 Claude Opus 5.5 ↑3 46.3%
- 4
Gemini 4 Argon ↓1 32.2%
- 5 Kimi K3.1 Pro ↓1 21.3%
- 6 Kimi K3 ↓1 13.1%
- 7 Claude Fable 5.1 ↑1 13.1%
- 8
Qwen 3.8 Max ↓1 8.20%
- 9
Gemini 3.8 Flash ↑1 2.00%
- 10 GLM-5.3-Flash ↑1 1.10%
T5.4: Repeat traversal
12 models we track have a score here.
- 1 Claude Opus 5.5 ↑5 79.5%
- 2 GPT-6 Astra 79.3%
- 3 GPT-6.1 Sol ↓2 78.2%
- 4
Gemini 4 Argon ↓1 67.8%
- 5 Claude Fable 5.1 ↑3 64.6%
- 6 Kimi K3.1 Pro ↓2 64%
- 7
Qwen 3.8 Max 62.3%
- 8 Kimi K3 ↓3 58.5%
- 9
Gemini 3.8 Flash ↑1 32.2%
- 10 GLM-5.3-Flash ↑1 30.6%
T6.1: Visible fact extraction
12 models we track have a score here.
- 1 GPT-6 Astra ↑1 100%
- 2
Gemini 4 Argon ↑1 100%
- 3 GPT-6.1 Sol ↓2 99.4%
- 4 Claude Opus 5.5 ↑2 99%
- 5 Kimi K3.1 Pro ↓1 98.3%
- 6 Kimi K3 ↓1 97.7%
- 7 Claude Fable 5.1 ↑1 97.5%
- 8
Qwen 3.8 Max ↓1 97.3%
- 9 GLM-5.3-Flash ↑2 97.2%
- 10
DeepSeek V4.1 Flash ↓1 93.1%
T6.2: Counting and conditional retrieval
12 models we track have a score here.
- 1
Qwen 3.8 Max ↑6 95.1%
- 2 GPT-6.1 Sol ↓1 93.4%
- 3
Gemini 4 Argon 93.4%
- 4 GPT-6 Astra ↓2 90.2%
- 5 Claude Fable 5.1 ↑3 73.8%
- 6
Gemini 3.8 Flash ↑4 70.5%
- 7 GLM-5.3-Flash ↑4 62.3%
- 8 Claude Opus 5.5 ↓2 62.3%
- 9
DeepSeek V4.1 Flash 59%
- 10 Kimi K3.1 Pro ↓6 57.4%
T6.3: Conditional musical analysis
12 models we track have a score here.
- 1
Gemini 4 Argon ↑2 91.3%
- 2 GPT-6 Astra 88.6%
- 3 GPT-6.1 Sol ↓2 86.5%
- 4 Claude Opus 5.5 ↑2 81%
- 5 Claude Fable 5.1 ↑3 61.1%
- 6 Kimi K3.1 Pro ↓2 58.1%
- 7
Qwen 3.8 Max 54.5%
- 8 Kimi K3 ↓3 43.7%
- 9
Gemini 3.8 Flash ↑1 25.6%
- 10 GLM-5.3-Flash ↑1 23.5%
T6.4: Fragment transformation
12 models we track have a score here.
- 1 GPT-6.1 Sol 91.2%
- 2 Claude Opus 5.5 ↑4 89.3%
- 3 GPT-6 Astra ↓1 89.1%
- 4
Gemini 4 Argon ↓1 79%
- 5 Claude Fable 5.1 ↑3 69%
- 6 Kimi K3.1 Pro ↓2 66.5%
- 7
Qwen 3.8 Max 62%
- 8 Kimi K3 ↓3 55.4%
- 9
Gemini 3.8 Flash ↑1 48.1%
- 10
DeepSeek V4.1 Flash ↓1 47.1%
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: The OmniScore-Bench paper
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official OmniScore-Bench leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Vision
- MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy.
Qwen 3.6 Plus 86%
- MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
- CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
- MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy.
Seed2.1 Pro 90.7%
- MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy.
Qwen 3.8 Max 97.7%
- MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy.
Qwen 3.8 Max 85.9%
Data audit
9 of 10 checks pass, 1 warning. Look first at the Task access check. We don’t know whether the tasks are public. Every benchmark
Catalog record 1 warning 1 pass
- Warning Task access We don’t know whether the tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the benchmark’s paper (12 entries).
- Pass Models scored 12 models we track.
- Pass Newest score The newest is from October 2026, for Kimi K3.1 Pro.
- Pass Frontier models 5 of the top 10 have a score, led by GPT-6 Astra (#4).
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on arxiv.org: 12 rows for models we track at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.