Benchmark Almanac

CharXiv Extended

Muse Spark 1.2 87.1% leads since August 2026
Not in model cards

Models answer descriptive and reasoning questions about scientific charts. Mercor adds 150 new expert-written tasks to the 5000 public ones and reports both scores.

Held-out extension of charxiv. Mercor shows the original and extended score for each model. The month is approximate. The Wayback Machine first saved the earliest extended pages on August 15th, 2026.

Made by
Mercor
Released
August 2026
Kind
Documents, charts and text in images
Level
University
Scored
Checked automatically
Answers
Short answer
Tasks
Public, plus a private test set
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
39
Entries on its own leaderboard, 38 models we track

Muse Spark 1.2 leads with 87.1%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

39 entries. Scroll sideways for the rest.

  1. 87.1 Muse Spark 1.2
  2. 86.4 Muse Spark 1.1
  3. 85.3 Muse Spark Open
  4. 85.1 GPT-6 Astra
  5. 85.1 GPT-6.1 Sol
  6. 84.7 Claude Opus 5.5
  7. 84.2 Claude Sonnet 5.5
  8. 83.8 GPT-6 Sol
  9. 82.9 GPT-5.6 Sol
  10. 82.9 Qwen 3.8 Max
  11. 82.9 Claude Haiku 5.5
  12. 82.7 GPT-5.4
  13. 82.7 GPT-5.5
  14. 82.2 Claude Fable 5.1
  15. 81.8 Muse Spark 1.3
  16. 81.3 Gemini 3.5 Flash
  17. 81.1 GPT-5.6 Terra
  18. 80.7 Claude Fable 5
  19. 79.8 Claude Opus 5
  20. 79.6 Muse Wonder
  21. 79.1 Gemini 3.7 Flash
  22. 78.9 Gemini 3.8 Flash
  23. 78.7 GPT-5.6 Luna
  24. 78.7 Kimi K3
  25. 78.4 Gemini 3.6 Flash
  26. 78.2 Grok 4.7
  27. 77.3 Grok 4.6
  28. 76.9 Gemini 3.1 Pro
  29. 76.7 Claude Opus 4.8
  30. 76.4 GPT-6 Luna
  31. 76.2 Qwen3.5
  32. 76.2 GLM-5.3-Flash
  33. 76.0 Claude Opus 4.7
  34. 72.7 Grok 4.5
  35. 71.1 Claude Sonnet 5
  36. 69.8 Inkling
  37. 68.9 DeepSeek V4.1 Flash
  38. 64.4 Claude Sonnet 4.6
  39. 58.4 MiniMax-M3
  1. 1 Muse Spark 1.2 xhigh 87.1%
  2. 2 Muse Spark 1.1 xhigh 86.4%
  3. 3 Muse Spark Open xhigh 85.3%
  4. 4 GPT-6 Astra xhigh 85.1%
  5. 5 GPT-6.1 Sol max 85.1%
  6. 6 Claude Opus 5.5 max 84.7%
  7. 7 Claude Sonnet 5.5 max 84.2%
  8. 8 GPT-6 Sol max 83.8%
  9. 9 GPT-5.6 Sol max 82.9%
  10. 10 Qwen 3.8 Max xhigh 82.9%
  11. 11 Claude Haiku 5.5 max 82.9%
  12. 12 GPT-5.4 xhigh 82.7%
  13. 13 GPT-5.5 xhigh 82.7%
  14. 14 Claude Fable 5.1 max 82.2%
  15. 15 Muse Spark 1.3 max 81.8%
  16. 16 Gemini 3.5 Flash high 81.3%
  17. 17 GPT-5.6 Terra max 81.1%
  18. 18 Claude Fable 5 max 80.7%
  19. 19 Claude Opus 5 max 79.8%
  20. 20 Muse Wonder max 79.6%
  21. 21 Gemini 3.7 Flash high 79.1%
  22. 22 Gemini 3.8 Flash high 78.9%
  23. 23 GPT-5.6 Luna max 78.7%
  24. 24 Kimi K3 max 78.7%
  25. 25 Gemini 3.6 Flash high 78.4%
  26. 26 Grok 4.7 xhigh 78.2%
  27. 27 Grok 4.6 xhigh 77.3%
  28. 28 Gemini 3.1 Pro high 76.9%
  29. 29 Claude Opus 4.8 max 76.7%
  30. 30 GPT-6 Luna max 76.4%
  31. 31 Qwen3.5 76.2%
  32. 32 GLM-5.3-Flash max 76.2%
  33. 33 Claude Opus 4.7 max 76%
  34. 34 Grok 4.5 high 72.7%
  35. 35 Claude Sonnet 5 max 71.1%
  36. 36 Inkling high 69.8%
  37. 37 DeepSeek V4.1 Flash max 68.9%
  38. 38 Claude Sonnet 4.6 high 64.4%
  39. 39 MiniMax-M3 high 58.4%

Source: Official CharXiv Extended leaderboard

#EntryScoreSettingDateSource
1 Muse Spark 1.2 87.1% xhigh Aug 2026 Official leaderboard
2 Muse Spark 1.1 86.4% xhigh Jul 2026 Official leaderboard
3 Muse Spark Open 85.3% xhigh Oct 2026 Official leaderboard
4 GPT-6 Astra 85.1% xhigh Sep 2026 Official leaderboard
5 GPT-6.1 Sol 85.1% max Sep 2026 Official leaderboard
6 Claude Opus 5.5 84.7% max Sep 2026 Official leaderboard
7 Claude Sonnet 5.5 84.2% max Sep 2026 Official leaderboard
8 GPT-6 Sol 83.8% max Sep 2026 Official leaderboard
9 GPT-5.6 Sol 82.9% max Jul 2026 Official leaderboard
10 Qwen 3.8 Max 82.9% xhigh Aug 2026 Official leaderboard
11 Claude Haiku 5.5 82.9% max Oct 2026 Official leaderboard
12 GPT-5.4 82.7% xhigh Mar 2026 Official leaderboard
13 GPT-5.5 82.7% xhigh Apr 2026 Official leaderboard
14 Claude Fable 5.1 82.2% max Sep 2026 Official leaderboard
15 Muse Spark 1.3 81.8% max Sep 2026 Official leaderboard
16 Gemini 3.5 Flash 81.3% high May 2026 Official leaderboard
17 GPT-5.6 Terra 81.1% max Jul 2026 Official leaderboard
18 Claude Fable 5 80.7% max Jun 2026 Official leaderboard
19 Claude Opus 5 79.8% max Jul 2026 Official leaderboard
20 Muse Wonder 79.6% max Oct 2026 Official leaderboard
21 Gemini 3.7 Flash 79.1% high Aug 2026 Official leaderboard
22 Gemini 3.8 Flash 78.9% high Sep 2026 Official leaderboard
23 GPT-5.6 Luna 78.7% max Jul 2026 Official leaderboard
24 Kimi K3 78.7% max Jul 2026 Official leaderboard
25 Gemini 3.6 Flash 78.4% high Jul 2026 Official leaderboard
26 Grok 4.7 78.2% xhigh Sep 2026 Official leaderboard
27 Grok 4.6 77.3% xhigh Aug 2026 Official leaderboard
28 Gemini 3.1 Pro 76.9% high Feb 2026 Official leaderboard
29 Claude Opus 4.8 76.7% max May 2026 Official leaderboard
30 GPT-6 Luna 76.4% max Sep 2026 Official leaderboard
31 Qwen3.5 76.2% Official leaderboard
32 GLM-5.3-Flash 76.2% max Aug 2026 Official leaderboard
33 Claude Opus 4.7 76% max Apr 2026 Official leaderboard
34 Grok 4.5 72.7% high Jul 2026 Official leaderboard
35 Claude Sonnet 5 71.1% max Jun 2026 Official leaderboard
36 Inkling 69.8% high Jul 2026 Official leaderboard
37 DeepSeek V4.1 Flash 68.9% max Sep 2026 Official leaderboard
38 Claude Sonnet 4.6 64.4% high Feb 2026 Official leaderboard
39 MiniMax-M3 58.4% high Jun 2026 Official leaderboard

2 models have held number one on the official leaderboard since its release in August 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta
Own leaderboard Passed 85% in Aug 2026, by Muse Spark 1.1
Muse Spark 1.2 87.1%

Source: Official CharXiv Extended leaderboard

Muse Spark 1.1 on the official leaderboard passed 85% in July 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Muse Spark 1.2 GPT-5.4 Gemini 3.1 Pro Claude Sonnet 4.6 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Muse Spark 1.2 Gemini 3.1 Pro

Source: Official CharXiv Extended leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%