Benchmark Almanac

Blueprint-Bench 2

Gemini 4 Carbon 58.6% leads since October 2026
In use

Agents turn about 20 interior photos of each of 50 apartments into 2D floor plans. Scores compare room connectivity with ground truth, normalized against random.

The leader now ties the human baseline, which was measured on only 12 apartments.

Made by
Andon Labs
Released
May 2026
Kind
Spatial reasoning and counting
Level
Everyday
Scored
Checked automatically
Answers
Open-ended
Tasks
Kept private
Strongest model tested
Claude Fable 5.2 #1 on the capability index
Last in a launch post
October 2026
Last model added
November 2026
Scores refreshed
November 12th, 2026
38
Models scored on its own leaderboard
3
Launch posts list it, from 2 labs
2
System cards report it, from 2 labs

Gemini 4 Carbon leads with 58.6%

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best.

38 models. Scroll sideways for the rest.

  1. 58.6 Gemini 4 Carbon
  2. 56.6 Claude Fable 5.2
  3. 54.4 Gemini 4 Argon
  4. 51.2 Claude Opus 5.5
  5. 49.7 GPT-6 Astra
  6. 43.2 GPT-6.1 Sol
  7. 42.1 GPT-6.1 Terra
  8. 41.9 Claude Fable 5.1
  9. 39.3 Claude Sonnet 5.5
  10. 38.6 Claude Fable 5
  11. 38.6 Gemini 3.8 Flash
  12. 36.9 GPT-6 Sol
  13. 36.2 GPT-5.5
  14. 33.6 Gemini 3.5 Flash
  15. 33.6 GPT-5.6 Sol
  16. 33.2 Grok 4.6
  17. 33.0 Claude Haiku 5.5
  18. 32.5 Grok 4.7
  19. 31.7 Muse Wonder
  20. 31.2 Gemini 3.6 Flash
  21. 31.2 GPT-6 Luna
  22. 30.8 GPT-5.6 Terra
  23. 30.4 Claude Opus 5
  24. 29.5 Kimi K3
  25. 27.3 Grok 4.5
  26. 27.1 GPT-5.4
  27. 26.5 Gemini 3.1 Pro
  28. 24.9 Claude Sonnet 5
  29. 24.5 Claude Opus 4.7
  30. 22.6 GPT-5.6 Luna
  31. 14.5 Claude Opus 4.8
  32. 6.70 Claude Sonnet 4.6
  33. 3.90 Kimi K2.6
  34. 0.0 Claude Haiku 4.5
  35. 0.0 Gemini 3 Flash
  36. 0.0 Grok 4.20
  37. 0.0 Gemini Robotics-ER 1.6
  38. 0.0 Grok 4.3
  1. 1 Gemini 4 Carbon 58.6%
  2. 2 Claude Fable 5.2 56.6%
  3. 3 Gemini 4 Argon 54.4%
  4. 4 Claude Opus 5.5 51.2%
  5. 5 GPT-6 Astra 49.7%
  6. 6 GPT-6.1 Sol 43.2%
  7. 7 GPT-6.1 Terra 42.1%
  8. 8 Claude Fable 5.1 41.9%
  9. 9 Claude Sonnet 5.5 39.3%
  10. 10 Claude Fable 5 38.6%
  11. 11 Gemini 3.8 Flash 38.6%
  12. 12 GPT-6 Sol 36.9%
  13. 13 GPT-5.5 36.2%
  14. 14 Gemini 3.5 Flash 33.6%
  15. 15 GPT-5.6 Sol 33.6%
  16. 16 Grok 4.6 33.2%
  17. 17 Claude Haiku 5.5 33%
  18. 18 Grok 4.7 32.5%
  19. 19 Muse Wonder 31.7%
  20. 20 Gemini 3.6 Flash 31.2%
  21. 21 GPT-6 Luna 31.2%
  22. 22 GPT-5.6 Terra 30.8%
  23. 23 Claude Opus 5 30.4%
  24. 24 Kimi K3 29.5%
  25. 25 Grok 4.5 27.3%
  26. 26 GPT-5.4 27.1%
  27. 27 Gemini 3.1 Pro 26.5%
  28. 28 Claude Sonnet 5 24.9%
  29. 29 Claude Opus 4.7 24.5%
  30. 30 GPT-5.6 Luna 22.6%
  31. 31 Claude Opus 4.8 14.5%
  32. 32 Claude Sonnet 4.6 6.70%
  33. 33 Kimi K2.6 3.90%
  34. 34 Claude Haiku 4.5 0%
  35. 35 Gemini 3 Flash 0%
  36. 36 Grok 4.20 reasoning 0%
  37. 37 Gemini Robotics-ER 1.6 0%
  38. 38 Grok 4.3 0%

Source: Official Blueprint-Bench 2 leaderboard

#ModelScoreSettingDateSource
1 Gemini 4 Carbon 58.6% Oct 2026 Official leaderboard
2 Claude Fable 5.2 56.6% Nov 2026 Official leaderboard
3 Gemini 4 Argon 54.4% Sep 2026 Official leaderboard
4 Claude Opus 5.5 51.2% Sep 2026 Official leaderboard
5 GPT-6 Astra 49.7% Sep 2026 Official leaderboard
6 GPT-6.1 Sol 43.2% Sep 2026 Official leaderboard
7 GPT-6.1 Terra 42.1% Oct 2026 Official leaderboard
8 Claude Fable 5.1 41.9% Sep 2026 Official leaderboard
9 Claude Sonnet 5.5 39.3% Sep 2026 Official leaderboard
10 Claude Fable 5 38.6% Jun 2026 Official leaderboard
11 Gemini 3.8 Flash 38.6% Sep 2026 Official leaderboard
12 GPT-6 Sol 36.9% Sep 2026 Official leaderboard
13 GPT-5.5 36.2% Apr 2026 Official leaderboard
14 Gemini 3.5 Flash 33.6% May 2026 Official leaderboard
15 GPT-5.6 Sol 33.6% Jul 2026 Official leaderboard
16 Grok 4.6 33.2% Aug 2026 Official leaderboard
17 Claude Haiku 5.5 33% Oct 2026 Official leaderboard
18 Grok 4.7 32.5% Sep 2026 Official leaderboard
19 Muse Wonder 31.7% Oct 2026 Official leaderboard
20 Gemini 3.6 Flash 31.2% Jul 2026 Official leaderboard
21 GPT-6 Luna 31.2% Sep 2026 Official leaderboard
22 GPT-5.6 Terra 30.8% Jul 2026 Official leaderboard
23 Claude Opus 5 30.4% Jul 2026 Official leaderboard
24 Kimi K3 29.5% Jul 2026 Official leaderboard
25 Grok 4.5 27.3% Jul 2026 Official leaderboard
26 GPT-5.4 27.1% Mar 2026 Official leaderboard
27 Gemini 3.1 Pro 26.5% Feb 2026 Official leaderboard
28 Claude Sonnet 5 24.9% Jun 2026 Official leaderboard
29 Claude Opus 4.7 24.5% Apr 2026 Official leaderboard
30 GPT-5.6 Luna 22.6% Jul 2026 Official leaderboard
31 Claude Opus 4.8 14.5% May 2026 Official leaderboard
32 Claude Sonnet 4.6 6.70% Feb 2026 Official leaderboard
33 Kimi K2.6 3.90% Apr 2026 Official leaderboard
34 Claude Haiku 4.5 0% Oct 2025 Official leaderboard
35 Gemini 3 Flash 0% Dec 2025 Official leaderboard
36 Grok 4.20 0% reasoning Feb 2026 Official leaderboard
37 Gemini Robotics-ER 1.6 0% Apr 2026 Official leaderboard
38 Grok 4.3 0% Apr 2026 Official leaderboard

Labs report scores for 3 models in their cards. We plot every source in the charts below.

7 models have held number one on the official leaderboard since its release in May 2026

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

OpenAI Anthropic Google DeepMind
Own leaderboard 7 leaders
Gemini 4 Carbon 58.6%
Lab claims 3 leaders
Gemini 4 Carbon 58.6%

Sources: Official Blueprint-Bench 2 leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Gemini 4 Carbon Gemini 4 Argon GPT-6 Astra Claude Fable 5.1 GPT-5.5 GPT-5.4 Claude Sonnet 4.6 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Gemini 4 Carbon Claude Opus 5.5

Sources: Official Blueprint-Bench 2 leaderboard and labs’ model cards

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
Anthropic
DateLaunch postScoreAs the post writes it
Oct 28, 2026 Gemini 4 Carbon 58.6 Blueprint-Bench 2 (Similarity score)
Jun 9, 2026 Claude Fable 5 38.6 Spatial reasoning Blueprint-Bench 2
May 19, 2026 Gemini 3.5 Flash 33.6 Blueprint-Bench 2 (Normalized score)

2 system cards report it, from 2 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Jun 9, 2026 Claude Fable 5 system card 38.6 Blueprint-Bench 2, normalized composite score on 50 apartments, run by Andon Labs, max effort with coding tools (Table 8.1.A, 8.16.2) Capabilities
May 19, 2026 Gemini 3.5 Flash model card 33.6 Blueprint-Bench 2, normalized score (results table, p.4) Capabilities

3 lab claims next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 3 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Gemini 3.5 Flash 33.6% Blueprint-Bench 2 (Normalized score) 33.6% Leaderboard Same
Claude Fable 5 38.6% Spatial reasoning Blueprint-Bench 2 38.6% Leaderboard Same
Gemini 4 Carbon 58.6% Blueprint-Bench 2 (Similarity score) 58.6% Leaderboard Same

Versions

Replaces Blueprint-Bench

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%