Benchmark Almanac

ThinkGeo

GPT-6 Sol 100% leads since September 2026
Not in model cards

Agents use remote sensing tools to interpret aerial and satellite images. The authors compare individual tool calls and use GPT-4o mini to grade final answers.

The authors report 486 tasks, 14 executable tools, and 1,778 expert-verified reasoning steps.

Released
May 2025
Kind
Apps and MCP servers
Level
University
Scored
Graded by a model
Answers
Agent task
Tasks
All public
Measures
Capability
Language
English
Interaction
Agentic
Environment
APIs and tools
Refresh
One fixed set
Run by
Anyone
In views
Agents, Vision
Strongest model tested
Claude Opus 5.5 #5 on the capability index
Last model added
October 2026
Scores refreshed
November 20th, 2026
17
Models scored on Axis Spatial’s leaderboard

GPT-6 Sol leads with 100%

Best score per model from Axis Spatial’s leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

  1. 1 GPT-6 Sol medium 100%
  2. 2 GPT-6.1 Sol xhigh 88%
  3. 3 Claude Opus 5.5 max 86%
  4. 4 Claude Fable 5.1 high 83%
  5. 5 Claude Sonnet 5.5 xhigh 81%
  6. 6 GLM-5.3-Flash max 77%
  7. 7 GPT-5.6 Sol high 74%
  8. 8 Claude Opus 5 high 70%
  9. 9 DeepSeek V4.1 Flash max 70%
  10. 10 Claude Haiku 5.5 max 70%
  11. 11 Claude Sonnet 5 high 67%
  12. 12 GPT-6 Luna xhigh 64%
  13. 13 GPT-5.6 Luna medium 62%
  14. 14 GPT-5.6 Terra high 62%
  15. 15 Claude Opus 4.7 high 61%
  16. 16 Claude Opus 4.8 high 61%
  17. 17 Claude Haiku 4.5 Reasoning 14%

Source: Axis Spatial’s leaderboard

#ModelScoreSettingDateSource
1 GPT-6 Sol 100% medium, Codex Sep 2026 Official leaderboard
2 GPT-6.1 Sol 88% xhigh, Codex Sep 2026 Official leaderboard
3 Claude Opus 5.5 86% max, Claude Code Sep 2026 Official leaderboard
4 Claude Fable 5.1 83% high, Claude Code Sep 2026 Official leaderboard
5 Claude Sonnet 5.5 81% xhigh, Claude Code Sep 2026 Official leaderboard
6 GLM-5.3-Flash 77% max, Claude Code Aug 2026 Official leaderboard
7 GPT-5.6 Sol 74% high, Codex Jul 2026 Official leaderboard
8 Claude Opus 5 70% high, Claude Code Jul 2026 Official leaderboard
9 DeepSeek V4.1 Flash 70% max, Codex Sep 2026 Official leaderboard
10 Claude Haiku 5.5 70% max, Claude Code Oct 2026 Official leaderboard
11 Claude Sonnet 5 67% high, Claude Code Jun 2026 Official leaderboard
12 GPT-6 Luna 64% xhigh, Codex Sep 2026 Official leaderboard
13 GPT-5.6 Luna 62% medium, Codex Jul 2026 Official leaderboard
14 GPT-5.6 Terra 62% high, Codex Jul 2026 Official leaderboard
15 Claude Opus 4.7 61% high, Claude Code Apr 2026 Official leaderboard
16 Claude Opus 4.8 61% high, Claude Code May 2026 Official leaderboard
17 Claude Haiku 4.5 14% Reasoning, Claude Code Oct 2025 Official leaderboard

The best score, 100%, costs $0.03 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-6 Sol, medium Claude Haiku 5.5, xhigh Claude Haiku 5.5, low GPT-6 Luna, medium GPT-6 Luna, low 0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-6 Sol, medium Claude Haiku 5.5, xhigh

Source: Axis Spatial’s leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-6 Sol medium Codex 100% $0.03 674
GPT-6.1 Sol xhigh Codex 88% $0.06 2,000
GPT-6.1 Sol medium Codex 87% $0.03 761
Claude Opus 5.5 max Claude Code 86% $2.26 46,000
Claude Fable 5.1 high Claude Code 83% $0.45 3,000
Claude Opus 5.5 xhigh Claude Code 83% $0.30 5,000
Claude Sonnet 5.5 xhigh Claude Code 81% $0.10 3,000
GPT-6.1 Sol high Codex 81% $0.04 1,000
Claude Opus 5.5 high Claude Code 80% $0.12 1,000
Claude Fable 5.1 medium Claude Code 79% $0.29 2,000
Claude Opus 5.5 medium Claude Code 78% $0.09 1,000
GLM-5.3-Flash max Claude Code 77% $0.35 69,000
Claude Sonnet 5.5 max Claude Code 77% $1.19 53,000
GPT-5.6 Sol high Codex 74% $0.22 3,000
Claude Opus 5.5 low Claude Code 72% $0.07 550
GPT-6 Sol high Codex 72% $0.06 2,000
Claude Opus 5 high Claude Code 70% $1.18 12,000
DeepSeek V4.1 Flash max Codex 70% $0.26 75,000
Claude Sonnet 5.5 medium Claude Code 70% $0.04 606
Claude Haiku 5.5 max Claude Code 70% $0.12 53,000
Claude Sonnet 5 high Claude Code 67% $0.65 17,000
Claude Haiku 5.5 xhigh Claude Code 67% $0.02 14,000
Claude Sonnet 5.5 high Claude Code 66% $0.04 947
Claude Sonnet 5.5 low Claude Code 66% $0.03 542
GPT-6 Luna xhigh Codex 64% $0.006 5,000
Claude Haiku 5.5 low Claude Code 64% $0.004 2,000
GPT-5.6 Luna medium Codex 62% $0.009 2,000
GPT-5.6 Luna xhigh Codex 62% $0.03 10,000
GPT-5.6 Terra high Codex 62% $0.13 3,000
Claude Opus 4.7 high Claude Code 61% $1.43 14,000
Claude Opus 4.8 high Claude Code 61% $0.65 9,000
GPT-6 Luna high Codex 60% $0.003 2,000
GPT-6 Luna medium Codex 60% $0.002 775
Claude Haiku 5.5 medium Claude Code 58% $0.005 3,000
GPT-5.6 Luna high Codex 57% $0.02 6,000
GPT-5.6 Luna low Codex 57% $0.005 1,000
Claude Haiku 5.5 high Claude Code 55% $0.009 6,000
GPT-6 Luna low Codex 23% $0.002 500
Claude Haiku 4.5 Reasoning Claude Code 14% $0.14 12,000

7 models have held number one on Axis Spatial’s leaderboard since its release in May 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic OpenAI Z.ai
Axis Spatial leaderboard Passed 85% in Sep 2026, by GPT-6 Sol
GPT-6 Sol 100%

Source: Axis Spatial’s leaderboard

GPT-6 Sol on the official leaderboard passed 85% in September 2026

That was 16 months after its release. By the same age, 26% of the benchmarks from 2025 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On Axis Spatial’s leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-6 Sol Claude Fable 5.1 GLM-5.3-Flash Claude Sonnet 5 Claude Opus 4.7 Claude Haiku 4.5 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-6 Sol Claude Fable 5.1

Source: Axis Spatial’s leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Tool use

All 265 in Tool use
  • AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state. Gemini 4 Neon 56.3%
  • MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. Gemini 4 Argon 90.7%
  • BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes. Qwen3.7-Max 75%
  • τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. Qwen 3.8 Max 55.1%
  • Toolathlon-Verified In use 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. GLM-5.3-Flash 78.4%
  • DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (17 entries).
  • Pass Models scored 17 models we track.
  • Pass Newest score The newest is from October 2026, for Claude Haiku 5.5.
  • Pass Frontier models 4 of the top 10 have a score, led by Claude Opus 5.5 (#5).
  • Note Headroom Saturated. The best is 100%, past 85% since September 2026.
Leaderboard import 3 pass
  • Pass Importer We read the board on Axis Spatial: 39 rows for models we track at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check