ThinkGeo
Agents use remote sensing tools to interpret aerial and satellite images. The authors compare individual tool calls and use GPT-4o mini to grade final answers.
The authors report 486 tasks, 14 executable tools, and 1,778 expert-verified reasoning steps.
- Released
- May 2025
- Kind
- Apps and MCP servers
- Level
- University
- Scored
- Graded by a model
- Answers
- Agent task
- Tasks
- All public
- Measures
- Capability
- Subject
- Earth and environmental sciences
- Language
- English
- In and out
- Text, Images, Maps and geodata → Actions, Text
- Interaction
- Agentic
- Environment
- APIs and tools
- Items from
- Real work and use, Written by experts
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
- Claude Opus 5.5 #5 on the capability index
- Last model added
- October 2026
- Scores refreshed
- November 20th, 2026
GPT-6 Sol leads with 100%
Best score per model from Axis Spatial’s leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.
- 100.0 GPT-6 Sol
- 88.0 GPT-6.1 Sol
- 86.0 Claude Opus 5.5
- 83.0 Claude Fable 5.1
- 81.0 Claude Sonnet 5.5
- 77.0 GLM-5.3-Flash
- 74.0 GPT-5.6 Sol
- 70.0 Claude Opus 5
- 70.0
DeepSeek V4.1 Flash
- 70.0 Claude Haiku 5.5
- 67.0 Claude Sonnet 5
- 64.0 GPT-6 Luna
- 62.0 GPT-5.6 Luna
- 62.0 GPT-5.6 Terra
- 61.0 Claude Opus 4.7
- 61.0 Claude Opus 4.8
- 14.0 Claude Haiku 4.5
- 1 GPT-6 Sol medium 100%
- 2 GPT-6.1 Sol xhigh 88%
- 3 Claude Opus 5.5 max 86%
- 4 Claude Fable 5.1 high 83%
- 5 Claude Sonnet 5.5 xhigh 81%
- 6 GLM-5.3-Flash max 77%
- 7 GPT-5.6 Sol high 74%
- 8 Claude Opus 5 high 70%
- 9
DeepSeek V4.1 Flash max 70%
- 10 Claude Haiku 5.5 max 70%
- 11 Claude Sonnet 5 high 67%
- 12 GPT-6 Luna xhigh 64%
- 13 GPT-5.6 Luna medium 62%
- 14 GPT-5.6 Terra high 62%
- 15 Claude Opus 4.7 high 61%
- 16 Claude Opus 4.8 high 61%
- 17 Claude Haiku 4.5 Reasoning 14%
Source: Axis Spatial’s leaderboard
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Sol | 100% | medium, Codex | Sep 2026 | Official leaderboard |
| 2 | GPT-6.1 Sol | 88% | xhigh, Codex | Sep 2026 | Official leaderboard |
| 3 | Claude Opus 5.5 | 86% | max, Claude Code | Sep 2026 | Official leaderboard |
| 4 | Claude Fable 5.1 | 83% | high, Claude Code | Sep 2026 | Official leaderboard |
| 5 | Claude Sonnet 5.5 | 81% | xhigh, Claude Code | Sep 2026 | Official leaderboard |
| 6 | GLM-5.3-Flash | 77% | max, Claude Code | Aug 2026 | Official leaderboard |
| 7 | GPT-5.6 Sol | 74% | high, Codex | Jul 2026 | Official leaderboard |
| 8 | Claude Opus 5 | 70% | high, Claude Code | Jul 2026 | Official leaderboard |
| 9 | | 70% | max, Codex | Sep 2026 | Official leaderboard |
| 10 | Claude Haiku 5.5 | 70% | max, Claude Code | Oct 2026 | Official leaderboard |
| 11 | Claude Sonnet 5 | 67% | high, Claude Code | Jun 2026 | Official leaderboard |
| 12 | GPT-6 Luna | 64% | xhigh, Codex | Sep 2026 | Official leaderboard |
| 13 | GPT-5.6 Luna | 62% | medium, Codex | Jul 2026 | Official leaderboard |
| 14 | GPT-5.6 Terra | 62% | high, Codex | Jul 2026 | Official leaderboard |
| 15 | Claude Opus 4.7 | 61% | high, Claude Code | Apr 2026 | Official leaderboard |
| 16 | Claude Opus 4.8 | 61% | high, Claude Code | May 2026 | Official leaderboard |
| 17 | Claude Haiku 4.5 | 14% | Reasoning, Claude Code | Oct 2025 | Official leaderboard |
The best score, 100%, costs $0.03 a task
Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
Source: Axis Spatial’s leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| GPT-6 Sol | medium | Codex | 100% | $0.03 | 674 |
| GPT-6.1 Sol | xhigh | Codex | 88% | $0.06 | 2,000 |
| GPT-6.1 Sol | medium | Codex | 87% | $0.03 | 761 |
| Claude Opus 5.5 | max | Claude Code | 86% | $2.26 | 46,000 |
| Claude Fable 5.1 | high | Claude Code | 83% | $0.45 | 3,000 |
| Claude Opus 5.5 | xhigh | Claude Code | 83% | $0.30 | 5,000 |
| Claude Sonnet 5.5 | xhigh | Claude Code | 81% | $0.10 | 3,000 |
| GPT-6.1 Sol | high | Codex | 81% | $0.04 | 1,000 |
| Claude Opus 5.5 | high | Claude Code | 80% | $0.12 | 1,000 |
| Claude Fable 5.1 | medium | Claude Code | 79% | $0.29 | 2,000 |
| Claude Opus 5.5 | medium | Claude Code | 78% | $0.09 | 1,000 |
| GLM-5.3-Flash | max | Claude Code | 77% | $0.35 | 69,000 |
| Claude Sonnet 5.5 | max | Claude Code | 77% | $1.19 | 53,000 |
| GPT-5.6 Sol | high | Codex | 74% | $0.22 | 3,000 |
| Claude Opus 5.5 | low | Claude Code | 72% | $0.07 | 550 |
| GPT-6 Sol | high | Codex | 72% | $0.06 | 2,000 |
| Claude Opus 5 | high | Claude Code | 70% | $1.18 | 12,000 |
| max | Codex | 70% | $0.26 | 75,000 | |
| Claude Sonnet 5.5 | medium | Claude Code | 70% | $0.04 | 606 |
| Claude Haiku 5.5 | max | Claude Code | 70% | $0.12 | 53,000 |
| Claude Sonnet 5 | high | Claude Code | 67% | $0.65 | 17,000 |
| Claude Haiku 5.5 | xhigh | Claude Code | 67% | $0.02 | 14,000 |
| Claude Sonnet 5.5 | high | Claude Code | 66% | $0.04 | 947 |
| Claude Sonnet 5.5 | low | Claude Code | 66% | $0.03 | 542 |
| GPT-6 Luna | xhigh | Codex | 64% | $0.006 | 5,000 |
| Claude Haiku 5.5 | low | Claude Code | 64% | $0.004 | 2,000 |
| GPT-5.6 Luna | medium | Codex | 62% | $0.009 | 2,000 |
| GPT-5.6 Luna | xhigh | Codex | 62% | $0.03 | 10,000 |
| GPT-5.6 Terra | high | Codex | 62% | $0.13 | 3,000 |
| Claude Opus 4.7 | high | Claude Code | 61% | $1.43 | 14,000 |
| Claude Opus 4.8 | high | Claude Code | 61% | $0.65 | 9,000 |
| GPT-6 Luna | high | Codex | 60% | $0.003 | 2,000 |
| GPT-6 Luna | medium | Codex | 60% | $0.002 | 775 |
| Claude Haiku 5.5 | medium | Claude Code | 58% | $0.005 | 3,000 |
| GPT-5.6 Luna | high | Codex | 57% | $0.02 | 6,000 |
| GPT-5.6 Luna | low | Codex | 57% | $0.005 | 1,000 |
| Claude Haiku 5.5 | high | Claude Code | 55% | $0.009 | 6,000 |
| GPT-6 Luna | low | Codex | 23% | $0.002 | 500 |
| Claude Haiku 4.5 | Reasoning | Claude Code | 14% | $0.14 | 12,000 |
7 models have held number one on Axis Spatial’s leaderboard since its release in May 2025
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Axis Spatial’s leaderboard
GPT-6 Sol on the official leaderboard passed 85% in September 2026
That was 16 months after its release. By the same age, 26% of the benchmarks from 2025 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Axis Spatial’s leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Tool use
- AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state.
Gemini 4 Neon 56.3%
- MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks.
Gemini 4 Argon 90.7%
- BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes.
Qwen3.7-Max 75%
- τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state.
Qwen 3.8 Max 55.1%
- Toolathlon-Verified In use 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. GLM-5.3-Flash 78.4%
- DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%
Data audit
All 10 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (17 entries).
- Pass Models scored 17 models we track.
- Pass Newest score The newest is from October 2026, for Claude Haiku 5.5.
- Pass Frontier models 4 of the top 10 have a score, led by Claude Opus 5.5 (#5).
- Note Headroom Saturated. The best is 100%, past 85% since September 2026.
Leaderboard import 3 pass
- Pass Importer We read the board on Axis Spatial: 39 rows for models we track at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.