Benchmark Almanac

Web Context (Nimble)

GPT-5.6 Sol 80% leads since October 2026
Not in model cards

Agents with only web search and page reading enrich company records, find matching entities or write research reports. LLM judges check each fact against source pages.

90 tasks across go-to-market, finance and social media. Scored by fact F1. 11 models in the Fireworks index. No maker page with details found.

Made by
Nimble
Released
October 2026
Kind
Deep research
Level
Expert
Scored
Graded by a model
Answers
Agent task
Tasks
Kept private
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
September 2026
Scores refreshed
November 12th, 2026
11
Models scored on its own leaderboard

GPT-5.6 Sol leads with 80% among models released by September 2026

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 80.0 GPT-5.6 Sol
  2. 77.3 Qwen 3.8 Max
  3. 76.5 GPT-6 Sol
  4. 73.5 DeepSeek V4.1 Flash
  5. 72.6 GPT-6 Astra
  6. 71.8 Claude Opus 5.5
  7. 69.8 GLM-5.3
  8. 69.4 Kimi K3
  9. 67.9 GPT-6 Luna
  10. 67.2 Claude Sonnet 5.5
  11. 64.2 GLM-5.3-Flash
  1. 1 GPT-5.6 Sol high 80%
  2. 2 Qwen 3.8 Max high 77.3%
  3. 3 GPT-6 Sol high 76.5%
  4. 4 DeepSeek V4.1 Flash high 73.5%
  5. 5 GPT-6 Astra high 72.6%
  6. 6 Claude Opus 5.5 high 71.8%
  7. 7 GLM-5.3 high 69.8%
  8. 8 Kimi K3 high 69.4%
  9. 9 GPT-6 Luna high 67.9%
  10. 10 Claude Sonnet 5.5 high 67.2%
  11. 11 GLM-5.3-Flash high 64.2%

Source: Official Web Context (Nimble) leaderboard

#ModelScoreSettingDateSource
1 GPT-5.6 Sol 80% high Jul 2026 Official leaderboard
2 Qwen 3.8 Max 77.3% high Aug 2026 Official leaderboard
3 GPT-6 Sol 76.5% high Sep 2026 Official leaderboard
4 DeepSeek V4.1 Flash 73.5% high Sep 2026 Official leaderboard
5 GPT-6 Astra 72.6% high Sep 2026 Official leaderboard
6 Claude Opus 5.5 71.8% high Sep 2026 Official leaderboard
7 GLM-5.3 69.8% high Aug 2026 Official leaderboard
8 Kimi K3 69.4% high Jul 2026 Official leaderboard
9 GPT-6 Luna 67.9% high Sep 2026 Official leaderboard
10 Claude Sonnet 5.5 67.2% high Sep 2026 Official leaderboard
11 GLM-5.3-Flash 64.2% high Aug 2026 Official leaderboard

Score vs. cost per task

The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

60% 70% 80% 90% 100% $0$2$4$6$8$10 Cost per task ($), lower is better Score Kimi K3 (high): Score 69.4%, cost per task $0.85 GLM-5.3 (high): Score 69.8%, cost per task $0.44 GPT-6 Astra (high): Score 72.6%, cost per task $7.81 Claude Opus 5.5 (high): Score 71.8%, cost per task $3.04 Claude Sonnet 5.5 (high): Score 67.2%, cost per task $0.66 GPT-5.6 Sol (high): Score 80%, cost per task $9.96 Qwen 3.8 Max (high): Score 77.3%, cost per task $8.02 GLM-5.3-Flash (high): Score 64.2%, cost per task $0.02 DeepSeek V4.1 Flash (high): Score 73.5%, cost per task $0.14 GPT-6 Luna (high): Score 67.9%, cost per task $0.07 GPT-6 Sol (high): Score 76.5%, cost per task $3.52 GPT-5.6 Sol (high)Qwen 3.8 Max (high)GPT-6 Sol (high)DeepSeek V4.1 Flash (high)GPT-6 Luna (high)GLM-5.3-Flash (high) 60% 70% 80% 90% 100% $0$2$4$6$8$10 Cost per task ($), lower is better Score Kimi K3 (high): Score 69.4%, cost per task $0.85 GLM-5.3 (high): Score 69.8%, cost per task $0.44 GPT-6 Astra (high): Score 72.6%, cost per task $7.81 Claude Opus 5.5 (high): Score 71.8%, cost per task $3.04 Claude Sonnet 5.5 (high): Score 67.2%, cost per task $0.66 GPT-5.6 Sol (high): Score 80%, cost per task $9.96 Qwen 3.8 Max (high): Score 77.3%, cost per task $8.02 GLM-5.3-Flash (high): Score 64.2%, cost per task $0.02 DeepSeek V4.1 Flash (high): Score 73.5%, cost per task $0.14 GPT-6 Luna (high): Score 67.9%, cost per task $0.07 GPT-6 Sol (high): Score 76.5%, cost per task $3.52 GPT-5.6 Sol (high)Qwen 3.8 Max (high)GPT-6 Sol (high)DeepSeek V4.1 Flash (high)GPT-6 Luna (high)GLM-5.3-Flash (high)
ModelSettingScoreCost per task
GPT-5.6 Sol high 80% $9.96
Qwen 3.8 Max high 77.3% $8.02
GPT-6 Sol high 76.5% $3.52
DeepSeek V4.1 Flash high 73.5% $0.14
GPT-6 Astra high 72.6% $7.81
Claude Opus 5.5 high 71.8% $3.04
GLM-5.3 high 69.8% $0.44
Kimi K3 high 69.4% $0.85
GPT-6 Luna high 67.9% $0.07
Claude Sonnet 5.5 high 67.2% $0.66
GLM-5.3-Flash high 64.2% $0.02

The best score, 80%, costs $9.96 a task

Runs on the cost frontier score higher than any run that's cheaper. GPT-6 Sol at high comes within 3.5 points of the best for $3.52 a task, 3 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Qwen Z.ai DeepSeek Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-5.6 Sol, high GPT-6 Sol, high DeepSeek V4.1 Flash, high GPT-6 Luna, high 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-5.6 Sol, high DeepSeek V4.1 Flash, high

Source: Official Web Context (Nimble) leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-5.6 Sol high 80% $9.96
Qwen 3.8 Max high 77.3% $8.02
GPT-6 Sol high 76.5% $3.52
DeepSeek V4.1 Flash high 73.5% $0.14
GPT-6 Astra high 72.6% $7.81
Claude Opus 5.5 high 71.8% $3.04
GLM-5.3 high 69.8% $0.44
Kimi K3 high 69.4% $0.85
GPT-6 Luna high 67.9% $0.07
Claude Sonnet 5.5 high 67.2% $0.66
GLM-5.3-Flash high 64.2% $0.02

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard 1 leader
GPT-5.6 Sol 80%

Source: Official Web Context (Nimble) leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0% 20% 40% 60% 80% 100% Near the ceiling (85%) GPT-5.6 Sol 0% 20% 40% 60% 80% 100% Near the ceiling (85%) GPT-5.6 Sol

Source: Official Web Context (Nimble) leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Tool use

All 144 in Tool use
  • AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state. Gemini 4 Carbon 53.9%
  • MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. Gemini 4 Argon 90.7%
  • BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes. Qwen3.7-Max 75%
  • τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. Qwen 3.8 Max 55.1%
  • Toolathlon-Verified In use 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. GLM-5.3-Flash 78.4%
  • DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%