SMB Bench
An agent runs the buying desk of a simulated fresh-produce wholesaler for a 90-day quarter, using SQL records, supplier quotes and memory files that are its only carryover between days. Scored on mean net profit per quarter across 18 tasks, compared with a fixed buying rule.
Six businesses, each played in normal, customer and supplier variants where the cost of shortfalls arrives weeks later. The fixed rule earns $6,412 a quarter and a scripted expert buyer $10,795. The simulator stays on ORO's hosted service, which needs a grant, and the grader is gated. Runs and traces are public on Hugging Face.
- Made by
- ORO AI
- Released
- November 2026
- Kind
- Running a business
- Level
- University
- Scored
- Checked automatically
- Answers
- Agent task
- Tasks
- Kept private
- Measures
- Capability
- Subject
- Economics and business
- Language
- English
- Interaction
- Agentic
- Environment
- A simulator, APIs and tools
- Items from
- Generated
- Refresh
- One fixed set
- Run by
- A host
- Occupations
- Business and financial operations
- In views
- Agents, Robotics, Law and finance
- Strongest model tested
- Claude Fable 5.2 #2 on the capability index
- Last model added
- November 2026
- Scores refreshed
- November 20th, 2026
GPT-6 Astra leads with $7,749
Best score per model from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.
- $7,749 GPT-6 Astra
- $7,703
Gemini 4 Carbon
- $7,275 Claude Fable 5.2
- $5,427 Claude Fable 5.1
- $5,404 GPT-6.1 Sol
- $2,560
Gemini 4 Hydrogen
- $1,501
Gemini 4 Argon
- $907 Claude Opus 5.5
- $-3,036
DeepSeek V4.1 Flash
- $-4,722 Grok 4.7
- $-5,383
Muse Ember
- $-5,669
Muse Spark 1.3
- $-6,454 GLM-5.3
- $-7,440 Claude Haiku 5.5
- $-8,291
Muse Wonder
- $-8,667 GPT-6 Luna
- 1 GPT-6 Astra gpt-6-astra; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 $7,749
- 2
Gemini 4 Carbon gemini-4-carbon; thinking level high; 65,536; runtime/v1.11-carbon; 3437a5c1 / 575d06be; 5d582f7f; GeminiAdapter, 300 s idle timeout; 2026-11-10 to 2026-11-12 $7,703
- 3 Claude Fable 5.2 claude-fable-5-2; effort high, adaptive thinking; 65,536; runtime/v1.11-fable52; 95bb6628 / bda9748d; 5d582f7f; Anthropic Messages, adaptive thinking summarized, 300 s idle timeout; 2026-11-10 $7,275
- 4 Claude Fable 5.1 claude-fable-5-1; effort high, adaptive thinking; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 $5,427
- 5 GPT-6.1 Sol gpt-6.1-sol; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 $5,404
- 6
Gemini 4 Hydrogen gemini-4-hydrogen; thinking level high; 65,536; runtime/v1.11-hydrogen; d490ef8c / b3b3b2ed; 40031d67; GeminiAdapter, 300 s idle timeout; 2026-11-12 to 2026-11-13 $2,560
- 7
Gemini 4 Argon gemini-4-argon; thinking level high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 $1,501
- 8 Claude Opus 5.5 claude-opus-5-5; effort high, adaptive thinking; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 $907
- 9
DeepSeek V4.1 Flash deepseek-ai/DeepSeek-V4.1-Flash-1017-Fast; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 $-3,036
- 10 Grok 4.7 grok-4.7; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 $-4,722
- 11
Muse Ember muse-ember-1.0; reasoning effort high; 65,536; runtime/v1.11-ember; 1be19945 / f8506595; 5d582f7f; Responses via a gateway, effort high, 90 s idle timeout; 2026-11-09 $-5,383
- 12
Muse Spark 1.3 meta/muse-spark-1.3; reasoning effort high; 65,536; runtime/v1.11-spark; c2304c55 / f791742a; 5d582f7f; Responses via a gateway, effort high, summary auto, 90 s idle timeout; 2026-11-08 to 2026-11-09 $-5,669
- 13 GLM-5.3 zai-org/GLM-5.3; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 $-6,454
- 14 Claude Haiku 5.5 claude-haiku-5-5; effort high, adaptive thinking; 65,536; runtime/v1.11-haiku; 83e2e25d / 2a2fde45; 5d582f7f; Anthropic Messages, adaptive thinking summarized, 300 s idle timeout; 2026-11-09 to 2026-11-10 $-7,440
- 15
Muse Wonder meta/muse-wonder-1.0; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 $-8,291
- 16 GPT-6 Luna gpt-6-luna; reasoning effort high; 65,536; runtime/v1.11-luna; six reruns runtime/v1.11-luna(-null) rr1; ad7b46dc / c4127aed; reruns 65cc113 / 1d5bd29; 5d582f7f; reruns e2bb04c1; Responses, effort high, summary auto, 90 s idle timeout; 2026-11-09 to 2026-11-11 $-8,667
Source: Official SMB Bench leaderboard
| # | Model | Score | Setting | Agent by | Released | Run on | Link | Source |
|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | $7,749 | gpt-6-astra; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05, ORO buyer loop | ORO | Sep 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 2 | | $7,703 | gemini-4-carbon; thinking level high; 65,536; runtime/v1.11-carbon; 3437a5c1 / 575d06be; 5d582f7f; GeminiAdapter, 300 s idle timeout; 2026-11-10 to 2026-11-12, ORO buyer loop | ORO | Oct 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 3 | Claude Fable 5.2 | $7,275 | claude-fable-5-2; effort high, adaptive thinking; 65,536; runtime/v1.11-fable52; 95bb6628 / bda9748d; 5d582f7f; Anthropic Messages, adaptive thinking summarized, 300 s idle timeout; 2026-11-10, ORO buyer loop | ORO | Nov 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 4 | Claude Fable 5.1 | $5,427 | claude-fable-5-1; effort high, adaptive thinking; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05, ORO buyer loop | ORO | Sep 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 5 | GPT-6.1 Sol | $5,404 | gpt-6.1-sol; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05, ORO buyer loop | ORO | Sep 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 6 | | $2,560 | gemini-4-hydrogen; thinking level high; 65,536; runtime/v1.11-hydrogen; d490ef8c / b3b3b2ed; 40031d67; GeminiAdapter, 300 s idle timeout; 2026-11-12 to 2026-11-13, ORO buyer loop | ORO | Nov 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 7 | | $1,501 | gemini-4-argon; thinking level high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05, ORO buyer loop | ORO | Sep 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 8 | Claude Opus 5.5 | $907 | claude-opus-5-5; effort high, adaptive thinking; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05, ORO buyer loop | ORO | Sep 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 9 | | $-3,036 | deepseek-ai/DeepSeek-V4.1-Flash-1017-Fast; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05, ORO buyer loop | ORO | Sep 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 10 | Grok 4.7 | $-4,722 | grok-4.7; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05, ORO buyer loop | ORO | Sep 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 11 | | $-5,383 | muse-ember-1.0; reasoning effort high; 65,536; runtime/v1.11-ember; 1be19945 / f8506595; 5d582f7f; Responses via a gateway, effort high, 90 s idle timeout; 2026-11-09, ORO buyer loop | ORO | Nov 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 12 | | $-5,669 | meta/muse-spark-1.3; reasoning effort high; 65,536; runtime/v1.11-spark; c2304c55 / f791742a; 5d582f7f; Responses via a gateway, effort high, summary auto, 90 s idle timeout; 2026-11-08 to 2026-11-09, ORO buyer loop | ORO | Sep 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 13 | GLM-5.3 | $-6,454 | zai-org/GLM-5.3; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05, ORO buyer loop | ORO | Aug 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 14 | Claude Haiku 5.5 | $-7,440 | claude-haiku-5-5; effort high, adaptive thinking; 65,536; runtime/v1.11-haiku; 83e2e25d / 2a2fde45; 5d582f7f; Anthropic Messages, adaptive thinking summarized, 300 s idle timeout; 2026-11-09 to 2026-11-10, ORO buyer loop | ORO | Oct 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 15 | | $-8,291 | meta/muse-wonder-1.0; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05, ORO buyer loop | ORO | Oct 2026 | Nov 2026 | oroagents.com | Official leaderboard |
| 16 | GPT-6 Luna | $-8,667 | gpt-6-luna; reasoning effort high; 65,536; runtime/v1.11-luna; six reruns runtime/v1.11-luna(-null) rr1; ad7b46dc / c4127aed; reruns 65cc113 / 1d5bd29; 5d582f7f; reruns e2bb04c1; Responses, effort high, summary auto, 90 s idle timeout; 2026-11-09 to 2026-11-11, ORO buyer loop | ORO | Sep 2026 | Nov 2026 | oroagents.com | Official leaderboard |
Rankings by part
We read 21 parts from the benchmark’s own leaderboard in 2 groups: category and task type. Each ranks models on one part of the test. Claude Fable 5.2 leads 9 of them, and 3 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
Customer
16 models we track have a score here.
- 1
Gemini 4 Carbon ↑1 $9,706
- 2 GPT-6 Astra ↓1 $8,809
- 3 Claude Fable 5.2 $6,526
- 4 Claude Fable 5.1 $5,121
- 5 GPT-6.1 Sol $4,596
- 6
Gemini 4 Argon ↑1 $2,509
- 7
Gemini 4 Hydrogen ↓1 $2,374
- 8 Claude Opus 5.5 $1,185
- 9
DeepSeek V4.1 Flash $-6,051
- 10
Muse Ember ↑1 $-7,216
Supplier
16 models we track have a score here.
- 1 GPT-6 Astra $4,293
- 2
Gemini 4 Carbon $4,056
- 3 Claude Fable 5.2 $2,753
- 4 Claude Fable 5.1 $1,396
- 5 GPT-6.1 Sol $1,286
- 6
Gemini 4 Argon ↑1 $-2,602
- 7
Gemini 4 Hydrogen ↓1 $-4,028
- 8 Claude Opus 5.5 $-7,865
- 9
DeepSeek V4.1 Flash $-10,801
- 10 Grok 4.7 $-11,927
Normal
16 models we track have a score here.
- 1 Claude Fable 5.2 ↑2 $12,545
- 2 GPT-6.1 Sol ↑3 $10,331
- 3 GPT-6 Astra ↓2 $10,145
- 4 Claude Fable 5.1 $9,764
- 5 Claude Opus 5.5 ↑3 $9,401
- 6
Gemini 4 Carbon ↓4 $9,346
- 7
Gemini 4 Hydrogen ↓1 $9,336
- 8
DeepSeek V4.1 Flash ↑1 $7,745
- 9 Grok 4.7 ↑1 $5,711
- 10
Gemini 4 Argon ↓3 $4,597
Task 1
16 models we track have a score here.
- 1
Gemini 4 Carbon ↑1 $19,242
- 2 GPT-6 Astra ↓1 $16,653
- 3 GPT-6.1 Sol ↑2 $13,235
- 4 Claude Fable 5.1 $13,057
- 5 Claude Fable 5.2 ↓2 $13,000
- 6 Claude Opus 5.5 ↑2 $12,243
- 7
Gemini 4 Hydrogen ↓1 $11,313
- 8
Gemini 4 Argon ↓1 $11,026
- 9 Grok 4.7 ↑1 $-1,527
- 10
DeepSeek V4.1 Flash ↓1 $-1,536
Task 2
16 models we track have a score here.
- 1 Claude Fable 5.2 ↑2 $20,185
- 2 GPT-6 Astra ↓1 $18,840
- 3
Gemini 4 Hydrogen ↑3 $17,166
- 4 Claude Opus 5.5 ↑4 $16,519
- 5 GPT-6.1 Sol $16,425
- 6
Gemini 4 Carbon ↓4 $16,327
- 7 Claude Fable 5.1 ↓3 $16,042
- 8
DeepSeek V4.1 Flash ↑1 $15,128
- 9
Gemini 4 Argon ↓2 $14,666
- 10
Muse Spark 1.3 ↑2 $12,633
Task 3
16 models we track have a score here.
- 1 Claude Fable 5.2 ↑2 $12,636
- 2
Gemini 4 Carbon $12,103
- 3 GPT-6 Astra ↓2 $8,785
- 4 Claude Fable 5.1 $7,703
- 5
Gemini 4 Argon ↑2 $5,372
- 6
Gemini 4 Hydrogen $2,058
- 7 GPT-6.1 Sol ↓2 $-3,222
- 8 Claude Opus 5.5 $-4,805
- 9
DeepSeek V4.1 Flash $-7,312
- 10 Grok 4.7 $-8,920
Task 4
16 models we track have a score here.
- 1 GPT-6 Astra $2,746
- 2 GPT-6.1 Sol ↑3 $2,740
- 3
Gemini 4 Carbon ↓1 $2,698
- 4 Claude Fable 5.2 ↓1 $899
- 5 Claude Fable 5.1 ↓1 $296
- 6
Gemini 4 Argon ↑1 $221
- 7
Gemini 4 Hydrogen ↓1 $-1,166
- 8 Claude Opus 5.5 $-2,807
- 9
DeepSeek V4.1 Flash $-4,316
- 10
Muse Spark 1.3 ↑2 $-4,453
Task 5
16 models we track have a score here.
- 1 Claude Fable 5.2 ↑2 $4,012
- 2
Gemini 4 Hydrogen ↑4 $3,459
- 3 GPT-6.1 Sol ↑2 $3,386
- 4 GPT-6 Astra ↓3 $3,345
- 5
Gemini 4 Carbon ↓3 $2,813
- 6 Claude Opus 5.5 ↑2 $2,375
- 7
Gemini 4 Argon $2,330
- 8
DeepSeek V4.1 Flash ↑1 $1,854
- 9 Claude Fable 5.1 ↓5 $1,391
- 10 Grok 4.7 $1,368
Task 6
16 models we track have a score here.
- 1
Gemini 4 Carbon ↑1 $-619
- 2 GPT-6 Astra ↓1 $-878
- 3
Gemini 4 Hydrogen ↑3 $-1,075
- 4 GPT-6.1 Sol ↑1 $-1,985
- 5 Claude Fable 5.1 ↓1 $-2,542
- 6 Claude Fable 5.2 ↓3 $-2,568
- 7 Claude Opus 5.5 ↑1 $-7,468
- 8
Gemini 4 Argon ↓1 $-8,231
- 9
DeepSeek V4.1 Flash $-8,274
- 10 Claude Haiku 5.5 ↑4 $-8,359
Task 7
16 models we track have a score here.
- 1 GPT-6 Astra $5,279
- 2
Gemini 4 Carbon $4,787
- 3
Gemini 4 Hydrogen ↑3 $3,489
- 4 Claude Fable 5.2 ↓1 $3,170
- 5
Gemini 4 Argon ↑2 $3,031
- 6 GPT-6.1 Sol ↓1 $2,919
- 7 Claude Fable 5.1 ↓3 $1,761
- 8 Claude Opus 5.5 $-2,860
- 9
Muse Spark 1.3 ↑3 $-3,526
- 10
Muse Ember ↑1 $-5,460
Task 8
16 models we track have a score here.
- 1 GPT-6 Astra $5,997
- 2 Claude Fable 5.2 ↑1 $5,997
- 3 GPT-6.1 Sol ↑2 $5,796
- 4
Gemini 4 Hydrogen ↑2 $5,508
- 5
Gemini 4 Carbon ↓3 $5,102
- 6 Claude Opus 5.5 ↑2 $4,269
- 7
DeepSeek V4.1 Flash ↑2 $4,134
- 8 Claude Fable 5.1 ↓4 $3,759
- 9 Grok 4.7 ↑1 $3,500
- 10
Gemini 4 Argon ↓3 $3,393
Task 9
16 models we track have a score here.
- 1 GPT-6 Astra $3,361
- 2
Gemini 4 Carbon $3,237
- 3 GPT-6.1 Sol ↑2 $709
- 4 Claude Fable 5.2 ↓1 $414
- 5 Claude Fable 5.1 ↓1 $-231
- 6
Gemini 4 Argon ↑1 $-1,938
- 7
Muse Ember ↑4 $-5,509
- 8 Claude Opus 5.5 $-6,827
- 9
Gemini 4 Hydrogen ↓3 $-6,922
- 10 GLM-5.3 ↑3 $-7,872
Task 10
16 models we track have a score here.
- 1
Gemini 4 Carbon ↑1 $7,650
- 2 Claude Fable 5.2 ↑1 $7,494
- 3 Claude Fable 5.1 ↑1 $7,350
- 4 GPT-6 Astra ↓3 $6,226
- 5 GPT-6.1 Sol $6,103
- 6
Gemini 4 Argon ↑1 $5,023
- 7
Gemini 4 Hydrogen ↓1 $3,269
- 8 Claude Opus 5.5 $-587
- 9
DeepSeek V4.1 Flash $-3,380
- 10
Muse Ember ↑1 $-3,801
Task 11
16 models we track have a score here.
- 1 Claude Fable 5.2 ↑2 $11,410
- 2 Claude Fable 5.1 ↑2 $10,146
- 3
Gemini 4 Carbon ↓1 $9,061
- 4 GPT-6 Astra ↓3 $8,221
- 5 GPT-6.1 Sol $7,559
- 6
Gemini 4 Hydrogen $7,360
- 7
DeepSeek V4.1 Flash ↑2 $6,568
- 8 Claude Opus 5.5 $6,311
- 9 Claude Haiku 5.5 ↑5 $4,613
- 10 Grok 4.7 $4,498
Task 12
16 models we track have a score here.
- 1 Claude Fable 5.2 ↑2 $4,111
- 2 Claude Fable 5.1 ↑2 $3,803
- 3
Gemini 4 Carbon ↓1 $3,261
- 4 GPT-6 Astra ↓3 $2,964
- 5
Gemini 4 Hydrogen ↑1 $286
- 6 GPT-6.1 Sol ↓1 $-633
- 7
Gemini 4 Argon $-1,345
- 8
DeepSeek V4.1 Flash ↑1 $-8,485
- 9 GLM-5.3 ↑4 $-9,450
- 10 Grok 4.7 $-9,580
Task 13
16 models we track have a score here.
- 1
Gemini 4 Carbon ↑1 $21,501
- 2 GPT-6 Astra ↓1 $17,439
- 3 Claude Fable 5.2 $14,177
- 4 Claude Fable 5.1 $9,045
- 5 GPT-6.1 Sol $5,458
- 6
Gemini 4 Hydrogen $2,310
- 7 Claude Opus 5.5 ↑1 $899
- 8
Gemini 4 Argon ↓1 $-2,436
- 9 Claude Haiku 5.5 ↑5 $-12,031
- 10
DeepSeek V4.1 Flash ↓1 $-14,158
Task 14
16 models we track have a score here.
- 1 Claude Fable 5.2 ↑2 $27,077
- 2 Claude Fable 5.1 ↑2 $23,802
- 3 GPT-6.1 Sol ↑2 $23,563
- 4 Claude Opus 5.5 ↑4 $23,043
- 5
Gemini 4 Carbon ↓3 $19,625
- 6 GPT-6 Astra ↓5 $19,247
- 7
Gemini 4 Hydrogen ↓1 $19,017
- 8
DeepSeek V4.1 Flash ↑1 $16,646
- 9 Grok 4.7 ↑1 $10,442
- 10 GLM-5.3 ↑3 $9,929
Task 15
16 models we track have a score here.
- 1 GPT-6.1 Sol ↑4 $18,057
- 2 GPT-6 Astra ↓1 $15,348
- 3
Gemini 4 Carbon ↓1 $8,074
- 4 Claude Fable 5.2 ↓1 $3,439
- 5 Claude Fable 5.1 ↓1 $3,325
- 6
Gemini 4 Argon ↑1 $-2,736
- 7 Claude Opus 5.5 ↑1 $-6,622
- 8
Gemini 4 Hydrogen ↓2 $-11,519
- 9
DeepSeek V4.1 Flash $-19,692
- 10 Grok 4.7 $-20,186
Task 16
16 models we track have a score here.
- 1 GPT-6 Astra $4,510
- 2
Gemini 4 Carbon $2,360
- 3 Claude Fable 5.2 $415
- 4 Claude Opus 5.5 ↑4 $224
- 5 Claude Fable 5.1 ↓1 $-782
- 6
Gemini 4 Argon ↑1 $-1,810
- 7 GPT-6.1 Sol ↓2 $-2,881
- 8
Gemini 4 Hydrogen ↓2 $-4,974
- 9
Muse Ember ↑2 $-5,449
- 10
DeepSeek V4.1 Flash ↓1 $-6,664
Task 17
16 models we track have a score here.
- 1 Claude Fable 5.2 ↑2 $6,591
- 2 GPT-6.1 Sol ↑3 $5,259
- 3 GPT-6 Astra ↓2 $5,218
- 4 Claude Opus 5.5 ↑4 $3,888
- 5
Gemini 4 Hydrogen ↑1 $3,504
- 6 Claude Fable 5.1 ↓2 $3,446
- 7
Gemini 4 Carbon ↓5 $3,146
- 8
Muse Spark 1.3 ↑4 $2,810
- 9
DeepSeek V4.1 Flash $2,137
- 10 Grok 4.7 $1,876
Task 18
16 models we track have a score here.
- 1 Claude Fable 5.2 ↑2 $-1,512
- 2
Gemini 4 Carbon $-1,721
- 3 Claude Fable 5.1 ↑1 $-3,680
- 4 GPT-6 Astra ↓3 $-3,819
- 5 GPT-6.1 Sol $-5,212
- 6
Gemini 4 Argon ↑1 $-6,735
- 7
Gemini 4 Hydrogen ↓1 $-6,996
- 8
Muse Spark 1.3 ↑4 $-9,364
- 9 Claude Opus 5.5 ↓1 $-9,739
- 10 Claude Haiku 5.5 ↑4 $-10,533
The best score, $7,749, costs $213 a task
Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
Source: Official SMB Bench leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| GPT-6 Astra | gpt-6-astra; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 | ORO buyer loop | $7,749 | $213 | |
| gemini-4-carbon; thinking level high; 65,536; runtime/v1.11-carbon; 3437a5c1 / 575d06be; 5d582f7f; GeminiAdapter, 300 s idle timeout; 2026-11-10 to 2026-11-12 | ORO buyer loop | $7,703 | $134 | ||
| Claude Fable 5.2 | claude-fable-5-2; effort high, adaptive thinking; 65,536; runtime/v1.11-fable52; 95bb6628 / bda9748d; 5d582f7f; Anthropic Messages, adaptive thinking summarized, 300 s idle timeout; 2026-11-10 | ORO buyer loop | $7,275 | $94 | |
| Claude Fable 5.1 | claude-fable-5-1; effort high, adaptive thinking; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 | ORO buyer loop | $5,427 | $106 | |
| GPT-6.1 Sol | gpt-6.1-sol; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 | ORO buyer loop | $5,404 | $32 | |
| gemini-4-hydrogen; thinking level high; 65,536; runtime/v1.11-hydrogen; d490ef8c / b3b3b2ed; 40031d67; GeminiAdapter, 300 s idle timeout; 2026-11-12 to 2026-11-13 | ORO buyer loop | $2,560 | $25 | ||
| gemini-4-argon; thinking level high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 | ORO buyer loop | $1,501 | $70 | ||
| Claude Opus 5.5 | claude-opus-5-5; effort high, adaptive thinking; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 | ORO buyer loop | $907 | $23 | |
| deepseek-ai/DeepSeek-V4.1-Flash-1017-Fast; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 | ORO buyer loop | $-3,036 | $34 | ||
| Grok 4.7 | grok-4.7; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 | ORO buyer loop | $-4,722 | $63 | |
| muse-ember-1.0; reasoning effort high; 65,536; runtime/v1.11-ember; 1be19945 / f8506595; 5d582f7f; Responses via a gateway, effort high, 90 s idle timeout; 2026-11-09 | ORO buyer loop | $-5,383 | $0.65 | ||
| meta/muse-spark-1.3; reasoning effort high; 65,536; runtime/v1.11-spark; c2304c55 / f791742a; 5d582f7f; Responses via a gateway, effort high, summary auto, 90 s idle timeout; 2026-11-08 to 2026-11-09 | ORO buyer loop | $-5,669 | $31 | ||
| GLM-5.3 | zai-org/GLM-5.3; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 | ORO buyer loop | $-6,454 | $29 | |
| Claude Haiku 5.5 | claude-haiku-5-5; effort high, adaptive thinking; 65,536; runtime/v1.11-haiku; 83e2e25d / 2a2fde45; 5d582f7f; Anthropic Messages, adaptive thinking summarized, 300 s idle timeout; 2026-11-09 to 2026-11-10 | ORO buyer loop | $-7,440 | $1.26 | |
| meta/muse-wonder-1.0; reasoning effort high; 65,536; campaign build, source revision b17b864b; per the release; 02a1d1d8; per model, see Models; 2026-11-03 to 2026-11-05 | ORO buyer loop | $-8,291 | $42 | ||
| GPT-6 Luna | gpt-6-luna; reasoning effort high; 65,536; runtime/v1.11-luna; six reruns runtime/v1.11-luna(-null) rr1; ad7b46dc / c4127aed; reruns 65cc113 / 1d5bd29; 5d582f7f; reruns e2bb04c1; Responses, effort high, summary auto, 90 s idle timeout; 2026-11-09 to 2026-11-11 | ORO buyer loop | $-8,667 | $2.09 |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official SMB Bench leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official SMB Bench leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Economic work
- JobBench In use 130 tasks from 35 occupations workers most want delegated. Agent works through messy reference files, graded by fact-anchored binary rubrics. GLM-5.3 61.4%
- GDPval In use Real work tasks from 44 occupations with reference files. Experts blindly compare model deliverables with expert work. Metric is win-or-tie rate.
Seed2.1 Pro 87.9%
- GDPval-AA v2.1 In use Agents with shell and web access complete 220 GDPval gold tasks from 44 occupations. Blind pairwise comparisons give Elo ratings. Claude Opus 5.5 1,846
- AA-Briefcase v1.1 In use Version 1.1 of Artificial Analysis's multi-hour office work evaluation, with Elo ratings fitted by Crowd-BT. Claude Opus 5.5 1,822
- Agents' Last Exam v1.1 In use Agents complete 122 public long-horizon professional workflow tasks across 55 sub-industries. Deterministic graders check outcomes against hidden references. GPT-6.2 Astra 47.5%
- CoWorkBench In use Agent completes long-horizon productivity tasks across computer science, finance, law and medicine. Qwen in-house scoring (method unpublished). Claude Fable 5 75.9%
Data audit
All 10 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access The tasks are private.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (16 entries).
- Pass Models scored 16 models we track.
- Pass Newest score The newest is from November 2026, for Gemini 4 Hydrogen.
- Pass Frontier models 7 of the top 10 have a score, led by Claude Fable 5.2 (#2).
- Note Headroom Scored as US dollars, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on huggingface.co: 16 rows for models we track at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.