BTF-3
Agents forecast already-resolved binary and numeric questions while researching a frozen snapshot of the web. Scoring pools Brier and ranked probability scores.
BTF-3 adds numeric questions scored by ranked probability score next to binary ones. The question set keeps growing, from 1,783 questions on the dataset card to 2,386 on FutureSearch's page.
- Made by
- FutureSearch
- Released
- June 2026
- Kind
- Real-world events
- Level
- Expert
- Scored
- Settled by events
- Answers
- Prediction
- Tasks
- All public
- Strongest model tested
- GPT-6 Astra #3 on the capability index
- Last model added
- September 2026
- Scores refreshed
- November 12th, 2026
FutureSearch SOTA tops the leaderboard with 0.12
Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Claude Opus 5, at 0.12.
The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- Claude Fable 5.2
-
Gemini 4 Carbon - Claude Opus 5.5
- GPT-6.1 Sol
- and 13 more
- 1 FutureSearch SOTA 0.12
- 2 Claude Opus 5 xhigh 0.12
- 3 Claude Opus 4.8 xhigh 0.13
- 4 GPT-6 Astra low 0.13
- 5 GPT-5.6 Sol medium 0.14
- 6
Muse Spark 1.3 xhigh 0.14 - 7 Claude Sonnet 5 xhigh 0.14
- 8 GLM-5.3-Flash 0.15
- 9 GLM-5.3 0.15
Source: Official BTF-3 leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | FutureSearch SOTA | 0.12 | Official leaderboard | ||
| 2 | Claude Opus 5 | 0.12 | xhigh | Jul 2026 | Official leaderboard |
| 3 | Claude Opus 4.8 | 0.13 | xhigh | May 2026 | Official leaderboard |
| 4 | GPT-6 Astra | 0.13 | low | Sep 2026 | Official leaderboard |
| 5 | GPT-5.6 Sol | 0.14 | medium | Jul 2026 | Official leaderboard |
| 6 | Muse Spark 1.3 | 0.14 | xhigh | Sep 2026 | Official leaderboard |
| 7 | Claude Sonnet 5 | 0.14 | xhigh | Jun 2026 | Official leaderboard |
| 8 | GLM-5.3-Flash | 0.15 | Aug 2026 | Official leaderboard | |
| 9 | GLM-5.3 | 0.15 | Aug 2026 | Official leaderboard |
Pooled score vs. forecast cost
Score uses the full cohort. Cost is mean agent LLM spend per forecast on the August 2026 cohort. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.
| Model | Setting | Pooled score | Cost per forecast |
|---|---|---|---|
| Claude Opus 5 | xhigh | 0.12 | $0.66 |
| Claude Opus 4.8 | xhigh | 0.13 | $0.58 |
| GPT-6 Astra | low | 0.13 | $0.31 |
| GPT-5.6 Sol | medium | 0.14 | $0.34 |
| Muse Spark 1.3 | xhigh | 0.14 | $0.21 |
| Claude Sonnet 5 | xhigh | 0.14 | $0.25 |
| GLM-5.3-Flash | 0.15 | $0.05 | |
| GLM-5.3 | 0.15 | $0.13 |
2 models have held number one on the official leaderboard since its release in June 2026
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official BTF-3 leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official BTF-3 leaderboard
No lab model card reports it yet
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.
Versions
More in Forecasting
- ForecastBench In use Models give probabilities for 500 auto-generated questions per round about future events, drawn from prediction markets and time-series datasets. Forecasts are scored with difficulty-adjusted Brier scores once the questions resolve. GPT-5.5 0.11
- Prophet Arena In use Models and agents forecast open Kalshi prediction-market events and are scored once the events resolve. The live board ranks a difficulty-adjusted Arena score and also reports calibration error and market return. Claude Opus 5.5 86.2
- Echo General AI Prediction Leaderboard Not in model cards Models forecast unresolved events drawn from prediction markets, synthesized news questions and expert-written questions. Forecasts made at aligned times are compared in pairwise battles, and an Elo fit ranks the models.
DeepSeek-V4-Pro 1,017 - Human Behavior Bench Not in model cards Models predict how specified groups of U.S. adults answer closed survey questions, as a distribution over options. Scored as 1 minus total variation distance. Gemini 4 Argon 80.6
- FutureX Not in model cards Agents predict upcoming events collected daily from 195 websites in 11 domains, within a one-week window. Answers are scored once real results are crawled, with harder tiers weighted more. Grok 4 27.9
- KellyBench Not in model cards Agents bet through a simulated 2023 to 24 English Premier League season with historical data. The score is long-term bankroll growth. GPT-5.4 -7.90%