Benchmark Almanac

HorizonMath

GPT-5.6 Sol 11.5% leads since July 2026
GPT-5.6 Sol 11.5% leads since July 2026
In use

Models attack 113 mostly unsolved problems in eight areas of mathematics and the mathematical sciences. A program checks each proposed closed form, construction or bound.

The paper reports six new solutions found by models. Most models score under 10 percent. Tencent reports pass@4.

Released
March 2026
Kind
Research mathematics
Level
Research frontier
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
GPT-5.6 Sol #13 on the capability index
Last in a launch post
August 2026
Last model added
August 2026
Scores refreshed
November 12th, 2026

In the authors’ words

Can AI make progress on important, unsolved mathematical problems? Large language models are now capable of sophisticated mathematical and scientific reasoning, but whether they can perform novel research is still widely debated and underexplored. We introduce HorizonMath, a benchmark of 113 predominantly unsolved problems spanning eight domains in mathematics and the mathematical sciences, paired with an open-source evaluation framework for automated verification. Our benchmark targets the generator-verifier gap: problems where discovery is hard and requires meaningful mathematical insight, but verification is computationally straightforward. This contrasts with most existing research-level benchmarks, which instead rely on formal proof verification or manual review, both of which are expensive to scale. Because these solutions are unknown, HorizonMath is resistant to data contamination, and most state-of-the-art models score under 10%. Using this framework, we identify six novel solutions to research problems that either resolve previously open questions or improve on the best-known published results, with GPT-5.4 Pro and GPT-5.6 Sol each discovering three of these solutions. Across seven frontier model families, reasoning efficiency and behavior also vary substantially. We release HorizonMath as an open challenge and a growing community resource, where each verified solution is a candidate contribution to the mathematical literature.

Abstract of Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification (arXiv:2603.15617, CC0)
8
Models scored on its own leaderboard
2
Launch posts list it, from 1 lab
1
System card reports it, from 1 lab

GPT-5.6 Sol leads with 11.5% among models released by July 2026

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best.

GPT-5.6 Sol leads with 11.5%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in July 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 GPT-5.6 Sol max 11.5%
  2. 2 GPT-5.4 Pro 7.10%
  3. 3 Claude Opus 4.6 2.70%
  4. 4 Gemini 3.1 Pro 2.70%
  5. 5 DeepSeek-V4-Pro 2.70%
  6. 6 Kimi K3 2.70%
  7. 7 GLM-5.1 1.80%
  8. 8 Kimi K2.6 0.90%

Source: Official HorizonMath leaderboard

#ModelScoreSettingDateSource
1 GPT-5.6 Sol 11.5% max Jul 2026 Official leaderboard
2 GPT-5.4 Pro 7.10% Mar 2026 Official leaderboard
3 Claude Opus 4.6 2.70% Feb 2026 Official leaderboard
4 Gemini 3.1 Pro 2.70% Feb 2026 Official leaderboard
5 DeepSeek-V4-Pro 2.70% Apr 2026 Official leaderboard
6 Kimi K3 2.70% Jul 2026 Official leaderboard
7 GLM-5.1 1.80% Apr 2026 Official leaderboard
8 Kimi K2.6 0.90% Apr 2026 Official leaderboard
#ModelScoreSettingDateSource
1 GPT-5.6 Sol 11.5% max Jul 2026 Official leaderboard
2 Tencent Hy4 preview 8.80% Aug 2026 Lab model card
3 GPT-5.4 Pro 7.10% Mar 2026 Official leaderboard
4 Tencent Hy3 7.10% Jul 2026 Lab model card
5 Claude Opus 4.6 2.70% Feb 2026 Official leaderboard
6 Gemini 3.1 Pro 2.70% Feb 2026 Official leaderboard
7 DeepSeek-V4-Pro 2.70% Apr 2026 Official leaderboard
8 Kimi K3 2.70% Jul 2026 Official leaderboard
9 Seed2.1 Pro 2.00% Jun 2026 Lab model card
10 GLM-5.1 1.80% Apr 2026 Official leaderboard
11 Kimi K2.6 0.90% Apr 2026 Official leaderboard

Labs report scores for 3 models in their cards. Switch the scores above to rank them together.

3 models have held number one on the official leaderboard since its release in March 2026 3 models have held number one with claims included since its release in March 2026

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic OpenAI ByteDance Seed Tencent
Own leaderboard Not tested since Jul 2026
GPT-5.6 Sol 11.5%
Lab claims Not tested since Aug 2026

Sources: Official HorizonMath leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.6 Sol GPT-5.4 Pro Claude Opus 4.6 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.6 Sol Claude Opus 4.6

Sources: Official HorizonMath leaderboard and labs’ model cards

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Tencent
DateLaunch postScoreAs the post writes it
Aug 28, 2026 Tencent Hy4 preview 8.8 HorizonMath (pass@4)
Jul 2, 2026 Tencent Hy3 7.1 HorizonMath (pass@12)

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Jun 23, 2026 Seed2.1 Pro model card 2 HorizonMath (Table 6) Capabilities

More in Math

All 113 in Math
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • MathArena Apex Shortlist In use Models solve 47 hard final-answer problems that MathArena selected from 2025 competitions, where the best models scored about 50% at selection. Scored by accuracy over repeated attempts. GPT-5.5 98.4%