FLTEval

Name: FLTEval
Published: 2026-03

Leanstral 1.5 43.2

Claude Opus 4.6 39.6 leads since March 2026

Leanstral 1.5 43.2 leads since July 2026

Fading

Labs’ own numbers All independent runs Include claimed scores

Agents complete the formal Lean proofs and define the new mathematical concepts in real pull requests to the Fermat's Last Theorem (FLT) formalization project. Lean and SafeVerify check each submitted diff.

Mistral introduced it with Leanstral in 2026-03 and open-sourced the harness with Leanstral 1.5 in 2026-07. The harness is shaped like SWE-bench and the repo ships a 162-instance sample.

Made by: Mistral AI
Released: March 2026
Kind: Formal proofs
Level: Research frontier
Scored: Checked automatically
Answers: Agent task
Tasks: All public

Strongest model tested: Claude Opus 4.6 #37 on the capability index
Last in a launch post: July 2026, dropped before it saturated
Last model added: July 2026
Scores refreshed: November 12th, 2026

Models scored in lab cards

Launch posts list it, from 1 lab

System cards report it, from 1 lab

Leanstral 1.5 leads with 43.2

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Claude Opus 4.6 leads with 39.6 across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Leanstral 1.5 leads with 43.2, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

Bars Table

1 Leanstral 1.5 43.2
2 Leanstral 31.9

Source: Labs’ model cards

Source: Official FLTEval leaderboard

Sources: Labs’ model cards and official FLTEval leaderboard

#	Model	Score	Setting	Date	Source
1	Leanstral 1.5	43.2		Jul 2026	Lab model card
2	Leanstral	31.9		Mar 2026	Lab model card

#	Model	Score	Setting	Date	Source
1	Claude Opus 4.6	39.6	Mistral Vibe	Feb 2026	Official leaderboard
2	Claude Sonnet 4.6	23.7	Mistral Vibe	Feb 2026	Official leaderboard
3	Claude Haiku 4.5	23.0	Mistral Vibe	Oct 2025	Official leaderboard
4	Leanstral	21.9	Mistral Vibe	Mar 2026	Official leaderboard

#	Model	Score	Setting	Date	Source
1	Leanstral 1.5	43.2		Jul 2026	Lab model card
2	Claude Opus 4.6	39.6	Mistral Vibe	Feb 2026	Official leaderboard
3	Leanstral	31.9		Mar 2026	Lab model card
4	Claude Sonnet 4.6	23.7	Mistral Vibe	Feb 2026	Official leaderboard
5	Claude Haiku 4.5	23.0	Mistral Vibe	Oct 2025	Official leaderboard

The benchmark's own leaderboard has 4 models. Switch the scores above to rank them together.

FLTEval score versus run cost

Cost covers the whole evaluation run, not one task. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

Chart Table

Model	Setting	FLTEval score	Total run cost
Claude Opus 4.6	Mistral Vibe	39.6 score	$1,650
Claude Sonnet 4.6	Mistral Vibe	23.7 score	$549
Claude Haiku 4.5	Mistral Vibe	23 score	$184
Leanstral	Mistral Vibe	21.9 score	$18

Number one over time Number one over time 2 models have held number one with claims included since its release in March 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic

Own leaderboard Not tested since Mar 2026

Claude Opus 4.6 39.6

Source: Official FLTEval leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card

Sources: Official FLTEval leaderboard and labs’ model cards

Labs headline it less often now, last in July 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Mistral AI

Date	Launch post	Score	As the post writes it
Jul 2, 2026	Leanstral 1.5	43.2	FLTEval pass@8 43.2 (pass@1 28.9)
Mar 16, 2026	Leanstral	31.9	FLTEval score, Leanstral pass@16 (pass@1 21.9)

2 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

Date	Document	Score	As the document writes it	Section
Jul 2, 2026	Leanstral technical report	31.9	FLTEval Leanstral 1.0 pass@16, cost $290. pass@8 31.0, pass@4 29.3, pass@2 26.3, pass@1 21.9 (Table 3)	Capabilities
Jul 2, 2026	Leanstral 1.5 technical report	43.2	FLTEval pass@8, cost $216. pass@4 38.9, pass@2 34.6, pass@1 28.9 (Table 3)	Capabilities

One lab claim next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.

Model	Lab says	Board says	Gap	What differs
Leanstral	31.9 FLTEval score, Leanstral pass@16 (pass@1 21.9)	21.9 Leaderboard, Mistral Vibe	+10.0	The lab reports success across sixteen attempts, while the board reports single-attempt success.