VeruSAGE-Bench
Version repository
Agents synthesize Verus proofs for 849 tasks extracted from eight real-world Rust systems projects.
The official leaderboard reports verified solve rates, time and cost for repository-level tasks.
- Made by
- Microsoft Research and collaborators
- Released
- December 2025
- Kind
- Fixing real repositories
- Level
- Research frontier
- Scored
- Checked automatically
- Answers
- Code
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Single turn
- Environment
- Code sandbox
- Items from
- Real work and use
- Refresh
- One fixed set
- Run by
- Anyone
- Based on
- Verus-Bench
VeruSAGE (Hands-off) + Claude Opus 4.5 tops the leaderboard with 93.9% No scores yet
Best score per entry from the benchmark's own leaderboard, read on November 21st, 2026. Where a model ran on its own with several settings, we keep its best. Every run is in the next chart. Agents other companies built around a model get a pale shade of that model's lab color. Gray marks blends and entries not in our model list, such as fine-tunes and older models. None of these count toward leaders. Agents built around a model, such as VeruSAGE (Hands-off) + Claude Opus 4.5, are their own entries. We don't count them as the model's score. This view shows only models we track, each at its best.
- 93.9 VeruSAGE (Hands-off) + Claude Opus 4.5
- 80.9 VeruSAGE (Hands-off) + Claude Sonnet 4.5
- 67.5 VeruSAGE (Hands-on) + Claude Sonnet 4.5
- 64.1 VeruSAGE (Hands-off) + Claude Sonnet 4
- 57.8 VeruSAGE (Hands-on) + Claude Sonnet 4
- 56.4 VeruSAGE (Hands-on) + GPT-5
- 50.1 VeruSAGE (Hands-off) + GPT-5
- 41.1 VeruSAGE (Hands-on) + o4-mini
- 19.8 AutoVerus (Baseline) + o4-mini
- 17.0 VeruSAGE (Hands-off) + o4-mini
- 1 VeruSAGE (Hands-off) + Claude Opus 4.5 Hands-off 93.9%
- 2 VeruSAGE (Hands-off) + Claude Sonnet 4.5 Hands-off 80.9%
- 3 VeruSAGE (Hands-on) + Claude Sonnet 4.5 Hands-on 67.5%
- 4 VeruSAGE (Hands-off) + Claude Sonnet 4 Hands-off 64.1%
- 5 VeruSAGE (Hands-on) + Claude Sonnet 4 Hands-on 57.8%
- 6 VeruSAGE (Hands-on) + GPT-5 Hands-on 56.4%
- 7 VeruSAGE (Hands-off) + GPT-5 Hands-off 50.1%
- 8 VeruSAGE (Hands-on) + o4-mini Hands-on 41.1%
- 9 AutoVerus (Baseline) + o4-mini AutoVerus (Baseline) 19.8%
- 10 VeruSAGE (Hands-off) + o4-mini Hands-off 17%
Source: Official VeruSAGE-Bench leaderboard
| # | Entry | Score | Setting | Released | Run on | Link | Source |
|---|---|---|---|---|---|---|---|
| 1 | VeruSAGE (Hands-off) + Claude Opus 4.5 | 93.9% | Hands-off | Nov 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| 2 | VeruSAGE (Hands-off) + Claude Sonnet 4.5 | 80.9% | Hands-off | Sep 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| 3 | VeruSAGE (Hands-on) + Claude Sonnet 4.5 | 67.5% | Hands-on | Sep 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| 4 | VeruSAGE (Hands-off) + Claude Sonnet 4 | 64.1% | Hands-off | May 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| 5 | VeruSAGE (Hands-on) + Claude Sonnet 4 | 57.8% | Hands-on | May 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| 6 | VeruSAGE (Hands-on) + GPT-5 | 56.4% | Hands-on | Aug 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| 7 | VeruSAGE (Hands-off) + GPT-5 | 50.1% | Hands-off | Aug 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| 8 | VeruSAGE (Hands-on) + o4-mini | 41.1% | Hands-on | Apr 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| 9 | AutoVerus (Baseline) + o4-mini | 19.8% | AutoVerus (Baseline) | Apr 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| 10 | VeruSAGE (Hands-off) + o4-mini | 17% | Hands-off | Apr 2025 | Jan 2026 | arxiv.org | Official leaderboard |
| # | Model | Score | Setting | Released | Run on | Link | Source |
|---|
The best score, 93.9%, costs $14 a task
Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
Source: Official VeruSAGE-Bench leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| VeruSAGE (Hands-off) + Claude Opus 4.5 | Hands-off | VeruSAGE (Hands-off) | 93.9% | $14 | |
| VeruSAGE (Hands-off) + Claude Sonnet 4.5 | Hands-off | VeruSAGE (Hands-off) | 80.9% | $5.61 | |
| VeruSAGE (Hands-on) + Claude Sonnet 4.5 | Hands-on | VeruSAGE (Hands-on) | 67.5% | $1.47 | |
| VeruSAGE (Hands-off) + Claude Sonnet 4 | Hands-off | VeruSAGE (Hands-off) | 64.1% | $6.65 | |
| VeruSAGE (Hands-on) + Claude Sonnet 4 | Hands-on | VeruSAGE (Hands-on) | 57.8% | $1.72 | |
| VeruSAGE (Hands-on) + GPT-5 | Hands-on | VeruSAGE (Hands-on) | 56.4% | $0.52 | |
| VeruSAGE (Hands-off) + GPT-5 | Hands-off | VeruSAGE (Hands-off) | 50.1% | $1.30 | |
| VeruSAGE (Hands-on) + o4-mini | Hands-on | VeruSAGE (Hands-on) | 41.1% | $0.67 | |
| AutoVerus (Baseline) + o4-mini | AutoVerus (Baseline) | AutoVerus (Baseline) | 19.8% | $0.74 | |
| VeruSAGE (Hands-off) + o4-mini | Hands-off | VeruSAGE (Hands-off) | 17% | $0.81 |
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
Versions
More in Coding
- SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
- LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests.
DeepSeek-V4-Pro 93.5%
- SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
- DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
- SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases.
Seed2.1 Pro 59.8%
- Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance.
DeepSeek V4.1 Flash 3,471
Data audit
All 8 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
- Pass Independent runs Scores from the official leaderboard (10 entries).
- Pass Models scored 10 entries on its leaderboard, none a model we track.
- Note Newest score No scores yet.
- Note Frontier models No models scored yet.
- Note Headroom No scores yet.
Leaderboard import 3 pass
- Pass Importer We read the board on microsoft.github.io: 10 rows for models we track at the last read.
- Pass Last read November 21st, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.