Benchmark Almanac

RSI-Exam

Claude Opus 5.5 53.6% leads since September 2026
Not in model cards

Agents get a weak method on a long research task and must turn it into one that scores better on hidden data. It measures recursive self-improvement on executable research.

35 public tasks with grading containers and 53 held back. Runs on Harbor with a sealed network policy. About 190 GitHub stars.

Released
August 2026
Kind
AI research tasks
Level
Research frontier
Scored
Checked automatically
Answers
Agent task
Tasks
Public, plus a private test set
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
September 2026
Scores refreshed
November 12th, 2026
19
Entries on its own leaderboard, 18 models we track

Claude Opus 5.5 leads with 53.6%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

19 entries. Scroll sideways for the rest.

  1. 53.6 Claude Opus 5.5
  2. 51.3 GPT-6 Astra
  3. 49.4 Gemini 4 Argon
  4. 48.1 Claude Fable 5.1
  5. 46.4 Claude Opus 5
  6. 43.3 GPT-5.6 Sol
  7. 40.3 GLM-5.3
  8. 39.2 Qwen3.8 Max (0902)
  9. 39.1 Muse Spark 1.3
  10. 38.4 Grok 4.7
  11. 38.2 Kimi K3
  12. 36.7 Grok 4.6
  13. 35.2 Step 5 Preview
  14. 34.1 Gemini 3.8 Flash
  15. 33.1 GPT-5.5
  16. 32.4 Seed Evolving-0909
  17. 32.3 DeepSeek-V4-Pro
  18. 32.2 Qwen 3.8 Max
  19. 30.9 Gemini 3.7 Flash
  1. 1 Claude Opus 5.5 max 53.6%
  2. 2 GPT-6 Astra max 51.3%
  3. 3 Gemini 4 Argon high 49.4%
  4. 4 Claude Fable 5.1 max 48.1%
  5. 5 Claude Opus 5 max 46.4%
  6. 6 GPT-5.6 Sol max 43.3%
  7. 7 GLM-5.3 max 40.3%
  8. 8 Qwen3.8 Max (0902) xhigh 39.2%
  9. 9 Muse Spark 1.3 max 39.1%
  10. 10 Grok 4.7 xhigh 38.4%
  11. 11 Kimi K3 max 38.2%
  12. 12 Grok 4.6 xhigh 36.7%
  13. 13 Step 5 Preview max 35.2%
  14. 14 Gemini 3.8 Flash high 34.1%
  15. 15 GPT-5.5 xhigh 33.1%
  16. 16 Seed Evolving-0909 max 32.4%
  17. 17 DeepSeek-V4-Pro max 32.3%
  18. 18 Qwen 3.8 Max xhigh 32.2%
  19. 19 Gemini 3.7 Flash high 30.9%

Source: Official RSI-Exam leaderboard

#EntryScoreSettingDateSource
1 Claude Opus 5.5 53.6% max, claude code Sep 2026 Official leaderboard
2 GPT-6 Astra 51.3% max, codex Sep 2026 Official leaderboard
3 Gemini 4 Argon 49.4% high, antigravity Sep 2026 Official leaderboard
4 Claude Fable 5.1 48.1% max, claude code Sep 2026 Official leaderboard
5 Claude Opus 5 46.4% max, claude code Jul 2026 Official leaderboard
6 GPT-5.6 Sol 43.3% max, codex Jul 2026 Official leaderboard
7 GLM-5.3 40.3% max, claude code Aug 2026 Official leaderboard
8 Qwen3.8 Max (0902) 39.2% xhigh, claude code Sep 2026 Official leaderboard
9 Muse Spark 1.3 39.1% max, musecode Sep 2026 Official leaderboard
10 Grok 4.7 38.4% xhigh, grok Sep 2026 Official leaderboard
11 Kimi K3 38.2% max, kimi code Jul 2026 Official leaderboard
12 Grok 4.6 36.7% xhigh, grok Aug 2026 Official leaderboard
13 Step 5 Preview 35.2% max, claude code Sep 2026 Official leaderboard
14 Gemini 3.8 Flash 34.1% high, antigravity Sep 2026 Official leaderboard
15 GPT-5.5 33.1% xhigh, codex Apr 2026 Official leaderboard
16 Seed Evolving-0909 32.4% max, claude code Official leaderboard
17 DeepSeek-V4-Pro 32.3% max, claude code Apr 2026 Official leaderboard
18 Qwen 3.8 Max 32.2% xhigh, qwen coder Aug 2026 Official leaderboard
19 Gemini 3.7 Flash 30.9% high, antigravity Aug 2026 Official leaderboard

Model resource comparison: average run time

Mean hidden-set score over the 88 published tasks against how long one task took on average. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

30 40 50 60 0200400600 Average run time per task (min), lower is better Mean held-out normalized score GPT-5.5 (xhigh): Mean held-out normalized score 33.1, average run time per task 101 min DeepSeek-V4-Pro (max): Mean held-out normalized score 32.3, average run time per task 282 min GPT-5.6 Sol (max): Mean held-out normalized score 43.3, average run time per task 274 min Kimi K3 (max): Mean held-out normalized score 38.2, average run time per task 289 min Claude Opus 5 (max): Mean held-out normalized score 46.4, average run time per task 333 min Qwen 3.8 Max (xhigh): Mean held-out normalized score 32.2, average run time per task 201 min Grok 4.6 (xhigh): Mean held-out normalized score 36.7, average run time per task 116 min Gemini 3.7 Flash (high): Mean held-out normalized score 30.9, average run time per task 161 min GLM-5.3 (max): Mean held-out normalized score 40.3, average run time per task 342 min Qwen3.8 Max (0902) (xhigh): Mean held-out normalized score 39.2, average run time per task 525 min Gemini 3.8 Flash (high): Mean held-out normalized score 34.1, average run time per task 101 min Muse Spark 1.3 (max): Mean held-out normalized score 39.1, average run time per task 233 min Claude Fable 5.1 (max): Mean held-out normalized score 48.1, average run time per task 224 min GPT-6 Astra (max): Mean held-out normalized score 51.3, average run time per task 263 min Grok 4.7 (xhigh): Mean held-out normalized score 38.4, average run time per task 89.7 min Claude Opus 5.5 (max): Mean held-out normalized score 53.6, average run time per task 355 min Claude Opus 5.5 (max)GPT-6 Astra (max)Claude Fable 5.1 (max)Grok 4.7 (xhigh) 30 40 50 60 0200400600 Average run time per task (min), lower is better Mean held-out normalized score GPT-5.5 (xhigh): Mean held-out normalized score 33.1, average run time per task 101 min DeepSeek-V4-Pro (max): Mean held-out normalized score 32.3, average run time per task 282 min GPT-5.6 Sol (max): Mean held-out normalized score 43.3, average run time per task 274 min Kimi K3 (max): Mean held-out normalized score 38.2, average run time per task 289 min Claude Opus 5 (max): Mean held-out normalized score 46.4, average run time per task 333 min Qwen 3.8 Max (xhigh): Mean held-out normalized score 32.2, average run time per task 201 min Grok 4.6 (xhigh): Mean held-out normalized score 36.7, average run time per task 116 min Gemini 3.7 Flash (high): Mean held-out normalized score 30.9, average run time per task 161 min GLM-5.3 (max): Mean held-out normalized score 40.3, average run time per task 342 min Qwen3.8 Max (0902) (xhigh): Mean held-out normalized score 39.2, average run time per task 525 min Gemini 3.8 Flash (high): Mean held-out normalized score 34.1, average run time per task 101 min Muse Spark 1.3 (max): Mean held-out normalized score 39.1, average run time per task 233 min Claude Fable 5.1 (max): Mean held-out normalized score 48.1, average run time per task 224 min GPT-6 Astra (max): Mean held-out normalized score 51.3, average run time per task 263 min Grok 4.7 (xhigh): Mean held-out normalized score 38.4, average run time per task 89.7 min Claude Opus 5.5 (max): Mean held-out normalized score 53.6, average run time per task 355 min Claude Opus 5.5 (max)GPT-6 Astra (max)Claude Fable 5.1 (max)Grok 4.7 (xhigh)
ModelSettingMean held-out normalized scoreAverage run time per task
Claude Opus 5.5 max, claude code 53.57 355.2 min
GPT-6 Astra max, codex 51.26 262.9 min
Claude Fable 5.1 max, claude code 48.13 224.3 min
Claude Opus 5 max, claude code 46.4 333.4 min
GPT-5.6 Sol max, codex 43.31 274.1 min
GLM-5.3 max, claude code 40.29 342.2 min
Qwen3.8 Max (0902) xhigh, claude code 39.23 525.3 min
Muse Spark 1.3 max, musecode 39.07 233 min
Grok 4.7 xhigh, grok 38.42 89.7 min
Kimi K3 max, kimi code 38.2 288.9 min
Grok 4.6 xhigh, grok 36.71 115.9 min
Gemini 3.8 Flash high, antigravity 34.06 101.3 min
GPT-5.5 xhigh, codex 33.12 100.6 min
DeepSeek-V4-Pro max, claude code 32.25 282 min
Qwen 3.8 Max xhigh, qwen coder 32.18 201.4 min
Gemini 3.7 Flash high, antigravity 30.88 160.8 min

The best score, 53.6%, costs $21 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Moonshot AI Z.ai Qwen Google DeepMind Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% $1.00 $2.00 $5.00 $10 $20 $50 $100 Cost per task, log scale Claude Opus 5.5, max GLM-5.3, max Qwen3.8 Max (0902), xhigh Gemini 3.8 Flash, high 0% 20% 40% 60% $1.00 $10 $100 Cost per task, log scale Claude Opus 5.5, max GLM-5.3, max

Source: Official RSI-Exam leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 max claude code 53.6% $21 280,471
GPT-6 Astra max codex 51.3% $53 340,814
Gemini 4 Argon high antigravity 49.4%
Claude Fable 5.1 max claude code 48.1% $29 230,270
Claude Opus 5 max claude code 46.4% $42 287,844
GPT-5.6 Sol max codex 43.3% $53 233,916
GLM-5.3 max claude code 40.3% $18 273,129
Qwen3.8 Max (0902) xhigh claude code 39.2% $12 327,964
Muse Spark 1.3 max musecode 39.1% $12 340,759
Grok 4.7 xhigh grok 38.4% $13 191,762
Kimi K3 max kimi code 38.2% $10 133,843
Grok 4.6 xhigh grok 36.7% $14 251,550
Step 5 Preview max claude code 35.2%
Gemini 3.8 Flash high antigravity 34.1% $5.22 203,554
GPT-5.5 xhigh codex 33.1% $13 69,741
DeepSeek-V4-Pro max claude code 32.3% $6.66 507,998
Qwen 3.8 Max xhigh qwen coder 32.2% $20 229,019
Gemini 3.7 Flash high antigravity 30.9% $9.34 395,387

4 models have held number one on the official leaderboard since its release in August 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic OpenAI
Own leaderboard 4 leaders
Claude Opus 5.5 53.6%

Source: Official RSI-Exam leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Opus 5.5 Claude Fable 5.1 GPT-5.6 Sol GPT-5.5 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Opus 5.5 Claude Opus 5

Source: Official RSI-Exam leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in AI research

All 104 in AI research
  • PaperBench In use Agent replicates ICML 2024 papers from scratch. LLM judge grades against author-written hierarchical rubrics. Qwen 3.8 Max 93%
  • MLS-Bench In use Agent must invent one algorithmic change in an ML research scaffold whose gain transfers across settings. Scored relative to human baselines. Claude Fable 5.1 50.3%
  • PostTrainBench 1.2 In use CLI agent gets a base model, one H100 and a time budget to post-train it for a target benchmark. Scored on normalized gain. Claude Fable 5.1 44.6%
  • MLE-bench In use Kaggle-style ML engineering competitions. Agent trains models and is scored against competition leaderboards. GPT-6 Astra 93.8%
  • KernelBench In use Replace PyTorch operators with faster custom GPU kernels. Scored on correctness and speedup. MiniMax-M3 28.8%
  • PMPP-Hard In use Agents write GPU kernels for 69 tasks that must be correct, and on 48 of them also run within 1.25 times the reference time. The score is the area under the solved-tasks curve over a 10-hour budget, out of 690 task-hours. Inkling 2 Small 494.3