Benchmark Almanac

FrontierMemory

Claude Opus 5.5 73.3% leads since September 2026
Not in model cards

A model reads 750 past agent trajectories and writes memory for a task agent. The score is how many new coding and knowledge-work tasks that agent then solves.

15 tasks adapted from DeepSWE, BigCodeBench, AutomationBench, APEX-Agents, EnterpriseOps-Gym and SkillsBench. 10 models in the Fireworks index. No maker page found.

Made by
Letta
Kind
AI research tasks
Level
Research frontier
Scored
Checked automatically
Answers
Agent task
Tasks
Kept private
Strongest model tested
Claude Opus 5.5 #4 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
7
Models scored on its own leaderboard

Claude Opus 5.5 leads with 73.3%

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

  1. 73.3 Claude Opus 5.5
  2. 71.1 Kimi K3
  3. 68.9 GLM-5.3
  4. 66.7 Gemini 4 Argon
  5. 51.1 GPT-6.1 Sol
  6. 51.1 Muse Wonder
  7. 37.8 Grok 4.7
  1. 1 Claude Opus 5.5 memory effort max; task Opus 5.5 (max) 73.3%
  2. 2 Kimi K3 memory effort max; task Opus 5.5 (xhigh) 71.1%
  3. 3 GLM-5.3 memory effort max; task Opus 5.5 (xhigh) 68.9%
  4. 4 Gemini 4 Argon memory effort high; task Opus 5.5 (xhigh) 66.7%
  5. 5 GPT-6.1 Sol memory effort max; task Opus 5.5 (max) 51.1%
  6. 6 Muse Wonder memory effort xhigh; task Opus 5.5 (max) 51.1%
  7. 7 Grok 4.7 memory effort xhigh; task Opus 5.5 (max) 37.8%

Source: Official FrontierMemory leaderboard

#ModelScoreSettingDateSource
1 Claude Opus 5.5 73.3% memory effort max; task Opus 5.5 (max), Letta Dream / Letta Code Sep 2026 Official leaderboard
2 Kimi K3 71.1% memory effort max; task Opus 5.5 (xhigh), Letta Dream / Letta Code Jul 2026 Official leaderboard
3 GLM-5.3 68.9% memory effort max; task Opus 5.5 (xhigh), Letta Dream / Letta Code Aug 2026 Official leaderboard
4 Gemini 4 Argon 66.7% memory effort high; task Opus 5.5 (xhigh), Letta Dream / Letta Code Sep 2026 Official leaderboard
5 GPT-6.1 Sol 51.1% memory effort max; task Opus 5.5 (max), Letta Dream / Letta Code Sep 2026 Official leaderboard
6 Muse Wonder 51.1% memory effort xhigh; task Opus 5.5 (max), Letta Dream / Letta Code Oct 2026 Official leaderboard
7 Grok 4.7 37.8% memory effort xhigh; task Opus 5.5 (max), Letta Dream / Letta Code Sep 2026 Official leaderboard

Memory model performance versus task cost

Costs exclude dreaming and omit trials without usage. Rows distinguish memory models, task models, and effort. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0% 20% 40% 60% 80% $0.1$1$10 Mean estimated task-model cost per trial ($), log scale, lower is better Full-pass rate Kimi K3 (memory effort max; task Opus 5.5 (low)): Full-pass rate 46.7%, mean estimated task-model cost per trial $0.3 Kimi K3 (memory effort max; task Opus 5.5 (medium)): Full-pass rate 57.8%, mean estimated task-model cost per trial $0.56 Kimi K3 (memory effort max; task Opus 5.5 (high)): Full-pass rate 60%, mean estimated task-model cost per trial $0.7 Kimi K3 (memory effort max; task Opus 5.5 (max)): Full-pass rate 71.1%, mean estimated task-model cost per trial $4.16 Kimi K3 (memory effort max; task GPT-6 Sol (low)): Full-pass rate 40%, mean estimated task-model cost per trial $0.3 Kimi K3 (memory effort max; task GPT-6 Sol (medium)): Full-pass rate 44.4%, mean estimated task-model cost per trial $0.39 Kimi K3 (memory effort max; task GPT-6 Sol (high)): Full-pass rate 57.8%, mean estimated task-model cost per trial $0.57 Kimi K3 (memory effort max; task GPT-6 Sol (xhigh)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.72 Kimi K3 (memory effort max; task GPT-6 Sol (max)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.96 GLM-5.3 (memory effort max; task Opus 5.5 (low)): Full-pass rate 40%, mean estimated task-model cost per trial $0.3 GLM-5.3 (memory effort max; task Opus 5.5 (medium)): Full-pass rate 51.1%, mean estimated task-model cost per trial $0.51 GLM-5.3 (memory effort max; task Opus 5.5 (high)): Full-pass rate 57.8%, mean estimated task-model cost per trial $0.67 GLM-5.3 (memory effort max; task Opus 5.5 (xhigh)): Full-pass rate 68.9%, mean estimated task-model cost per trial $1.33 GLM-5.3 (memory effort max; task Opus 5.5 (max)): Full-pass rate 68.9%, mean estimated task-model cost per trial $4.08 GLM-5.3 (memory effort max; task GPT-6 Sol (medium)): Full-pass rate 46.7%, mean estimated task-model cost per trial $0.38 GLM-5.3 (memory effort max; task GPT-6 Sol (high)): Full-pass rate 44.4%, mean estimated task-model cost per trial $0.52 GLM-5.3 (memory effort max; task GPT-6 Sol (xhigh)): Full-pass rate 51.1%, mean estimated task-model cost per trial $0.67 GLM-5.3 (memory effort max; task GPT-6 Sol (max)): Full-pass rate 48.9%, mean estimated task-model cost per trial $1 Grok 4.7 (memory effort xhigh; task Opus 5.5 (low)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.29 Grok 4.7 (memory effort xhigh; task Opus 5.5 (medium)): Full-pass rate 24.4%, mean estimated task-model cost per trial $0.49 Grok 4.7 (memory effort xhigh; task Opus 5.5 (high)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.64 Grok 4.7 (memory effort xhigh; task Opus 5.5 (xhigh)): Full-pass rate 28.9%, mean estimated task-model cost per trial $1.64 Grok 4.7 (memory effort xhigh; task Opus 5.5 (max)): Full-pass rate 37.8%, mean estimated task-model cost per trial $3.7 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (medium)): Full-pass rate 31.1%, mean estimated task-model cost per trial $0.36 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (high)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.51 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (xhigh)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.63 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (max)): Full-pass rate 31.1%, mean estimated task-model cost per trial $1.02 Claude Opus 5.5 (memory effort max; task Opus 5.5 (low)): Full-pass rate 48.9%, mean estimated task-model cost per trial $0.28 Claude Opus 5.5 (memory effort max; task Opus 5.5 (high)): Full-pass rate 66.7%, mean estimated task-model cost per trial $0.63 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (xhigh)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.68 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (max)): Full-pass rate 64.4%, mean estimated task-model cost per trial $0.98 GPT-6.1 Sol (memory effort max; task Opus 5.5 (low)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.3 GPT-6.1 Sol (memory effort max; task Opus 5.5 (medium)): Full-pass rate 31.1%, mean estimated task-model cost per trial $0.56 GPT-6.1 Sol (memory effort max; task Opus 5.5 (high)): Full-pass rate 40%, mean estimated task-model cost per trial $0.7 GPT-6.1 Sol (memory effort max; task Opus 5.5 (xhigh)): Full-pass rate 44.4%, mean estimated task-model cost per trial $1.46 GPT-6.1 Sol (memory effort max; task Opus 5.5 (max)): Full-pass rate 51.1%, mean estimated task-model cost per trial $4 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (low)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.23 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (medium)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.37 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (high)): Full-pass rate 37.8%, mean estimated task-model cost per trial $0.56 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (xhigh)): Full-pass rate 44.4%, mean estimated task-model cost per trial $0.67 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (max)): Full-pass rate 42.2%, mean estimated task-model cost per trial $1.01 Gemini 4 Argon (memory effort high; task Opus 5.5 (low)): Full-pass rate 44.4%, mean estimated task-model cost per trial $0.28 Gemini 4 Argon (memory effort high; task Opus 5.5 (medium)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.57 Gemini 4 Argon (memory effort high; task Opus 5.5 (high)): Full-pass rate 64.4%, mean estimated task-model cost per trial $0.75 Gemini 4 Argon (memory effort high; task Opus 5.5 (xhigh)): Full-pass rate 66.7%, mean estimated task-model cost per trial $1.46 Gemini 4 Argon (memory effort high; task Opus 5.5 (max)): Full-pass rate 64.4%, mean estimated task-model cost per trial $3.78 Gemini 4 Argon (memory effort high; task GPT-6 Sol (low)): Full-pass rate 46.7%, mean estimated task-model cost per trial $0.29 Gemini 4 Argon (memory effort high; task GPT-6 Sol (medium)): Full-pass rate 55.6%, mean estimated task-model cost per trial $0.44 Gemini 4 Argon (memory effort high; task GPT-6 Sol (high)): Full-pass rate 57.8%, mean estimated task-model cost per trial $0.58 Gemini 4 Argon (memory effort high; task GPT-6 Sol (xhigh)): Full-pass rate 66.7%, mean estimated task-model cost per trial $0.74 Gemini 4 Argon (memory effort high; task GPT-6 Sol (max)): Full-pass rate 64.4%, mean estimated task-model cost per trial $1.05 Muse Wonder (memory effort xhigh; task Opus 5.5 (low)): Full-pass rate 28.9%, mean estimated task-model cost per trial $0.28 Muse Wonder (memory effort xhigh; task Opus 5.5 (medium)): Full-pass rate 31.1%, mean estimated task-model cost per trial $0.51 Muse Wonder (memory effort xhigh; task Opus 5.5 (high)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.64 Muse Wonder (memory effort xhigh; task Opus 5.5 (xhigh)): Full-pass rate 40%, mean estimated task-model cost per trial $1.32 Muse Wonder (memory effort xhigh; task Opus 5.5 (max)): Full-pass rate 51.1%, mean estimated task-model cost per trial $3.56 Muse Wonder (memory effort xhigh; task GPT-6 Sol (low)): Full-pass rate 20%, mean estimated task-model cost per trial $0.23 Muse Wonder (memory effort xhigh; task GPT-6 Sol (medium)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.4 Muse Wonder (memory effort xhigh; task GPT-6 Sol (high)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.52 Muse Wonder (memory effort xhigh; task GPT-6 Sol (xhigh)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.74 Muse Wonder (memory effort xhigh; task GPT-6 Sol (max)): Full-pass rate 26.7%, mean estimated task-model cost per trial $1.09 Kimi K3 (memory effort max; task Opus 5.5 (xhigh)): Full-pass rate 71.1%, mean estimated task-model cost per trial $1.49 GLM-5.3 (memory effort max; task GPT-6 Sol (low)): Full-pass rate 37.8%, mean estimated task-model cost per trial $0.23 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (low)): Full-pass rate 20%, mean estimated task-model cost per trial $0.22 Claude Opus 5.5 (memory effort max; task Opus 5.5 (medium)): Full-pass rate 64.4%, mean estimated task-model cost per trial $0.51 Claude Opus 5.5 (memory effort max; task Opus 5.5 (xhigh)): Full-pass rate 68.9%, mean estimated task-model cost per trial $1.31 Claude Opus 5.5 (memory effort max; task Opus 5.5 (max)): Full-pass rate 73.3%, mean estimated task-model cost per trial $3.42 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (low)): Full-pass rate 48.9%, mean estimated task-model cost per trial $0.27 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (medium)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.38 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (high)): Full-pass rate 66.7%, mean estimated task-model cost per trial $0.55 Claude Opus 5.5 (memory effort max; task Opus 5.5 (max))Kimi K3 (memory effort max; task Opus 5.5 (xhigh))Claude Opus 5.5 (memory effort max; task Opus 5.5 (xhigh))Claude Opus 5.5 (memory effort max; task GPT-6 Sol (high))Claude Opus 5.5 (memory effort max; task Opus 5.5 (medium))Claude Opus 5.5 (memory effort max; task GPT-6 Sol (medium))Claude Opus 5.5 (memory effort max; task GPT-6 Sol (low))GLM-5.3 (memory effort max; task GPT-6 Sol (low))Grok 4.7 (memory effort xhigh; task GPT-6 Sol (low)) 0% 20% 40% 60% 80% $0.1$1$10 Mean estimated task-model cost per trial ($), log scale, lower is better Full-pass rate Kimi K3 (memory effort max; task Opus 5.5 (low)): Full-pass rate 46.7%, mean estimated task-model cost per trial $0.3 Kimi K3 (memory effort max; task Opus 5.5 (medium)): Full-pass rate 57.8%, mean estimated task-model cost per trial $0.56 Kimi K3 (memory effort max; task Opus 5.5 (high)): Full-pass rate 60%, mean estimated task-model cost per trial $0.7 Kimi K3 (memory effort max; task Opus 5.5 (max)): Full-pass rate 71.1%, mean estimated task-model cost per trial $4.16 Kimi K3 (memory effort max; task GPT-6 Sol (low)): Full-pass rate 40%, mean estimated task-model cost per trial $0.3 Kimi K3 (memory effort max; task GPT-6 Sol (medium)): Full-pass rate 44.4%, mean estimated task-model cost per trial $0.39 Kimi K3 (memory effort max; task GPT-6 Sol (high)): Full-pass rate 57.8%, mean estimated task-model cost per trial $0.57 Kimi K3 (memory effort max; task GPT-6 Sol (xhigh)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.72 Kimi K3 (memory effort max; task GPT-6 Sol (max)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.96 GLM-5.3 (memory effort max; task Opus 5.5 (low)): Full-pass rate 40%, mean estimated task-model cost per trial $0.3 GLM-5.3 (memory effort max; task Opus 5.5 (medium)): Full-pass rate 51.1%, mean estimated task-model cost per trial $0.51 GLM-5.3 (memory effort max; task Opus 5.5 (high)): Full-pass rate 57.8%, mean estimated task-model cost per trial $0.67 GLM-5.3 (memory effort max; task Opus 5.5 (xhigh)): Full-pass rate 68.9%, mean estimated task-model cost per trial $1.33 GLM-5.3 (memory effort max; task Opus 5.5 (max)): Full-pass rate 68.9%, mean estimated task-model cost per trial $4.08 GLM-5.3 (memory effort max; task GPT-6 Sol (medium)): Full-pass rate 46.7%, mean estimated task-model cost per trial $0.38 GLM-5.3 (memory effort max; task GPT-6 Sol (high)): Full-pass rate 44.4%, mean estimated task-model cost per trial $0.52 GLM-5.3 (memory effort max; task GPT-6 Sol (xhigh)): Full-pass rate 51.1%, mean estimated task-model cost per trial $0.67 GLM-5.3 (memory effort max; task GPT-6 Sol (max)): Full-pass rate 48.9%, mean estimated task-model cost per trial $1 Grok 4.7 (memory effort xhigh; task Opus 5.5 (low)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.29 Grok 4.7 (memory effort xhigh; task Opus 5.5 (medium)): Full-pass rate 24.4%, mean estimated task-model cost per trial $0.49 Grok 4.7 (memory effort xhigh; task Opus 5.5 (high)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.64 Grok 4.7 (memory effort xhigh; task Opus 5.5 (xhigh)): Full-pass rate 28.9%, mean estimated task-model cost per trial $1.64 Grok 4.7 (memory effort xhigh; task Opus 5.5 (max)): Full-pass rate 37.8%, mean estimated task-model cost per trial $3.7 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (medium)): Full-pass rate 31.1%, mean estimated task-model cost per trial $0.36 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (high)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.51 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (xhigh)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.63 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (max)): Full-pass rate 31.1%, mean estimated task-model cost per trial $1.02 Claude Opus 5.5 (memory effort max; task Opus 5.5 (low)): Full-pass rate 48.9%, mean estimated task-model cost per trial $0.28 Claude Opus 5.5 (memory effort max; task Opus 5.5 (high)): Full-pass rate 66.7%, mean estimated task-model cost per trial $0.63 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (xhigh)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.68 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (max)): Full-pass rate 64.4%, mean estimated task-model cost per trial $0.98 GPT-6.1 Sol (memory effort max; task Opus 5.5 (low)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.3 GPT-6.1 Sol (memory effort max; task Opus 5.5 (medium)): Full-pass rate 31.1%, mean estimated task-model cost per trial $0.56 GPT-6.1 Sol (memory effort max; task Opus 5.5 (high)): Full-pass rate 40%, mean estimated task-model cost per trial $0.7 GPT-6.1 Sol (memory effort max; task Opus 5.5 (xhigh)): Full-pass rate 44.4%, mean estimated task-model cost per trial $1.46 GPT-6.1 Sol (memory effort max; task Opus 5.5 (max)): Full-pass rate 51.1%, mean estimated task-model cost per trial $4 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (low)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.23 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (medium)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.37 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (high)): Full-pass rate 37.8%, mean estimated task-model cost per trial $0.56 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (xhigh)): Full-pass rate 44.4%, mean estimated task-model cost per trial $0.67 GPT-6.1 Sol (memory effort max; task GPT-6 Sol (max)): Full-pass rate 42.2%, mean estimated task-model cost per trial $1.01 Gemini 4 Argon (memory effort high; task Opus 5.5 (low)): Full-pass rate 44.4%, mean estimated task-model cost per trial $0.28 Gemini 4 Argon (memory effort high; task Opus 5.5 (medium)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.57 Gemini 4 Argon (memory effort high; task Opus 5.5 (high)): Full-pass rate 64.4%, mean estimated task-model cost per trial $0.75 Gemini 4 Argon (memory effort high; task Opus 5.5 (xhigh)): Full-pass rate 66.7%, mean estimated task-model cost per trial $1.46 Gemini 4 Argon (memory effort high; task Opus 5.5 (max)): Full-pass rate 64.4%, mean estimated task-model cost per trial $3.78 Gemini 4 Argon (memory effort high; task GPT-6 Sol (low)): Full-pass rate 46.7%, mean estimated task-model cost per trial $0.29 Gemini 4 Argon (memory effort high; task GPT-6 Sol (medium)): Full-pass rate 55.6%, mean estimated task-model cost per trial $0.44 Gemini 4 Argon (memory effort high; task GPT-6 Sol (high)): Full-pass rate 57.8%, mean estimated task-model cost per trial $0.58 Gemini 4 Argon (memory effort high; task GPT-6 Sol (xhigh)): Full-pass rate 66.7%, mean estimated task-model cost per trial $0.74 Gemini 4 Argon (memory effort high; task GPT-6 Sol (max)): Full-pass rate 64.4%, mean estimated task-model cost per trial $1.05 Muse Wonder (memory effort xhigh; task Opus 5.5 (low)): Full-pass rate 28.9%, mean estimated task-model cost per trial $0.28 Muse Wonder (memory effort xhigh; task Opus 5.5 (medium)): Full-pass rate 31.1%, mean estimated task-model cost per trial $0.51 Muse Wonder (memory effort xhigh; task Opus 5.5 (high)): Full-pass rate 33.3%, mean estimated task-model cost per trial $0.64 Muse Wonder (memory effort xhigh; task Opus 5.5 (xhigh)): Full-pass rate 40%, mean estimated task-model cost per trial $1.32 Muse Wonder (memory effort xhigh; task Opus 5.5 (max)): Full-pass rate 51.1%, mean estimated task-model cost per trial $3.56 Muse Wonder (memory effort xhigh; task GPT-6 Sol (low)): Full-pass rate 20%, mean estimated task-model cost per trial $0.23 Muse Wonder (memory effort xhigh; task GPT-6 Sol (medium)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.4 Muse Wonder (memory effort xhigh; task GPT-6 Sol (high)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.52 Muse Wonder (memory effort xhigh; task GPT-6 Sol (xhigh)): Full-pass rate 26.7%, mean estimated task-model cost per trial $0.74 Muse Wonder (memory effort xhigh; task GPT-6 Sol (max)): Full-pass rate 26.7%, mean estimated task-model cost per trial $1.09 Kimi K3 (memory effort max; task Opus 5.5 (xhigh)): Full-pass rate 71.1%, mean estimated task-model cost per trial $1.49 GLM-5.3 (memory effort max; task GPT-6 Sol (low)): Full-pass rate 37.8%, mean estimated task-model cost per trial $0.23 Grok 4.7 (memory effort xhigh; task GPT-6 Sol (low)): Full-pass rate 20%, mean estimated task-model cost per trial $0.22 Claude Opus 5.5 (memory effort max; task Opus 5.5 (medium)): Full-pass rate 64.4%, mean estimated task-model cost per trial $0.51 Claude Opus 5.5 (memory effort max; task Opus 5.5 (xhigh)): Full-pass rate 68.9%, mean estimated task-model cost per trial $1.31 Claude Opus 5.5 (memory effort max; task Opus 5.5 (max)): Full-pass rate 73.3%, mean estimated task-model cost per trial $3.42 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (low)): Full-pass rate 48.9%, mean estimated task-model cost per trial $0.27 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (medium)): Full-pass rate 62.2%, mean estimated task-model cost per trial $0.38 Claude Opus 5.5 (memory effort max; task GPT-6 Sol (high)): Full-pass rate 66.7%, mean estimated task-model cost per trial $0.55 Claude Opus 5.5 (memory effort max; task Opus 5.5 (max))Kimi K3 (memory effort max; task Opus 5.5 (xhigh))Claude Opus 5.5 (memory effort max; task Opus 5.5 (xhigh))Claude Opus 5.5 (memory effort max; task GPT-6 Sol (high))Claude Opus 5.5 (memory effort max; task Opus 5.5 (medium))Claude Opus 5.5 (memory effort max; task GPT-6 Sol (medium))Claude Opus 5.5 (memory effort max; task GPT-6 Sol (low))GLM-5.3 (memory effort max; task GPT-6 Sol (low))Grok 4.7 (memory effort xhigh; task GPT-6 Sol (low))
ModelSettingFull-pass rateMean estimated task-model cost per trial
Claude Opus 5.5 memory effort max; task Opus 5.5 (max), Letta Dream / Letta Code 73.3% $3.42
Kimi K3 memory effort max; task Opus 5.5 (xhigh), Letta Dream / Letta Code 71.1% $1.49
Kimi K3 memory effort max; task Opus 5.5 (max), Letta Dream / Letta Code 71.1% $4.16
Claude Opus 5.5 memory effort max; task Opus 5.5 (xhigh), Letta Dream / Letta Code 68.9% $1.31
GLM-5.3 memory effort max; task Opus 5.5 (xhigh), Letta Dream / Letta Code 68.9% $1.33
GLM-5.3 memory effort max; task Opus 5.5 (max), Letta Dream / Letta Code 68.9% $4.08
Claude Opus 5.5 memory effort max; task GPT-6 Sol (high), Letta Dream / Letta Code 66.7% $0.55
Claude Opus 5.5 memory effort max; task Opus 5.5 (high), Letta Dream / Letta Code 66.7% $0.63
Gemini 4 Argon memory effort high; task GPT-6 Sol (xhigh), Letta Dream / Letta Code 66.7% $0.74
Gemini 4 Argon memory effort high; task Opus 5.5 (xhigh), Letta Dream / Letta Code 66.7% $1.46
Claude Opus 5.5 memory effort max; task Opus 5.5 (medium), Letta Dream / Letta Code 64.4% $0.51
Gemini 4 Argon memory effort high; task Opus 5.5 (high), Letta Dream / Letta Code 64.4% $0.75
Claude Opus 5.5 memory effort max; task GPT-6 Sol (max), Letta Dream / Letta Code 64.4% $0.98
Gemini 4 Argon memory effort high; task GPT-6 Sol (max), Letta Dream / Letta Code 64.4% $1.05
Gemini 4 Argon memory effort high; task Opus 5.5 (max), Letta Dream / Letta Code 64.4% $3.78
Claude Opus 5.5 memory effort max; task GPT-6 Sol (medium), Letta Dream / Letta Code 62.2% $0.38
Gemini 4 Argon memory effort high; task Opus 5.5 (medium), Letta Dream / Letta Code 62.2% $0.57
Claude Opus 5.5 memory effort max; task GPT-6 Sol (xhigh), Letta Dream / Letta Code 62.2% $0.68
Kimi K3 memory effort max; task GPT-6 Sol (xhigh), Letta Dream / Letta Code 62.2% $0.72
Kimi K3 memory effort max; task GPT-6 Sol (max), Letta Dream / Letta Code 62.2% $0.96
Kimi K3 memory effort max; task Opus 5.5 (high), Letta Dream / Letta Code 60% $0.7
Kimi K3 memory effort max; task Opus 5.5 (medium), Letta Dream / Letta Code 57.8% $0.56
Kimi K3 memory effort max; task GPT-6 Sol (high), Letta Dream / Letta Code 57.8% $0.57
Gemini 4 Argon memory effort high; task GPT-6 Sol (high), Letta Dream / Letta Code 57.8% $0.58
GLM-5.3 memory effort max; task Opus 5.5 (high), Letta Dream / Letta Code 57.8% $0.67
Gemini 4 Argon memory effort high; task GPT-6 Sol (medium), Letta Dream / Letta Code 55.6% $0.44
GLM-5.3 memory effort max; task Opus 5.5 (medium), Letta Dream / Letta Code 51.1% $0.51
GLM-5.3 memory effort max; task GPT-6 Sol (xhigh), Letta Dream / Letta Code 51.1% $0.67
Muse Wonder memory effort xhigh; task Opus 5.5 (max), Letta Dream / Letta Code 51.1% $3.56
GPT-6.1 Sol memory effort max; task Opus 5.5 (max), Letta Dream / Letta Code 51.1% $4
Claude Opus 5.5 memory effort max; task GPT-6 Sol (low), Letta Dream / Letta Code 48.9% $0.27
Claude Opus 5.5 memory effort max; task Opus 5.5 (low), Letta Dream / Letta Code 48.9% $0.28
GLM-5.3 memory effort max; task GPT-6 Sol (max), Letta Dream / Letta Code 48.9% $1
Gemini 4 Argon memory effort high; task GPT-6 Sol (low), Letta Dream / Letta Code 46.7% $0.29
Kimi K3 memory effort max; task Opus 5.5 (low), Letta Dream / Letta Code 46.7% $0.3
GLM-5.3 memory effort max; task GPT-6 Sol (medium), Letta Dream / Letta Code 46.7% $0.38
Gemini 4 Argon memory effort high; task Opus 5.5 (low), Letta Dream / Letta Code 44.4% $0.28
Kimi K3 memory effort max; task GPT-6 Sol (medium), Letta Dream / Letta Code 44.4% $0.39
GLM-5.3 memory effort max; task GPT-6 Sol (high), Letta Dream / Letta Code 44.4% $0.52
GPT-6.1 Sol memory effort max; task GPT-6 Sol (xhigh), Letta Dream / Letta Code 44.4% $0.67
GPT-6.1 Sol memory effort max; task Opus 5.5 (xhigh), Letta Dream / Letta Code 44.4% $1.46
GPT-6.1 Sol memory effort max; task GPT-6 Sol (max), Letta Dream / Letta Code 42.2% $1.01
Kimi K3 memory effort max; task GPT-6 Sol (low), Letta Dream / Letta Code 40% $0.3
GLM-5.3 memory effort max; task Opus 5.5 (low), Letta Dream / Letta Code 40% $0.3
GPT-6.1 Sol memory effort max; task Opus 5.5 (high), Letta Dream / Letta Code 40% $0.7
Muse Wonder memory effort xhigh; task Opus 5.5 (xhigh), Letta Dream / Letta Code 40% $1.32
GLM-5.3 memory effort max; task GPT-6 Sol (low), Letta Dream / Letta Code 37.8% $0.23
GPT-6.1 Sol memory effort max; task GPT-6 Sol (high), Letta Dream / Letta Code 37.8% $0.56
Grok 4.7 memory effort xhigh; task Opus 5.5 (max), Letta Dream / Letta Code 37.8% $3.7
GPT-6.1 Sol memory effort max; task GPT-6 Sol (low), Letta Dream / Letta Code 33.3% $0.23
GPT-6.1 Sol memory effort max; task Opus 5.5 (low), Letta Dream / Letta Code 33.3% $0.3
GPT-6.1 Sol memory effort max; task GPT-6 Sol (medium), Letta Dream / Letta Code 33.3% $0.37
Grok 4.7 memory effort xhigh; task GPT-6 Sol (xhigh), Letta Dream / Letta Code 33.3% $0.63
Muse Wonder memory effort xhigh; task Opus 5.5 (high), Letta Dream / Letta Code 33.3% $0.64
Grok 4.7 memory effort xhigh; task GPT-6 Sol (medium), Letta Dream / Letta Code 31.1% $0.36
Muse Wonder memory effort xhigh; task Opus 5.5 (medium), Letta Dream / Letta Code 31.1% $0.51
GPT-6.1 Sol memory effort max; task Opus 5.5 (medium), Letta Dream / Letta Code 31.1% $0.56
Grok 4.7 memory effort xhigh; task GPT-6 Sol (max), Letta Dream / Letta Code 31.1% $1.02
Muse Wonder memory effort xhigh; task Opus 5.5 (low), Letta Dream / Letta Code 28.9% $0.28
Grok 4.7 memory effort xhigh; task Opus 5.5 (xhigh), Letta Dream / Letta Code 28.9% $1.64
Grok 4.7 memory effort xhigh; task Opus 5.5 (low), Letta Dream / Letta Code 26.7% $0.29
Muse Wonder memory effort xhigh; task GPT-6 Sol (medium), Letta Dream / Letta Code 26.7% $0.4
Grok 4.7 memory effort xhigh; task GPT-6 Sol (high), Letta Dream / Letta Code 26.7% $0.51
Muse Wonder memory effort xhigh; task GPT-6 Sol (high), Letta Dream / Letta Code 26.7% $0.52
Grok 4.7 memory effort xhigh; task Opus 5.5 (high), Letta Dream / Letta Code 26.7% $0.64
Muse Wonder memory effort xhigh; task GPT-6 Sol (xhigh), Letta Dream / Letta Code 26.7% $0.74
Muse Wonder memory effort xhigh; task GPT-6 Sol (max), Letta Dream / Letta Code 26.7% $1.09
Grok 4.7 memory effort xhigh; task Opus 5.5 (medium), Letta Dream / Letta Code 24.4% $0.49
Grok 4.7 memory effort xhigh; task GPT-6 Sol (low), Letta Dream / Letta Code 20% $0.22
Muse Wonder memory effort xhigh; task GPT-6 Sol (low), Letta Dream / Letta Code 20% $0.23

The best score, 73.3%, costs $3.42 a task

Runs on the cost frontier score higher than any run that's cheaper. Claude Opus 5.5 at memory effort max; task Opus 5.5 (xhigh) comes within 4.4 points of the best for $1.31 a task, 3 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Moonshot AI Z.ai xAI Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.10 $0.20 $0.50 $1.00 $2.00 $5.00 $10 Cost per task, log scale Claude Opus 5.5, memory effort max; task Opus 5.5 (max) Claude Opus 5.5, memory effort max; task Opus 5.5 (xhigh) Claude Opus 5.5, memory effort max; task Opus 5.5 (medium) Claude Opus 5.5, memory effort max; task GPT-6 Sol (low) GLM-5.3, memory effort max; task GPT-6 Sol (low) Grok 4.7, memory effort xhigh; task GPT-6 Sol (low) 0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 Cost per task, log scale Claude Opus 5.5, memory effort max; task Opus 5.5 (max) Claude Opus 5.5, memory effort max; task GPT-6 Sol (high)

Source: Official FrontierMemory leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 memory effort max; task Opus 5.5 (max) Letta Dream / Letta Code 73.3% $3.42
Kimi K3 memory effort max; task Opus 5.5 (xhigh) Letta Dream / Letta Code 71.1% $1.49
Kimi K3 memory effort max; task Opus 5.5 (max) Letta Dream / Letta Code 71.1% $4.16
GLM-5.3 memory effort max; task Opus 5.5 (xhigh) Letta Dream / Letta Code 68.9% $1.33
GLM-5.3 memory effort max; task Opus 5.5 (max) Letta Dream / Letta Code 68.9% $4.08
Claude Opus 5.5 memory effort max; task Opus 5.5 (xhigh) Letta Dream / Letta Code 68.9% $1.31
Claude Opus 5.5 memory effort max; task Opus 5.5 (high) Letta Dream / Letta Code 66.7% $0.63
Claude Opus 5.5 memory effort max; task GPT-6 Sol (high) Letta Dream / Letta Code 66.7% $0.55
Gemini 4 Argon memory effort high; task Opus 5.5 (xhigh) Letta Dream / Letta Code 66.7% $1.46
Gemini 4 Argon memory effort high; task GPT-6 Sol (xhigh) Letta Dream / Letta Code 66.7% $0.74
Claude Opus 5.5 memory effort max; task Opus 5.5 (medium) Letta Dream / Letta Code 64.4% $0.51
Claude Opus 5.5 memory effort max; task GPT-6 Sol (max) Letta Dream / Letta Code 64.4% $0.98
Claude Opus 5.5 memory effort max; task Opus 5.5 (max) Hermes Agent / Hermes Agent 64.4%
Gemini 4 Argon memory effort high; task Opus 5.5 (high) Letta Dream / Letta Code 64.4% $0.75
Gemini 4 Argon memory effort high; task Opus 5.5 (max) Letta Dream / Letta Code 64.4% $3.78
Gemini 4 Argon memory effort high; task GPT-6 Sol (max) Letta Dream / Letta Code 64.4% $1.05
Kimi K3 memory effort max; task GPT-6 Sol (xhigh) Letta Dream / Letta Code 62.2% $0.72
Kimi K3 memory effort max; task GPT-6 Sol (max) Letta Dream / Letta Code 62.2% $0.96
Claude Opus 5.5 memory effort max; task GPT-6 Sol (medium) Letta Dream / Letta Code 62.2% $0.38
Claude Opus 5.5 memory effort max; task GPT-6 Sol (xhigh) Letta Dream / Letta Code 62.2% $0.68
Gemini 4 Argon memory effort high; task Opus 5.5 (medium) Letta Dream / Letta Code 62.2% $0.57
Kimi K3 memory effort max; task Opus 5.5 (high) Letta Dream / Letta Code 60% $0.70
Kimi K3 memory effort max; task Opus 5.5 (medium) Letta Dream / Letta Code 57.8% $0.56
Kimi K3 memory effort max; task GPT-6 Sol (high) Letta Dream / Letta Code 57.8% $0.57
GLM-5.3 memory effort max; task Opus 5.5 (high) Letta Dream / Letta Code 57.8% $0.67
Gemini 4 Argon memory effort high; task GPT-6 Sol (high) Letta Dream / Letta Code 57.8% $0.58
Claude Opus 5.5 memory effort max; task Opus 5.5 (max) Claude Managed Agents / Claude Code 55.6%
Gemini 4 Argon memory effort high; task GPT-6 Sol (medium) Letta Dream / Letta Code 55.6% $0.44
Claude Opus 5.5 memory effort max; task Opus 5.5 (max) Prime Agent / Prime Agent 53.3%
GLM-5.3 memory effort max; task Opus 5.5 (medium) Letta Dream / Letta Code 51.1% $0.51
GLM-5.3 memory effort max; task GPT-6 Sol (xhigh) Letta Dream / Letta Code 51.1% $0.67
GPT-6.1 Sol memory effort max; task Opus 5.5 (max) Letta Dream / Letta Code 51.1% $4.00
Muse Wonder memory effort xhigh; task Opus 5.5 (max) Letta Dream / Letta Code 51.1% $3.56
GLM-5.3 memory effort max; task GPT-6 Sol (max) Letta Dream / Letta Code 48.9% $1.00
Claude Opus 5.5 memory effort max; task Opus 5.5 (low) Letta Dream / Letta Code 48.9% $0.28
Claude Opus 5.5 memory effort max; task GPT-6 Sol (low) Letta Dream / Letta Code 48.9% $0.27
Kimi K3 memory effort max; task Opus 5.5 (low) Letta Dream / Letta Code 46.7% $0.30
GLM-5.3 memory effort max; task GPT-6 Sol (medium) Letta Dream / Letta Code 46.7% $0.38
Gemini 4 Argon memory effort high; task GPT-6 Sol (low) Letta Dream / Letta Code 46.7% $0.29
Kimi K3 memory effort max; task GPT-6 Sol (medium) Letta Dream / Letta Code 44.4% $0.39
GLM-5.3 memory effort max; task GPT-6 Sol (high) Letta Dream / Letta Code 44.4% $0.52
GPT-6.1 Sol memory effort max; task Opus 5.5 (xhigh) Letta Dream / Letta Code 44.4% $1.46
GPT-6.1 Sol memory effort max; task GPT-6 Sol (xhigh) Letta Dream / Letta Code 44.4% $0.67
Gemini 4 Argon memory effort high; task Opus 5.5 (low) Letta Dream / Letta Code 44.4% $0.28
GPT-6.1 Sol memory effort max; task GPT-6 Sol (max) Letta Dream / Letta Code 42.2% $1.01
Kimi K3 memory effort max; task GPT-6 Sol (low) Letta Dream / Letta Code 40% $0.30
GLM-5.3 memory effort max; task Opus 5.5 (low) Letta Dream / Letta Code 40% $0.30
GPT-6.1 Sol memory effort max; task Opus 5.5 (high) Letta Dream / Letta Code 40% $0.70
Muse Wonder memory effort xhigh; task Opus 5.5 (xhigh) Letta Dream / Letta Code 40% $1.32
GLM-5.3 memory effort max; task GPT-6 Sol (low) Letta Dream / Letta Code 37.8% $0.23
Grok 4.7 memory effort xhigh; task Opus 5.5 (max) Letta Dream / Letta Code 37.8% $3.70
GPT-6.1 Sol memory effort max; task GPT-6 Sol (high) Letta Dream / Letta Code 37.8% $0.56
Grok 4.7 memory effort xhigh; task GPT-6 Sol (xhigh) Letta Dream / Letta Code 33.3% $0.63
GPT-6.1 Sol memory effort max; task Opus 5.5 (low) Letta Dream / Letta Code 33.3% $0.30
GPT-6.1 Sol memory effort max; task GPT-6 Sol (low) Letta Dream / Letta Code 33.3% $0.23
GPT-6.1 Sol memory effort max; task GPT-6 Sol (medium) Letta Dream / Letta Code 33.3% $0.37
Muse Wonder memory effort xhigh; task Opus 5.5 (high) Letta Dream / Letta Code 33.3% $0.64
Grok 4.7 memory effort xhigh; task GPT-6 Sol (medium) Letta Dream / Letta Code 31.1% $0.36
Grok 4.7 memory effort xhigh; task GPT-6 Sol (max) Letta Dream / Letta Code 31.1% $1.02
GPT-6.1 Sol memory effort max; task Opus 5.5 (medium) Letta Dream / Letta Code 31.1% $0.56
Muse Wonder memory effort xhigh; task Opus 5.5 (medium) Letta Dream / Letta Code 31.1% $0.51
Grok 4.7 memory effort xhigh; task Opus 5.5 (xhigh) Letta Dream / Letta Code 28.9% $1.64
Muse Wonder memory effort xhigh; task Opus 5.5 (low) Letta Dream / Letta Code 28.9% $0.28
Grok 4.7 memory effort xhigh; task Opus 5.5 (low) Letta Dream / Letta Code 26.7% $0.29
Grok 4.7 memory effort xhigh; task Opus 5.5 (high) Letta Dream / Letta Code 26.7% $0.64
Grok 4.7 memory effort xhigh; task GPT-6 Sol (high) Letta Dream / Letta Code 26.7% $0.51
GPT-6.1 Sol memory effort max; task GPT-6 Sol (max) OpenAI Agents SDK / Codex 26.7%
Muse Wonder memory effort xhigh; task GPT-6 Sol (medium) Letta Dream / Letta Code 26.7% $0.40
Muse Wonder memory effort xhigh; task GPT-6 Sol (high) Letta Dream / Letta Code 26.7% $0.52
Muse Wonder memory effort xhigh; task GPT-6 Sol (xhigh) Letta Dream / Letta Code 26.7% $0.74
Muse Wonder memory effort xhigh; task GPT-6 Sol (max) Letta Dream / Letta Code 26.7% $1.09
Grok 4.7 memory effort xhigh; task Opus 5.5 (medium) Letta Dream / Letta Code 24.4% $0.49
Grok 4.7 memory effort xhigh; task GPT-6 Sol (low) Letta Dream / Letta Code 20% $0.22
Muse Wonder memory effort xhigh; task GPT-6 Sol (low) Letta Dream / Letta Code 20% $0.23
GPT-6.1 Sol memory effort max; task GPT-6 Sol (max) Prime Agent / Prime Agent 13.3%
GPT-6.1 Sol memory effort max; task GPT-6 Sol (max) Hermes Agent / Hermes Agent 4.40%

2 models have held number one on the official leaderboard since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Moonshot AI Anthropic
Own leaderboard 2 leaders
Claude Opus 5.5 73.3%

Source: Official FrontierMemory leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% Near the ceiling (85%) Claude Opus 5.5 Kimi K3 0% 20% 40% 60% 80% 100% Near the ceiling (85%) Claude Opus 5.5

Source: Official FrontierMemory leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in AI research

All 104 in AI research
  • PaperBench In use Agent replicates ICML 2024 papers from scratch. LLM judge grades against author-written hierarchical rubrics. Qwen 3.8 Max 93%
  • MLS-Bench In use Agent must invent one algorithmic change in an ML research scaffold whose gain transfers across settings. Scored relative to human baselines. Claude Fable 5.1 50.3%
  • PostTrainBench 1.2 In use CLI agent gets a base model, one H100 and a time budget to post-train it for a target benchmark. Scored on normalized gain. Claude Fable 5.1 44.6%
  • MLE-bench In use Kaggle-style ML engineering competitions. Agent trains models and is scored against competition leaderboards. GPT-6 Astra 93.8%
  • KernelBench In use Replace PyTorch operators with faster custom GPU kernels. Scored on correctness and speedup. MiniMax-M3 28.8%
  • PMPP-Hard In use Agents write GPU kernels for 69 tasks that must be correct, and on 48 of them also run within 1.25 times the reference time. The score is the area under the solved-tasks curve over a 10-hour budget, out of 690 task-hours. Inkling 2 Small 494.3