Benchmark Almanac

Letta

We track 4 benchmarks that Letta made or helped make.

0 in use 3 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Letta Leaderboard Agents read, write and update core and archival memory while answering synthetic factual questions. A model judge scores answers and penalizes unnecessary memory operations. Learning from the context May 2025 93.0 Not in cards
FrontierMemory A model reads 750 past agent trajectories and writes memory for a task agent. The score is how many new coding and knowledge-work tasks that agent then solves. AI research tasks 73.3% Not in cards
Deep Memory Retrieval Agents answer one fact question from earlier sessions in a multi-session conversation. A judge compares the answer with the target fact. Reading comprehension Oct 2023 Not in cards
Recovery-Bench Agents repair a task after a weaker agent leaves failed actions and a corrupted environment. The score is the share of recovery tasks that pass the original task verifier. Long terminal sessions Aug 2025 Not in cards