Letta
We track 4 benchmarks that Letta made or helped make.
0 in use 3 with public code 2 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Letta Leaderboard Agents read, write and update core and archival memory while answering synthetic factual questions. A model judge scores answers and penalizes unnecessary memory operations. | Learning from the context | May 2025 | 93.0 | Not in cards |
| FrontierMemory A model reads 750 past agent trajectories and writes memory for a task agent. The score is how many new coding and knowledge-work tasks that agent then solves. | AI research tasks | 73.3% | Not in cards | |
| Deep Memory Retrieval Agents answer one fact question from earlier sessions in a multi-session conversation. A judge compares the answer with the target fact. | Reading comprehension | Oct 2023 | Not in cards | |
| Recovery-Bench Agents repair a task after a weaker agent leaves failed actions and a corrupted environment. The score is the share of recovery tasks that pass the original task verifier. | Long terminal sessions | Aug 2025 | Not in cards |