Recognition of model-written user turns (Anthropic)
Models write synthetic user turns for real Claude Code transcripts, then judge which version is real. Score is accuracy above chance.
The same model acts as generator and judge. Anthropic reports the probability of a correct choice minus 0.5.
- Made by
- Anthropic
- Kind
- Other
- Level
- Expert
- Scored
- Checked automatically
- Answers
- Prediction
- Tasks
- Kept private
- Last in a system card
- April 2026
- Last model added
- April 2026
Claude Mythos Preview leads with 0.35
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
Source: Labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Claude Mythos Preview | 0.35 | Apr 2026 | Lab model card |
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Labs’ model cards
Labs report it only in system cards
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark. Some system cards do, below.
One system card reports it
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Apr 7, 2026 | Claude Mythos Preview system card | 0.35 | Model ability to distinguish its own simulated user turns from real user turns, probability of a correct choice minus 0.5, value printed on chart (7.7, Figure 7.7.A) | Other |
More in Other
- Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
- Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness.
Gemini 4 Carbon 59.0 - SAGE v1.1 (Vals) In use Models grade handwritten undergraduate math and computer science work against rubrics. Scored by class-balanced accuracy of each rubric check against expert grading. Claude Opus 5.5 91.3%
- MTEB Fading Massive Text Embedding Benchmark: retrieval, clustering, classification across languages and modalities.
sonar 42% - Anthropic ECI (AECI) Only in system cards Index that combines a basket of capability benchmarks into one capability score to track trends across Anthropic models. Claude Opus 5.5 169.4
- SPEED-Bench Only in system cards Measures speculative decoding across varied prompts and domains. Reports the average acceptance length of draft tokens, so higher means faster generation at the same output.
Nemotron 3 Ultra 5.15