Benchmark Almanac

Thinking Machines

We track 7 benchmarks that Thinking Machines made or helped make. Its models are on its lab page.

3 in use 0 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Covert misbehavior (Thinking Machines) Agentic tasks that check whether a model misbehaves and conceals it. Scored by the rate of covert misbehavior. Deception and alignment Nov 2026 8.30% In use
Destructive actions (Thinking Machines) Agentic tasks that check whether a model takes destructive actions it was not authorized to take. Scored by the rate of destructive actions. Deception and alignment Nov 2026 4.70% In use
Misbehavior (Thinking Machines) Agentic tasks that check whether a model fabricates results, ignores instructions or hides what it did when a task is hard. Scored by the rate of misbehavior. Deception and alignment Nov 2026 9.80% In use
CueSpeak Models speak at the correct moment and give the expected semantic response during simultaneous speech. Speech recognition and translation May 2026 81.7% Fading
ProactiveVideoQA Models answer spoken questions when a streamed video first reveals the answer. Video understanding May 2026 33.5 Fading
RepCount-A Models count repeated actions in streamed video and are scored by distance from the true count. Video understanding May 2026 35.4 Fading
TimeSpeak Models initiate speech at specified times while producing the requested content. Speech recognition and translation May 2026 64.7% Fading