Benchmark Almanac

Toloka AI

We track 2 benchmarks that Toloka AI made or helped make.

0 in use 2 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
U-MATH Models solve 1,100 unpublished open-ended university math problems from teaching materials, balanced across six subjects. About 20% include images, and an LLM judge checks each solution. Word problems Dec 2024 86.8% Not in cards
SemEval 2026 Task 4: Narrative Story Similarity and Narrative Representation Learning Models choose which story is more similar to a reference story. A second track compares story embeddings. Each track scores accuracy on 400 triples. Reading comprehension Jul 2025 Not in cards