Sakana AI
We track 10 benchmarks that Sakana AI made or helped make.
0 in use 7 with public code 5 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Sudoku-Bench Models solve Sudoku variants with unusual rule sets that need creative, multi-step deduction. Scored by the share of puzzles solved. | Abstract puzzles and logic | May 2025 | 14% | Dropped |
| ALE-Bench Agents iteratively write solutions to 40 hard optimization problems from AtCoder Heuristic Contests. Scores convert to an Elo-like performance rating against human contestants. | Competitive programming | Jun 2025 | 3,164 | Not in cards |
| JFinBench Models answer questions from six Japanese certification exams in accounting, tax, financial planning, and banking. The authors report scores by exam. | Finance | Nov 2026 | 97.3% | Not in cards |
| EDINET-Bench Models read Japanese financial disclosures to detect fraud, forecast earnings and identify industries. | Finance | Jun 2025 | 0.73 | Not in cards |
| ChouBun Models answer questions about long Japanese documents and summarize company securities reports. | Reasoning over long documents | Oct 2024 | | Not in cards |
| JA-VLM-Bench-In-the-Wild Models answer Japanese questions about images of places, objects and culture in Japan. | General visual questions | Mar 2024 | 41.1 | Not in cards |
| CoffeeBench Agents manage a coffee roaster and trade with other firms for 90 simulated days. | Running a business | Jun 2026 | Not in cards | |
| robust-kbench Agents write CUDA kernels that are tested across several weight initializations and input shapes, so shape and magnitude exploits fail. Correctness checks and profiling score each kernel. | GPU kernels | Sep 2025 | Not in cards | |
| Sakana Japanese-English translation evaluation Models translate Japanese text into English while handling Japan-specific proper nouns and honorifics. | Translation | Aug 2026 | Not in cards | |
| Sakana kana-letter reading-order set Model recovers the reading order of 25 expert-annotated classical Japanese kana letters (chirashigaki). Scored by normalized edit distance. | Documents, charts and text in images | Jun 2026 | Not in cards |