Benchmark Almanac

PinchBench 2.0

Claude Opus 4.8 93.5% leads since May 2026
Not in model cards

Models drive an OpenClaw agent through 148 real-world tasks, including data and log analysis, meeting notes, DevOps, PDFs, research and writing. Graded by checks and a Haiku judge.

Up from about 25 tasks in v1.x. The default judge changed to Claude Haiku, so scores do not compare with v1. Released May 6th, 2026.

Made by
Kilo AI
Released
May 2026
Kind
General assistants
Level
Everyday
Scored
Graded by a model
Answers
Agent task
Tasks
All public
Strongest model tested
Claude Fable 5 #11 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
60
Entries on its own leaderboard, 57 models we track

Claude Opus 4.8 leads with 93.5%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

60 entries. Scroll sideways for the rest.

  1. 93.5 Claude Opus 4.8
  2. 92.5 Qwen3.7-Max
  3. 90.3 Nemotron 3.5 Bolt 74B-A9B
  4. 89.9 Nemotron 3 Ultra
  5. 89.7 MiMo-V2.5
  6. 88.9 Grok Build 0.1
  7. 88.7 GPT-5.6 Luna
  8. 88.1 Qwen 3.6 Flash
  9. 87.8 nemotron-3.5-bolt-74b-a9b-24gb
  10. 87.5 MiMo-V2.5-Pro
  11. 87.0 GLM-5.2
  12. 86.4 Nemotron 3.5 Lightning 30B-A3B
  13. 84.2 GPT-5.6 Sol
  14. 82.6 Ling-2.6-1T
  15. 81.7 DeepSeek-V4-Flash
  16. 81.0 Gemini 3.1 Pro
  17. 80.5 Gemini 3.1 Flash-Lite
  18. 80.3 Grok 4.20
  19. 79.3 Step 3.5 Flash
  20. 79.2 GPT-5.4 Mini
  21. 76.1 Kimi K2.7 Code
  22. 76.0 Claude Opus 4.7
  23. 75.9 GPT-5.6 Terra
  24. 75.7 GPT-5.4
  25. 75.5 GPT-5.5
  26. 75.2 Grok 4.5
  27. 75.0 Seed2.0 Lite (0215)
  28. 74.2 Gemini 3.5 Flash
  29. 73.7 Grok 4.3
  30. 72.5 Qwen 3.6 Plus
  31. 72.1 Gemini 3 Flash
  32. 71.8 GLM-5-Turbo
  33. 69.9 Claude Opus 4.6
  34. 69.5 fugu-ultra
  35. 69.4 Devstral 2
  36. 69.0 GPT-5.4 Nano
  37. 67.7 Claude Haiku 4.5
  38. 67.6 GLM-5V-Turbo
  39. 66.8 MiniMax-M2.7
  40. 65.7 Trinity Large Thinking
  41. 64.6 Mistral Small 4
  42. 62.7 Claude Sonnet 4.6
  43. 61.1 aion-3.0
  44. 61.1 DeepSeek-V4-Pro
  45. 60.0 GLM-5.1
  46. 56.4 Gemma 4 26B A4B
  47. 54.8 Claude Fable 5
  48. 54.6 Kimi K2.5
  49. 54.5 Mistral Large 3
  50. 53.0 Trinity Large Preview
  51. 52.7 Gemma 4 31B IT
  52. 48.8 Claude Sonnet 4
  53. 44.8 gpt-oss-120b
  54. 42.2 Nemotron 3 Super
  55. 39.5 Mercury 2
  56. 37.6 Nova 2 Lite
  57. 36.3 gpt-oss-20b
  58. 21.4 GPT-5.5 Pro
  59. 10.7 Llama 3.1-70B
  60. 3.19 Llama 4 Scout
  1. 1 Claude Opus 4.8 fast mode 93.5%
  2. 2 Qwen3.7-Max 92.5%
  3. 3 Nemotron 3.5 Bolt 74B-A9B 90.3%
  4. 4 Nemotron 3 Ultra 89.9%
  5. 5 MiMo-V2.5 89.7%
  6. 6 Grok Build 0.1 88.9%
  7. 7 GPT-5.6 Luna 88.7%
  8. 8 Qwen 3.6 Flash 88.1%
  9. 9 nemotron-3.5-bolt-74b-a9b-24gb 87.8%
  10. 10 MiMo-V2.5-Pro 87.5%
  11. 11 GLM-5.2 87%
  12. 12 Nemotron 3.5 Lightning 30B-A3B 86.4%
  13. 13 GPT-5.6 Sol 84.2%
  14. 14 Ling-2.6-1T 82.6%
  15. 15 DeepSeek-V4-Flash 81.7%
  16. 16 Gemini 3.1 Pro 81%
  17. 17 Gemini 3.1 Flash-Lite 80.5%
  18. 18 Grok 4.20 80.3%
  19. 19 Step 3.5 Flash 79.3%
  20. 20 GPT-5.4 Mini 79.2%
  21. 21 Kimi K2.7 Code 76.1%
  22. 22 Claude Opus 4.7 76%
  23. 23 GPT-5.6 Terra 75.9%
  24. 24 GPT-5.4 75.7%
  25. 25 GPT-5.5 75.5%
  26. 26 Grok 4.5 75.2%
  27. 27 Seed2.0 Lite (0215) 75%
  28. 28 Gemini 3.5 Flash 74.2%
  29. 29 Grok 4.3 73.7%
  30. 30 Qwen 3.6 Plus 72.5%
  31. 31 Gemini 3 Flash 72.1%
  32. 32 GLM-5-Turbo 71.8%
  33. 33 Claude Opus 4.6 69.9%
  34. 34 fugu-ultra 69.5%
  35. 35 Devstral 2 69.4%
  36. 36 GPT-5.4 Nano 69%
  37. 37 Claude Haiku 4.5 67.7%
  38. 38 GLM-5V-Turbo 67.6%
  39. 39 MiniMax-M2.7 66.8%
  40. 40 Trinity Large Thinking 65.7%
  41. 41 Mistral Small 4 64.6%
  42. 42 Claude Sonnet 4.6 62.7%
  43. 43 aion-3.0 61.1%
  44. 44 DeepSeek-V4-Pro 61.1%
  45. 45 GLM-5.1 60%
  46. 46 Gemma 4 26B A4B 56.4%
  47. 47 Claude Fable 5 54.8%
  48. 48 Kimi K2.5 54.6%
  49. 49 Mistral Large 3 54.5%
  50. 50 Trinity Large Preview 53%
  51. 51 Gemma 4 31B IT 52.7%
  52. 52 Claude Sonnet 4 48.8%
  53. 53 gpt-oss-120b 44.8%
  54. 54 Nemotron 3 Super 42.2%
  55. 55 Mercury 2 39.5%
  56. 56 Nova 2 Lite 37.6%
  57. 57 gpt-oss-20b 36.3%
  58. 58 GPT-5.5 Pro 21.4%
  59. 59 Llama 3.1-70B 10.7%
  60. 60 Llama 4 Scout 3.19%

Source: Official PinchBench 2.0 leaderboard

#EntryScoreSettingDateSource
1 Claude Opus 4.8 93.5% fast mode, OpenClaw May 2026 Official leaderboard
2 Qwen3.7-Max 92.5% OpenClaw May 2026 Official leaderboard
3 Nemotron 3.5 Bolt 74B-A9B 90.3% OpenClaw Oct 2026 Official leaderboard
4 Nemotron 3 Ultra 89.9% OpenClaw Jun 2026 Official leaderboard
5 MiMo-V2.5 89.7% OpenClaw Apr 2026 Official leaderboard
6 Grok Build 0.1 88.9% OpenClaw May 2026 Official leaderboard
7 GPT-5.6 Luna 88.7% OpenClaw Jul 2026 Official leaderboard
8 Qwen 3.6 Flash 88.1% OpenClaw Apr 2026 Official leaderboard
9 nemotron-3.5-bolt-74b-a9b-24gb 87.8% OpenClaw Oct 2026 Official leaderboard
10 MiMo-V2.5-Pro 87.5% OpenClaw Apr 2026 Official leaderboard
11 GLM-5.2 87% OpenClaw Jun 2026 Official leaderboard
12 Nemotron 3.5 Lightning 30B-A3B 86.4% OpenClaw Aug 2026 Official leaderboard
13 GPT-5.6 Sol 84.2% OpenClaw Jul 2026 Official leaderboard
14 Ling-2.6-1T 82.6% OpenClaw Apr 2026 Official leaderboard
15 DeepSeek-V4-Flash 81.7% OpenClaw Apr 2026 Official leaderboard
16 Gemini 3.1 Pro 81% OpenClaw Feb 2026 Official leaderboard
17 Gemini 3.1 Flash-Lite 80.5% OpenClaw Mar 2026 Official leaderboard
18 Grok 4.20 80.3% OpenClaw Feb 2026 Official leaderboard
19 Step 3.5 Flash 79.3% OpenClaw Feb 2026 Official leaderboard
20 GPT-5.4 Mini 79.2% OpenClaw Mar 2026 Official leaderboard
21 Kimi K2.7 Code 76.1% OpenClaw Jun 2026 Official leaderboard
22 Claude Opus 4.7 76% OpenClaw Apr 2026 Official leaderboard
23 GPT-5.6 Terra 75.9% OpenClaw Jul 2026 Official leaderboard
24 GPT-5.4 75.7% OpenClaw Mar 2026 Official leaderboard
25 GPT-5.5 75.5% OpenClaw Apr 2026 Official leaderboard
26 Grok 4.5 75.2% OpenClaw Jul 2026 Official leaderboard
27 Seed2.0 Lite (0215) 75% OpenClaw Feb 2026 Official leaderboard
28 Gemini 3.5 Flash 74.2% OpenClaw May 2026 Official leaderboard
29 Grok 4.3 73.7% OpenClaw Apr 2026 Official leaderboard
30 Qwen 3.6 Plus 72.5% OpenClaw Mar 2026 Official leaderboard
31 Gemini 3 Flash 72.1% OpenClaw Dec 2025 Official leaderboard
32 GLM-5-Turbo 71.8% OpenClaw Mar 2026 Official leaderboard
33 Claude Opus 4.6 69.9% OpenClaw Feb 2026 Official leaderboard
34 fugu-ultra 69.5% OpenClaw Jun 2026 Official leaderboard
35 Devstral 2 69.4% OpenClaw Dec 2025 Official leaderboard
36 GPT-5.4 Nano 69% OpenClaw Mar 2026 Official leaderboard
37 Claude Haiku 4.5 67.7% OpenClaw Oct 2025 Official leaderboard
38 GLM-5V-Turbo 67.6% OpenClaw Apr 2026 Official leaderboard
39 MiniMax-M2.7 66.8% OpenClaw Mar 2026 Official leaderboard
40 Trinity Large Thinking 65.7% OpenClaw Apr 2026 Official leaderboard
41 Mistral Small 4 64.6% OpenClaw Mar 2026 Official leaderboard
42 Claude Sonnet 4.6 62.7% OpenClaw Feb 2026 Official leaderboard
43 aion-3.0 61.1% OpenClaw Jul 2026 Official leaderboard
44 DeepSeek-V4-Pro 61.1% OpenClaw Apr 2026 Official leaderboard
45 GLM-5.1 60% OpenClaw Apr 2026 Official leaderboard
46 Gemma 4 26B A4B 56.4% OpenClaw Apr 2026 Official leaderboard
47 Claude Fable 5 54.8% OpenClaw Jun 2026 Official leaderboard
48 Kimi K2.5 54.6% OpenClaw Jan 2026 Official leaderboard
49 Mistral Large 3 54.5% OpenClaw Dec 2025 Official leaderboard
50 Trinity Large Preview 53% OpenClaw Jan 2026 Official leaderboard
51 Gemma 4 31B IT 52.7% OpenClaw Apr 2026 Official leaderboard
52 Claude Sonnet 4 48.8% OpenClaw May 2025 Official leaderboard
53 gpt-oss-120b 44.8% OpenClaw Aug 2025 Official leaderboard
54 Nemotron 3 Super 42.2% OpenClaw Mar 2026 Official leaderboard
55 Mercury 2 39.5% OpenClaw Feb 2026 Official leaderboard
56 Nova 2 Lite 37.6% OpenClaw Dec 2025 Official leaderboard
57 gpt-oss-20b 36.3% OpenClaw Aug 2025 Official leaderboard
58 GPT-5.5 Pro 21.4% OpenClaw Apr 2026 Official leaderboard
59 Llama 3.1-70B 10.7% OpenClaw Jul 2024 Official leaderboard
60 Llama 4 Scout 3.19% OpenClaw Apr 2025 Official leaderboard

Success Rate vs. Execution Time

Each point is one model: its average score against its average execution time for a full run, over its official submissions. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0% 20% 40% 60% 80% 100% 1,00010K100K Execution time per run (s), log scale, lower is better Success rate Llama 3.1-70B: Success rate 10.7%, execution time per run 16K s Llama 4 Scout: Success rate 3.19%, execution time per run 14K s Claude Sonnet 4: Success rate 48.8%, execution time per run 21K s gpt-oss-120b: Success rate 44.8%, execution time per run 11K s gpt-oss-20b: Success rate 36.3%, execution time per run 6,665 s Claude Haiku 4.5: Success rate 67.7%, execution time per run 14K s Mistral Large 3: Success rate 54.5%, execution time per run 16K s Nova 2 Lite: Success rate 37.6%, execution time per run 10K s Devstral 2: Success rate 69.4%, execution time per run 15K s Gemini 3 Flash: Success rate 72.1%, execution time per run 12K s Kimi K2.5: Success rate 54.6%, execution time per run 22K s Trinity Large Preview: Success rate 53%, execution time per run 13K s Step 3.5 Flash: Success rate 79.3%, execution time per run 15K s Claude Opus 4.6: Success rate 69.9%, execution time per run 14K s Seed2.0 Lite (0215): Success rate 75%, execution time per run 15K s Claude Sonnet 4.6: Success rate 62.7%, execution time per run 19K s Grok 4.20: Success rate 80.3%, execution time per run 12K s Gemini 3.1 Pro: Success rate 81%, execution time per run 10K s GPT-5.4: Success rate 75.7%, execution time per run 16K s Nemotron 3 Super: Success rate 42.2%, execution time per run 22K s GLM-5-Turbo: Success rate 71.8%, execution time per run 17K s Mistral Small 4: Success rate 64.6%, execution time per run 16K s GPT-5.4 Mini: Success rate 79.2%, execution time per run 12K s GPT-5.4 Nano: Success rate 69%, execution time per run 14K s MiniMax-M2.7: Success rate 66.8%, execution time per run 18K s Qwen 3.6 Plus: Success rate 72.5%, execution time per run 18K s GLM-5V-Turbo: Success rate 67.6%, execution time per run 18K s Trinity Large Thinking: Success rate 65.7%, execution time per run 13K s Gemma 4 26B A4B: Success rate 56.4%, execution time per run 18K s Gemma 4 31B IT: Success rate 52.7%, execution time per run 21K s GLM-5.1: Success rate 60%, execution time per run 19K s Claude Opus 4.7: Success rate 76%, execution time per run 16K s GPT-5.5: Success rate 75.5%, execution time per run 17K s GPT-5.5 Pro: Success rate 21.4%, execution time per run 28K s MiMo-V2.5: Success rate 89.7%, execution time per run 12K s MiMo-V2.5-Pro: Success rate 87.5%, execution time per run 15K s DeepSeek-V4-Flash: Success rate 81.7%, execution time per run 17K s DeepSeek-V4-Pro: Success rate 61.1%, execution time per run 23K s Qwen 3.6 Flash: Success rate 88.1%, execution time per run 13K s Ling-2.6-1T: Success rate 82.6%, execution time per run 14K s Grok 4.3: Success rate 73.7%, execution time per run 12K s Gemini 3.5 Flash: Success rate 74.2%, execution time per run 9,865 s Qwen3.7-Max: Success rate 92.5%, execution time per run 12K s Claude Opus 4.8: Success rate 90.5%, execution time per run 15K s Grok Build 0.1: Success rate 88.9%, execution time per run 13K s Nemotron 3 Ultra: Success rate 89.9%, execution time per run 9,034 s Claude Fable 5: Success rate 54.8%, execution time per run 13K s Kimi K2.7 Code: Success rate 76.1%, execution time per run 18K s GLM-5.2: Success rate 87%, execution time per run 16K s Grok 4.5: Success rate 75.2%, execution time per run 14K s GPT-5.6 Luna: Success rate 88.7%, execution time per run 10K s GPT-5.6 Sol: Success rate 84.2%, execution time per run 16K s GPT-5.6 Terra: Success rate 75.9%, execution time per run 9,122 s Nemotron 3.5 Lightning 30B-A3B: Success rate 86.4%, execution time per run 14K s nemotron-3.5-bolt-74b-a9b-24gb: Success rate 87.8%, execution time per run 10K s. Not in our model list fugu-ultra: Success rate 69.5%, execution time per run 27K s. Not in our model list aion-3.0: Success rate 61.1%, execution time per run 19K s. Not in our model list Mercury 2: Success rate 39.5%, execution time per run 2,467 s Gemini 3.1 Flash-Lite: Success rate 80.5%, execution time per run 5,634 s Claude Opus 4.8 (fast mode): Success rate 93.5%, execution time per run 9,169 s Nemotron 3.5 Bolt 74B-A9B: Success rate 90.3%, execution time per run 8,338 s Claude Opus 4.8 (fast mode)Nemotron 3.5 Bolt 74B-A9BGemini 3.1 Flash-LiteMercury 2 0% 20% 40% 60% 80% 100% 1,00010K100K Execution time per run (s), log scale, lower is better Success rate Llama 3.1-70B: Success rate 10.7%, execution time per run 16K s Llama 4 Scout: Success rate 3.19%, execution time per run 14K s Claude Sonnet 4: Success rate 48.8%, execution time per run 21K s gpt-oss-120b: Success rate 44.8%, execution time per run 11K s gpt-oss-20b: Success rate 36.3%, execution time per run 6,665 s Claude Haiku 4.5: Success rate 67.7%, execution time per run 14K s Mistral Large 3: Success rate 54.5%, execution time per run 16K s Nova 2 Lite: Success rate 37.6%, execution time per run 10K s Devstral 2: Success rate 69.4%, execution time per run 15K s Gemini 3 Flash: Success rate 72.1%, execution time per run 12K s Kimi K2.5: Success rate 54.6%, execution time per run 22K s Trinity Large Preview: Success rate 53%, execution time per run 13K s Step 3.5 Flash: Success rate 79.3%, execution time per run 15K s Claude Opus 4.6: Success rate 69.9%, execution time per run 14K s Seed2.0 Lite (0215): Success rate 75%, execution time per run 15K s Claude Sonnet 4.6: Success rate 62.7%, execution time per run 19K s Grok 4.20: Success rate 80.3%, execution time per run 12K s Gemini 3.1 Pro: Success rate 81%, execution time per run 10K s GPT-5.4: Success rate 75.7%, execution time per run 16K s Nemotron 3 Super: Success rate 42.2%, execution time per run 22K s GLM-5-Turbo: Success rate 71.8%, execution time per run 17K s Mistral Small 4: Success rate 64.6%, execution time per run 16K s GPT-5.4 Mini: Success rate 79.2%, execution time per run 12K s GPT-5.4 Nano: Success rate 69%, execution time per run 14K s MiniMax-M2.7: Success rate 66.8%, execution time per run 18K s Qwen 3.6 Plus: Success rate 72.5%, execution time per run 18K s GLM-5V-Turbo: Success rate 67.6%, execution time per run 18K s Trinity Large Thinking: Success rate 65.7%, execution time per run 13K s Gemma 4 26B A4B: Success rate 56.4%, execution time per run 18K s Gemma 4 31B IT: Success rate 52.7%, execution time per run 21K s GLM-5.1: Success rate 60%, execution time per run 19K s Claude Opus 4.7: Success rate 76%, execution time per run 16K s GPT-5.5: Success rate 75.5%, execution time per run 17K s GPT-5.5 Pro: Success rate 21.4%, execution time per run 28K s MiMo-V2.5: Success rate 89.7%, execution time per run 12K s MiMo-V2.5-Pro: Success rate 87.5%, execution time per run 15K s DeepSeek-V4-Flash: Success rate 81.7%, execution time per run 17K s DeepSeek-V4-Pro: Success rate 61.1%, execution time per run 23K s Qwen 3.6 Flash: Success rate 88.1%, execution time per run 13K s Ling-2.6-1T: Success rate 82.6%, execution time per run 14K s Grok 4.3: Success rate 73.7%, execution time per run 12K s Gemini 3.5 Flash: Success rate 74.2%, execution time per run 9,865 s Qwen3.7-Max: Success rate 92.5%, execution time per run 12K s Claude Opus 4.8: Success rate 90.5%, execution time per run 15K s Grok Build 0.1: Success rate 88.9%, execution time per run 13K s Nemotron 3 Ultra: Success rate 89.9%, execution time per run 9,034 s Claude Fable 5: Success rate 54.8%, execution time per run 13K s Kimi K2.7 Code: Success rate 76.1%, execution time per run 18K s GLM-5.2: Success rate 87%, execution time per run 16K s Grok 4.5: Success rate 75.2%, execution time per run 14K s GPT-5.6 Luna: Success rate 88.7%, execution time per run 10K s GPT-5.6 Sol: Success rate 84.2%, execution time per run 16K s GPT-5.6 Terra: Success rate 75.9%, execution time per run 9,122 s Nemotron 3.5 Lightning 30B-A3B: Success rate 86.4%, execution time per run 14K s nemotron-3.5-bolt-74b-a9b-24gb: Success rate 87.8%, execution time per run 10K s. Not in our model list fugu-ultra: Success rate 69.5%, execution time per run 27K s. Not in our model list aion-3.0: Success rate 61.1%, execution time per run 19K s. Not in our model list Mercury 2: Success rate 39.5%, execution time per run 2,467 s Gemini 3.1 Flash-Lite: Success rate 80.5%, execution time per run 5,634 s Claude Opus 4.8 (fast mode): Success rate 93.5%, execution time per run 9,169 s Nemotron 3.5 Bolt 74B-A9B: Success rate 90.3%, execution time per run 8,338 s Claude Opus 4.8 (fast mode)Nemotron 3.5 Bolt 74B-A9BGemini 3.1 Flash-LiteMercury 2
ModelSettingSuccess rateExecution time per run
Claude Opus 4.8 fast mode, OpenClaw 93.5% 9,169 s
Qwen3.7-Max OpenClaw 92.5% 12,005 s
Claude Opus 4.8 OpenClaw 90.5% 14,690 s
Nemotron 3.5 Bolt 74B-A9B OpenClaw 90.4% 8,338 s
Nemotron 3 Ultra OpenClaw 89.9% 9,034 s
MiMo-V2.5 OpenClaw 89.7% 11,850 s
Grok Build 0.1 OpenClaw 88.9% 13,240 s
GPT-5.6 Luna OpenClaw 88.7% 10,228 s
Qwen 3.6 Flash OpenClaw 88.1% 13,373 s
nemotron-3.5-bolt-74b-a9b-24gb OpenClaw 87.8% 10,267 s
MiMo-V2.5-Pro OpenClaw 87.5% 15,079 s
GLM-5.2 OpenClaw 87% 15,799 s
Nemotron 3.5 Lightning 30B-A3B OpenClaw 86.4% 14,308 s
GPT-5.6 Sol OpenClaw 84.2% 15,916 s
Ling-2.6-1T OpenClaw 82.6% 14,397 s
DeepSeek-V4-Flash OpenClaw 81.7% 16,898 s
Gemini 3.1 Pro OpenClaw 81% 10,214 s
Gemini 3.1 Flash-Lite OpenClaw 80.5% 5,634 s
Grok 4.20 OpenClaw 80.3% 12,311 s
Step 3.5 Flash OpenClaw 79.4% 14,567 s
GPT-5.4 Mini OpenClaw 79.2% 11,949 s
Kimi K2.7 Code OpenClaw 76.1% 17,703 s
Claude Opus 4.7 OpenClaw 76% 16,293 s
GPT-5.6 Terra OpenClaw 75.9% 9,122 s
GPT-5.4 OpenClaw 75.7% 16,351 s
GPT-5.5 OpenClaw 75.5% 16,968 s
Grok 4.5 OpenClaw 75.2% 13,686 s
Seed2.0 Lite (0215) OpenClaw 75% 14,715 s
Gemini 3.5 Flash OpenClaw 74.2% 9,865 s
Grok 4.3 OpenClaw 73.7% 11,985 s
Qwen 3.6 Plus OpenClaw 72.5% 17,879 s
Gemini 3 Flash OpenClaw 72.1% 11,836 s
GLM-5-Turbo OpenClaw 71.8% 17,229 s
Claude Opus 4.6 OpenClaw 69.9% 13,566 s
fugu-ultra OpenClaw 69.6% 26,792 s
Devstral 2 OpenClaw 69.4% 14,655 s
GPT-5.4 Nano OpenClaw 69% 14,266 s
Claude Haiku 4.5 OpenClaw 67.7% 13,982 s
GLM-5V-Turbo OpenClaw 67.6% 17,861 s
MiniMax-M2.7 OpenClaw 66.8% 18,336 s
Trinity Large Thinking OpenClaw 65.7% 13,237 s
Mistral Small 4 OpenClaw 64.6% 16,215 s
Claude Sonnet 4.6 OpenClaw 62.7% 19,441 s
aion-3.0 OpenClaw 61.1% 19,004 s
DeepSeek-V4-Pro OpenClaw 61.1% 22,509 s
GLM-5.1 OpenClaw 60% 18,666 s
Gemma 4 26B A4B OpenClaw 56.4% 17,892 s
Claude Fable 5 OpenClaw 54.8% 12,817 s
Kimi K2.5 OpenClaw 54.6% 21,748 s
Mistral Large 3 OpenClaw 54.5% 15,940 s
Trinity Large Preview OpenClaw 53% 13,096 s
Gemma 4 31B IT OpenClaw 52.7% 21,154 s
Claude Sonnet 4 OpenClaw 48.8% 20,635 s
gpt-oss-120b OpenClaw 44.8% 10,841 s
Nemotron 3 Super OpenClaw 42.2% 21,743 s
Mercury 2 OpenClaw 39.6% 2,467 s
Nova 2 Lite OpenClaw 37.6% 10,406 s
gpt-oss-20b OpenClaw 36.3% 6,665 s
GPT-5.5 Pro OpenClaw 21.4% 28,484 s
Llama 3.1-70B OpenClaw 10.7% 16,326 s
Llama 4 Scout OpenClaw 3.2% 13,564 s

The best score, 93.5%, costs $165 for the full run

Runs on the cost frontier score higher than any run that's cheaper. MiMo-V2.5 comes within 3.8 points of the best for $6.23 for the full run, 27 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Meta OpenAI StepFun Xiaomi Corp DeepSeek Qwen Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 $1,000 Cost of the full run, log scale Claude Opus 4.8, fast mode Qwen3.7-Max MiMo-V2.5 DeepSeek-V4-Flash DeepSeek-V4-Pro gpt-oss-120b gpt-oss-20b 0% 20% 40% 60% 80% 100% $0.10 $1.00 $10 $100 $1,000 Cost of the full run, log scale Claude Opus 4.8, fast mode DeepSeek-V4-Flash

Source: Official PinchBench 2.0 leaderboard

ModelEffortHarnessScoreCost of the full runOutput tokens
Claude Opus 4.8 fast mode OpenClaw 93.5% $165
Qwen3.7-Max OpenClaw 92.5% $21
Claude Opus 4.8 OpenClaw 90.5% $82
Nemotron 3.5 Bolt 74B-A9B OpenClaw 90.3%
Nemotron 3 Ultra OpenClaw 89.9%
MiMo-V2.5 OpenClaw 89.7% $6.23
Grok Build 0.1 OpenClaw 88.9% $21
GPT-5.6 Luna OpenClaw 88.7% $17
Qwen 3.6 Flash OpenClaw 88.1% $14
MiMo-V2.5-Pro OpenClaw 87.5% $12
GLM-5.2 OpenClaw 87% $19
Nemotron 3.5 Lightning 30B-A3B OpenClaw 86.4%
GPT-5.6 Sol OpenClaw 84.2% $69
Ling-2.6-1T OpenClaw 82.6% $7.00
DeepSeek-V4-Flash OpenClaw 81.7% $1.44
Gemini 3.1 Pro OpenClaw 81% $26
Gemini 3.1 Flash-Lite OpenClaw 80.5% $3.79
Grok 4.20 OpenClaw 80.3% $22
Step 3.5 Flash OpenClaw 79.3% $1.12
GPT-5.4 Mini OpenClaw 79.2% $4.87
Kimi K2.7 Code OpenClaw 76.1% $11
Claude Opus 4.7 OpenClaw 76% $63
GPT-5.6 Terra OpenClaw 75.9% $27
GPT-5.4 OpenClaw 75.7% $19
GPT-5.5 OpenClaw 75.5% $32
Grok 4.5 OpenClaw 75.2% $25
Seed2.0 Lite (0215) OpenClaw 75% $4.45
Gemini 3.5 Flash OpenClaw 74.2% $34
Grok 4.3 OpenClaw 73.7% $15
Qwen 3.6 Plus OpenClaw 72.5% $12
Gemini 3 Flash OpenClaw 72.1% $4.99
GLM-5-Turbo OpenClaw 71.8% $9.29
Claude Opus 4.6 OpenClaw 69.9% $47
Devstral 2 OpenClaw 69.4% $4.81
GPT-5.4 Nano OpenClaw 69% $1.41
Claude Haiku 4.5 OpenClaw 67.7% $12
GLM-5V-Turbo OpenClaw 67.6% $22
MiniMax-M2.7 OpenClaw 66.8% $3.11
Trinity Large Thinking OpenClaw 65.7% $3.05
Mistral Small 4 OpenClaw 64.6% $3.45
Claude Sonnet 4.6 OpenClaw 62.7% $31
DeepSeek-V4-Pro OpenClaw 61.1% $0.58
GLM-5.1 OpenClaw 60% $16
Gemma 4 26B A4B OpenClaw 56.4% $1.23
Claude Fable 5 OpenClaw 54.8% $123
Kimi K2.5 OpenClaw 54.6% $7.15
Mistral Large 3 OpenClaw 54.5% $6.57
Trinity Large Preview OpenClaw 53% $4.56
Gemma 4 31B IT OpenClaw 52.7% $1.84
Claude Sonnet 4 OpenClaw 48.8% $24
gpt-oss-120b OpenClaw 44.8% $0.50
Nemotron 3 Super OpenClaw 42.2%
Mercury 2 OpenClaw 39.5% $0.65
Nova 2 Lite OpenClaw 37.6% $4.67
gpt-oss-20b OpenClaw 36.3% $0.45
GPT-5.5 Pro OpenClaw 21.4% $239
Llama 3.1-70B OpenClaw 10.7% $5.00
Llama 4 Scout OpenClaw 3.19% $0.24

3 models have held number one on the official leaderboard since its release in May 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Xiaomi Corp Qwen Anthropic
Own leaderboard Passed 85% in May 2026, by MiMo-V2.5
Claude Opus 4.8 93.5%

Source: Official PinchBench 2.0 leaderboard

MiMo-V2.5 on the official leaderboard passed 85% in April 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Gemini 3.1 Pro Gemini 3 Flash Claude Haiku 4.5 Claude Sonnet 4 Llama 3.1-70B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Gemini 3.1 Pro

Source: Official PinchBench 2.0 leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Xiaomi Corp Arcee AI Kuaishou Qwen Anthropic
PinchBench 2.0 Passed 85% in May 2026, by MiMo-V2.5
Claude Opus 4.8 93.5%
PinchBench Passed 85% in Apr 2026, by Trinity Large Thinking
KAT-Coder-Pro V2.5 94.9%
PinchBench 2.0 Passed 85% in May 2026, by MiMo-V2.5
Claude Opus 4.8 93.5%

Sources: Official leaderboards and labs’ model cards

Replaces PinchBench

More in Tool use

All 144 in Tool use
  • AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state. Gemini 4 Carbon 53.9%
  • MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. Gemini 4 Argon 90.7%
  • BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes. Qwen3.7-Max 75%
  • τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. Qwen 3.8 Max 55.1%
  • Toolathlon-Verified In use 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. GLM-5.3-Flash 78.4%
  • DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%