Benchmark Almanac

tau2-bench

GPT-5.4 98.9% leads since March 2026
Gemini 3 Pro 85.4% leads since November 2025
GPT-5.4 98.9% leads since March 2026
In use

Dual-control customer-service simulations (retail, airline, telecom) where agent and simulated user both act. Graded on final state.

τ³-bench (v1.0.0, 2026-03) lives in the tau2-bench repo and adds banking_knowledge and voice modes. V1.0.1 fixed banking grading. Telecom scores depend on the user-simulator model.

Made by
Sierra Research
Released
June 2025
Kind
Customer service
Level
Everyday
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Strongest model tested
GPT-5.5 #20 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
Scores refreshed
November 12th, 2026
79
Models scored in lab cards
82
Launch posts list it, from 29 labs
40
System cards report it, from 21 labs

GPT-5.4 leads with 98.9%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Gemini 3 Pro leads with 85.4% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

GPT-5.4 leads with 98.9%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

79 models. Scroll sideways for the rest.

  1. 98.9 GPT-5.4
  2. 98.0 GPT-5.5
  3. 97.8 MiniMax-M2.5
  4. 96.8 MiMo-V2-Pro
  5. 95.6 Claude Opus 4.6
  6. 95.3 Ring-2.6-1T
  7. 94.8 Claude Sonnet 4.6
  8. 93.6 Claude Opus 4.5
  9. 93.4 GPT-5.4 Mini
  10. 92.7 Nova 2.0 Pro Preview
  11. 92.5 GPT-5.4 Nano
  12. 92.3 Seed2.0 Pro
  13. 91.5 Muse Spark
  14. 91.3 Trinity Large Thinking
  15. 90.3 GPT-5.2
  16. 90.2 Gemini 3 Flash
  17. 89.7 GLM-5
  18. 88.2 LongCat-Flash-Thinking-2601
  19. 88.2 Step 3.5 Flash
  20. 87.4 GLM-4.7
  21. 87.0 MiniMax-M2
  22. 87.0 MiniMax-M2.1
  23. 86.9 Spark-X2.6-4B
  24. 86.7 Qwen3.5 397B-A17B
  25. 85.4 Gemini 3 Pro
  26. 85.0 Command A+
  27. 84.7 Claude Sonnet 4.5
  28. 82.6 Keye-VL-2.0-30B-A3B
  29. 82.1 Qwen3-Max
  30. 81.2 Qwen3.5-35B-A3B
  31. 81.0 Qwen3.5-Omni-Plus
  32. 80.9 Intern-S1-Pro
  33. 80.3 DeepSeek-V3.2
  34. 80.3 MiMo-V2-Flash
  35. 80.2 GPT-5.1
  36. 80.1 GPT-5
  37. 80.0 Nova 2 Omni Preview
  38. 79.9 Qwen3.5-4B
  39. 79.5 glm-4.7-flash
  40. 79.5 Qwen3.5-122B-A10B
  41. 79.1 Qwen3.5-9B
  42. 79.0 Qwen3.5-27B
  43. 78.8 ERNIE 5.0
  44. 78.4 Ring-2.5-1T
  45. 78.4 Ling-2.6-1T
  46. 76.9 Gemma 4 31B IT
  47. 76.6 Claude Haiku 4.5
  48. 76.0 Nova 2 Lite
  49. 75.9 GLM-4.6
  50. 74.0 LongCat-Flash-Thinking
  51. 73.2 K-EXAONE
  52. 73.0 Nemotron 3.5 Super VL
  53. 72.4 Tencent HY 2.0 Think
  54. 72.3 Solar Pro 3 (March 2026 update)
  55. 72.0 Seed1.8
  56. 72.0 EXAONE 4.5 33B
  57. 71.7 MAI-Code-1-Flash
  58. 70.8 GPT-5 mini
  59. 69.0 Gemma 4 12B
  60. 68.3 Sarvam 105B
  61. 68.2 Gemma 4 26B A4B
  62. 67.7 LongCat-Flash-Chat
  63. 66.1 Kimi K2
  64. 64.0 LFM2.5-8B-A1B
  65. 61.1 Nemotron 3 Super
  66. 58.9 Nemotron-Cascade 2 30B-A3B
  67. 58.5 Qwen3-235B-A22B-Thinking
  68. 57.4 Qwen3-Next-80B-A3B
  69. 56.2 DiffusionGemma 26B A4B
  70. 55.8 Solar Open 100B
  71. 52.3 Nemotron 3.5 Bolt E4B
  72. 49.0 Nemotron 3 Nano 30B-A3B
  73. 48.8 Qwen3.5-2B
  74. 46.3 GPT-5 nano
  75. 45.7 Sarvam 30B
  76. 42.2 Gemma 4 E4B
  77. 35.4 Nemotron 3 Nano 4B
  78. 24.5 Gemma 4 E2B
  79. 11.6 Qwen3.5-0.8B
  1. 1 GPT-5.4 98.9%
  2. 2 GPT-5.5 98%
  3. 3 MiniMax-M2.5 97.8%
  4. 4 MiMo-V2-Pro 96.8%
  5. 5 Claude Opus 4.6 95.6%
  6. 6 Ring-2.6-1T 95.3%
  7. 7 Claude Sonnet 4.6 94.8%
  8. 8 Claude Opus 4.5 93.6%
  9. 9 GPT-5.4 Mini 93.4%
  10. 10 Nova 2.0 Pro Preview 92.7%
  11. 11 GPT-5.4 Nano 92.5%
  12. 12 Seed2.0 Pro 92.3%
  13. 13 Muse Spark 91.5%
  14. 14 Trinity Large Thinking 91.3%
  15. 15 GPT-5.2 90.3%
  16. 16 Gemini 3 Flash 90.2%
  17. 17 GLM-5 89.7%
  18. 18 LongCat-Flash-Thinking-2601 88.2%
  19. 19 Step 3.5 Flash 88.2%
  20. 20 GLM-4.7 87.4%
  21. 21 MiniMax-M2 87%
  22. 22 MiniMax-M2.1 87%
  23. 23 Spark-X2.6-4B 86.9%
  24. 24 Qwen3.5 397B-A17B 86.7%
  25. 25 Gemini 3 Pro 85.4%
  26. 26 Command A+ 85%
  27. 27 Claude Sonnet 4.5 84.7%
  28. 28 Keye-VL-2.0-30B-A3B 82.6%
  29. 29 Qwen3-Max 82.1%
  30. 30 Qwen3.5-35B-A3B 81.2%
  31. 31 Qwen3.5-Omni-Plus 81%
  32. 32 Intern-S1-Pro 80.9%
  33. 33 DeepSeek-V3.2 80.3%
  34. 34 MiMo-V2-Flash 80.3%
  35. 35 GPT-5.1 80.2%
  36. 36 GPT-5 80.1%
  37. 37 Nova 2 Omni Preview 80%
  38. 38 Qwen3.5-4B 79.9%
  39. 39 glm-4.7-flash 79.5%
  40. 40 Qwen3.5-122B-A10B 79.5%
  41. 41 Qwen3.5-9B 79.1%
  42. 42 Qwen3.5-27B 79%
  43. 43 ERNIE 5.0 78.8%
  44. 44 Ring-2.5-1T 78.4%
  45. 45 Ling-2.6-1T 78.4%
  46. 46 Gemma 4 31B IT 76.9%
  47. 47 Claude Haiku 4.5 76.6%
  48. 48 Nova 2 Lite 76%
  49. 49 GLM-4.6 75.9%
  50. 50 LongCat-Flash-Thinking 74%
  51. 51 K-EXAONE 73.2%
  52. 52 Nemotron 3.5 Super VL 73%
  53. 53 Tencent HY 2.0 Think 72.4%
  54. 54 Solar Pro 3 (March 2026 update) 72.3%
  55. 55 Seed1.8 72%
  56. 56 EXAONE 4.5 33B 72%
  57. 57 MAI-Code-1-Flash 71.7%
  58. 58 GPT-5 mini 70.8%
  59. 59 Gemma 4 12B 69%
  60. 60 Sarvam 105B 68.3%
  61. 61 Gemma 4 26B A4B 68.2%
  62. 62 LongCat-Flash-Chat 67.7%
  63. 63 Kimi K2 Jul 2025 66.1%
  64. 64 LFM2.5-8B-A1B 64%
  65. 65 Nemotron 3 Super 61.1%
  66. 66 Nemotron-Cascade 2 30B-A3B 58.9%
  67. 67 Qwen3-235B-A22B-Thinking Jul 2025 58.5%
  68. 68 Qwen3-Next-80B-A3B 57.4%
  69. 69 DiffusionGemma 26B A4B 56.2%
  70. 70 Solar Open 100B 55.8%
  71. 71 Nemotron 3.5 Bolt E4B 52.3%
  72. 72 Nemotron 3 Nano 30B-A3B 49%
  73. 73 Qwen3.5-2B 48.8%
  74. 74 GPT-5 nano 46.3%
  75. 75 Sarvam 30B 45.7%
  76. 76 Gemma 4 E4B 42.2%
  77. 77 Nemotron 3 Nano 4B 35.4%
  78. 78 Gemma 4 E2B 24.5%
  79. 79 Qwen3.5-0.8B 11.6%

Source: Labs’ model cards

Source: Official tau2-bench leaderboard

83 models. Scroll sideways for the rest.

  1. 98.9 GPT-5.4
  2. 98.0 GPT-5.5
  3. 97.8 MiniMax-M2.5
  4. 96.8 MiMo-V2-Pro
  5. 95.6 Claude Opus 4.6
  6. 95.3 Ring-2.6-1T
  7. 94.8 Claude Sonnet 4.6
  8. 93.6 Claude Opus 4.5
  9. 93.4 GPT-5.4 Mini
  10. 92.7 Nova 2.0 Pro Preview
  11. 92.5 GPT-5.4 Nano
  12. 92.3 Seed2.0 Pro
  13. 91.5 Muse Spark
  14. 91.3 Trinity Large Thinking
  15. 90.3 GPT-5.2
  16. 90.2 Gemini 3 Flash
  17. 89.7 GLM-5
  18. 88.2 LongCat-Flash-Thinking-2601
  19. 88.2 Step 3.5 Flash
  20. 87.4 GLM-4.7
  21. 87.0 MiniMax-M2
  22. 87.0 MiniMax-M2.1
  23. 86.9 Spark-X2.6-4B
  24. 86.7 Qwen3.5 397B-A17B
  25. 85.4 Gemini 3 Pro
  26. 85.0 Command A+
  27. 84.7 Claude Sonnet 4.5
  28. 82.6 Keye-VL-2.0-30B-A3B
  29. 82.2 Qwen3-Max
  30. 81.2 Qwen3.5-35B-A3B
  31. 81.0 Qwen3.5-Omni-Plus
  32. 80.9 Intern-S1-Pro
  33. 80.4 DeepSeek-V3.2
  34. 80.3 MiMo-V2-Flash
  35. 80.2 GPT-5.1
  36. 80.1 GPT-5
  37. 80.0 Nova 2 Omni Preview
  38. 79.9 Qwen3.5-4B
  39. 79.5 glm-4.7-flash
  40. 79.5 Qwen3.5-122B-A10B
  41. 79.1 Qwen3.5-9B
  42. 79.0 Qwen3.5-27B
  43. 78.8 ERNIE 5.0
  44. 78.4 Ring-2.5-1T
  45. 78.4 Ling-2.6-1T
  46. 76.9 Gemma 4 31B IT
  47. 76.6 Claude Haiku 4.5
  48. 76.0 Nova 2 Lite
  49. 75.9 GLM-4.6
  50. 74.0 LongCat-Flash-Thinking
  51. 73.2 K-EXAONE
  52. 73.0 Nemotron 3.5 Super VL
  53. 72.4 Tencent HY 2.0 Think
  54. 72.3 Solar Pro 3 (March 2026 update)
  55. 72.0 Seed1.8
  56. 72.0 EXAONE 4.5 33B
  57. 71.7 MAI-Code-1-Flash
  58. 70.8 GPT-5 mini
  59. 69.0 Gemma 4 12B
  60. 68.3 Sarvam 105B
  61. 68.2 Gemma 4 26B A4B
  62. 67.7 LongCat-Flash-Chat
  63. 66.1 Kimi K2
  64. 64.0 LFM2.5-8B-A1B
  65. 61.8 Claude 3.7 Sonnet
  66. 61.1 Nemotron 3 Super
  67. 58.9 Nemotron-Cascade 2 30B-A3B
  68. 58.5 Qwen3-235B-A22B-Thinking
  69. 57.4 Qwen3-Next-80B-A3B
  70. 56.9 o4-mini
  71. 56.2 DiffusionGemma 26B A4B
  72. 55.8 Solar Open 100B
  73. 54.7 GPT-4.1
  74. 53.0 GPT-4.1 mini
  75. 52.3 Nemotron 3.5 Bolt E4B
  76. 49.0 Nemotron 3 Nano 30B-A3B
  77. 48.8 Qwen3.5-2B
  78. 46.3 GPT-5 nano
  79. 45.7 Sarvam 30B
  80. 42.2 Gemma 4 E4B
  81. 35.4 Nemotron 3 Nano 4B
  82. 24.5 Gemma 4 E2B
  83. 11.6 Qwen3.5-0.8B
  1. 1 GPT-5.4 98.9%
  2. 2 GPT-5.5 98%
  3. 3 MiniMax-M2.5 97.8%
  4. 4 MiMo-V2-Pro 96.8%
  5. 5 Claude Opus 4.6 95.6%
  6. 6 Ring-2.6-1T 95.3%
  7. 7 Claude Sonnet 4.6 94.8%
  8. 8 Claude Opus 4.5 93.6%
  9. 9 GPT-5.4 Mini 93.4%
  10. 10 Nova 2.0 Pro Preview 92.7%
  11. 11 GPT-5.4 Nano 92.5%
  12. 12 Seed2.0 Pro 92.3%
  13. 13 Muse Spark 91.5%
  14. 14 Trinity Large Thinking 91.3%
  15. 15 GPT-5.2 90.3%
  16. 16 Gemini 3 Flash 90.2%
  17. 17 GLM-5 89.7%
  18. 18 LongCat-Flash-Thinking-2601 88.2%
  19. 19 Step 3.5 Flash 88.2%
  20. 20 GLM-4.7 87.4%
  21. 21 MiniMax-M2 87%
  22. 22 MiniMax-M2.1 87%
  23. 23 Spark-X2.6-4B 86.9%
  24. 24 Qwen3.5 397B-A17B 86.7%
  25. 25 Gemini 3 Pro 85.4%
  26. 26 Command A+ 85%
  27. 27 Claude Sonnet 4.5 84.7%
  28. 28 Keye-VL-2.0-30B-A3B 82.6%
  29. 29 Qwen3-Max 82.2%
  30. 30 Qwen3.5-35B-A3B 81.2%
  31. 31 Qwen3.5-Omni-Plus 81%
  32. 32 Intern-S1-Pro 80.9%
  33. 33 DeepSeek-V3.2 80.4%
  34. 34 MiMo-V2-Flash 80.3%
  35. 35 GPT-5.1 80.2%
  36. 36 GPT-5 80.1%
  37. 37 Nova 2 Omni Preview 80%
  38. 38 Qwen3.5-4B 79.9%
  39. 39 glm-4.7-flash 79.5%
  40. 40 Qwen3.5-122B-A10B 79.5%
  41. 41 Qwen3.5-9B 79.1%
  42. 42 Qwen3.5-27B 79%
  43. 43 ERNIE 5.0 78.8%
  44. 44 Ring-2.5-1T 78.4%
  45. 45 Ling-2.6-1T 78.4%
  46. 46 Gemma 4 31B IT 76.9%
  47. 47 Claude Haiku 4.5 76.6%
  48. 48 Nova 2 Lite 76%
  49. 49 GLM-4.6 75.9%
  50. 50 LongCat-Flash-Thinking 74%
  51. 51 K-EXAONE 73.2%
  52. 52 Nemotron 3.5 Super VL 73%
  53. 53 Tencent HY 2.0 Think 72.4%
  54. 54 Solar Pro 3 (March 2026 update) 72.3%
  55. 55 Seed1.8 72%
  56. 56 EXAONE 4.5 33B 72%
  57. 57 MAI-Code-1-Flash 71.7%
  58. 58 GPT-5 mini 70.8%
  59. 59 Gemma 4 12B 69%
  60. 60 Sarvam 105B 68.3%
  61. 61 Gemma 4 26B A4B 68.2%
  62. 62 LongCat-Flash-Chat 67.7%
  63. 63 Kimi K2 Jul 2025 66.1%
  64. 64 LFM2.5-8B-A1B 64%
  65. 65 Claude 3.7 Sonnet 61.8%
  66. 66 Nemotron 3 Super 61.1%
  67. 67 Nemotron-Cascade 2 30B-A3B 58.9%
  68. 68 Qwen3-235B-A22B-Thinking Jul 2025 58.5%
  69. 69 Qwen3-Next-80B-A3B 57.4%
  70. 70 o4-mini 56.9%
  71. 71 DiffusionGemma 26B A4B 56.2%
  72. 72 Solar Open 100B 55.8%
  73. 73 GPT-4.1 54.7%
  74. 74 GPT-4.1 mini 53%
  75. 75 Nemotron 3.5 Bolt E4B 52.3%
  76. 76 Nemotron 3 Nano 30B-A3B 49%
  77. 77 Qwen3.5-2B 48.8%
  78. 78 GPT-5 nano 46.3%
  79. 79 Sarvam 30B 45.7%
  80. 80 Gemma 4 E4B 42.2%
  81. 81 Nemotron 3 Nano 4B 35.4%
  82. 82 Gemma 4 E2B 24.5%
  83. 83 Qwen3.5-0.8B 11.6%

Sources: Labs’ model cards and official tau2-bench leaderboard

#ModelScoreSettingDateSource
1 GPT-5.4 98.9% Mar 2026 Lab model card
2 GPT-5.5 98% Apr 2026 Lab model card
3 MiniMax-M2.5 97.8% Feb 2026 Lab model card
4 MiMo-V2-Pro 96.8% Mar 2026 Lab model card
5 Claude Opus 4.6 95.6% Feb 2026 Lab model card
6 Ring-2.6-1T 95.3% May 2026 Lab model card
7 Claude Sonnet 4.6 94.8% Feb 2026 Lab model card
8 Claude Opus 4.5 93.6% Nov 2025 Lab model card
9 GPT-5.4 Mini 93.4% Mar 2026 Lab model card
10 Nova 2.0 Pro Preview 92.7% Dec 2025 Lab model card
11 GPT-5.4 Nano 92.5% Mar 2026 Lab model card
12 Seed2.0 Pro 92.3% Feb 2026 Lab model card
13 Muse Spark 91.5% Apr 2026 Lab model card
14 Trinity Large Thinking 91.3% Apr 2026 Lab model card
15 GPT-5.2 90.3% Dec 2025 Lab model card
16 Gemini 3 Flash 90.2% Dec 2025 Lab model card
17 GLM-5 89.7% Feb 2026 Lab model card
18 LongCat-Flash-Thinking-2601 88.2% Jan 2026 Lab model card
19 Step 3.5 Flash 88.2% Feb 2026 Lab model card
20 GLM-4.7 87.4% Dec 2025 Lab model card
21 MiniMax-M2 87% Oct 2025 Lab model card
22 MiniMax-M2.1 87% Dec 2025 Lab model card
23 Spark-X2.6-4B 86.9% Oct 2026 Lab model card
24 Qwen3.5 397B-A17B 86.7% Feb 2026 Lab model card
25 Gemini 3 Pro 85.4% Nov 2025 Lab model card
26 Command A+ 85% May 2026 Lab model card
27 Claude Sonnet 4.5 84.7% Sep 2025 Lab model card
28 Keye-VL-2.0-30B-A3B 82.6% May 2026 Lab model card
29 Qwen3-Max 82.1% Jan 2026 Lab model card
30 Qwen3.5-35B-A3B 81.2% Feb 2026 Lab model card
31 Qwen3.5-Omni-Plus 81% Mar 2026 Lab model card
32 Intern-S1-Pro 80.9% Feb 2026 Lab model card
33 DeepSeek-V3.2 80.3% Dec 2025 Lab model card
34 MiMo-V2-Flash 80.3% Dec 2025 Lab model card
35 GPT-5.1 80.2% Nov 2025 Lab model card
36 GPT-5 80.1% Aug 2025 Lab model card
37 Nova 2 Omni Preview 80% Dec 2025 Lab model card
38 Qwen3.5-4B 79.9% Mar 2026 Lab model card
39 glm-4.7-flash 79.5% Jan 2026 Lab model card
40 Qwen3.5-122B-A10B 79.5% Feb 2026 Lab model card
41 Qwen3.5-9B 79.1% Mar 2026 Lab model card
42 Qwen3.5-27B 79% Feb 2026 Lab model card
43 ERNIE 5.0 78.8% Feb 2026 Lab model card
44 Ring-2.5-1T 78.4% Feb 2026 Lab model card
45 Ling-2.6-1T 78.4% Apr 2026 Lab model card
46 Gemma 4 31B IT 76.9% Apr 2026 Lab model card
47 Claude Haiku 4.5 76.6% Oct 2025 Lab model card
48 Nova 2 Lite 76% Dec 2025 Lab model card
49 GLM-4.6 75.9% Sep 2025 Lab model card
50 LongCat-Flash-Thinking 74% Sep 2025 Lab model card
51 K-EXAONE 73.2% Jan 2026 Lab model card
52 Nemotron 3.5 Super VL 73% Oct 2026 Lab model card
53 Tencent HY 2.0 Think 72.4% Dec 2025 Lab model card
54 Solar Pro 3 (March 2026 update) 72.3% Mar 2026 Lab model card
55 Seed1.8 72% Dec 2025 Lab model card
56 EXAONE 4.5 33B 72% Apr 2026 Lab model card
57 MAI-Code-1-Flash 71.7% Jun 2026 Lab model card
58 GPT-5 mini 70.8% Aug 2025 Lab model card
59 Gemma 4 12B 69% Jun 2026 Lab model card
60 Sarvam 105B 68.3% Mar 2026 Lab model card
61 Gemma 4 26B A4B 68.2% Apr 2026 Lab model card
62 LongCat-Flash-Chat 67.7% Aug 2025 Lab model card
63 Kimi K2 Jul 2025 66.1% Jul 2025 Lab model card
64 LFM2.5-8B-A1B 64% May 2026 Lab model card
65 Nemotron 3 Super 61.1% Mar 2026 Lab model card
66 Nemotron-Cascade 2 30B-A3B 58.9% Mar 2026 Lab model card
67 Qwen3-235B-A22B-Thinking Jul 2025 58.5% Jul 2025 Lab model card
68 Qwen3-Next-80B-A3B 57.4% Sep 2025 Lab model card
69 DiffusionGemma 26B A4B 56.2% Jun 2026 Lab model card
70 Solar Open 100B 55.8% Dec 2025 Lab model card
71 Nemotron 3.5 Bolt E4B 52.3% Oct 2026 Lab model card
72 Nemotron 3 Nano 30B-A3B 49% Dec 2025 Lab model card
73 Qwen3.5-2B 48.8% Mar 2026 Lab model card
74 GPT-5 nano 46.3% Aug 2025 Lab model card
75 Sarvam 30B 45.7% Mar 2026 Lab model card
76 Gemma 4 E4B 42.2% Apr 2026 Lab model card
77 Nemotron 3 Nano 4B 35.4% Mar 2026 Lab model card
78 Gemma 4 E2B 24.5% Apr 2026 Lab model card
79 Qwen3.5-0.8B 11.6% Mar 2026 Lab model card
#ModelScoreSettingDateSource
1 Gemini 3 Pro 85.4% Nov 2025 Official leaderboard
2 Claude Sonnet 4.5 84.7% Sep 2025 Official leaderboard
3 Qwen3-Max 82.2% Sep 2025 Official leaderboard
4 DeepSeek-V3.2 80.4% Dec 2025 Official leaderboard
5 GPT-5 80% Aug 2025 Official leaderboard
6 Kimi K2 Jul 2025 64.3% Jul 2025 Official leaderboard
7 Claude 3.7 Sonnet 61.8% Feb 2025 Official leaderboard
8 o4-mini 56.9% Apr 2025 Official leaderboard
9 GPT-4.1 54.7% Apr 2025 Official leaderboard
10 GPT-4.1 mini 53% Apr 2025 Official leaderboard
#ModelScoreSettingDateSource
1 GPT-5.4 98.9% Mar 2026 Lab model card
2 GPT-5.5 98% Apr 2026 Lab model card
3 MiniMax-M2.5 97.8% Feb 2026 Lab model card
4 MiMo-V2-Pro 96.8% Mar 2026 Lab model card
5 Claude Opus 4.6 95.6% Feb 2026 Lab model card
6 Ring-2.6-1T 95.3% May 2026 Lab model card
7 Claude Sonnet 4.6 94.8% Feb 2026 Lab model card
8 Claude Opus 4.5 93.6% Nov 2025 Lab model card
9 GPT-5.4 Mini 93.4% Mar 2026 Lab model card
10 Nova 2.0 Pro Preview 92.7% Dec 2025 Lab model card
11 GPT-5.4 Nano 92.5% Mar 2026 Lab model card
12 Seed2.0 Pro 92.3% Feb 2026 Lab model card
13 Muse Spark 91.5% Apr 2026 Lab model card
14 Trinity Large Thinking 91.3% Apr 2026 Lab model card
15 GPT-5.2 90.3% Dec 2025 Lab model card
16 Gemini 3 Flash 90.2% Dec 2025 Lab model card
17 GLM-5 89.7% Feb 2026 Lab model card
18 LongCat-Flash-Thinking-2601 88.2% Jan 2026 Lab model card
19 Step 3.5 Flash 88.2% Feb 2026 Lab model card
20 GLM-4.7 87.4% Dec 2025 Lab model card
21 MiniMax-M2 87% Oct 2025 Lab model card
22 MiniMax-M2.1 87% Dec 2025 Lab model card
23 Spark-X2.6-4B 86.9% Oct 2026 Lab model card
24 Qwen3.5 397B-A17B 86.7% Feb 2026 Lab model card
25 Gemini 3 Pro 85.4% Nov 2025 Official leaderboard
26 Command A+ 85% May 2026 Lab model card
27 Claude Sonnet 4.5 84.7% Sep 2025 Official leaderboard
28 Keye-VL-2.0-30B-A3B 82.6% May 2026 Lab model card
29 Qwen3-Max 82.2% Sep 2025 Official leaderboard
30 Qwen3.5-35B-A3B 81.2% Feb 2026 Lab model card
31 Qwen3.5-Omni-Plus 81% Mar 2026 Lab model card
32 Intern-S1-Pro 80.9% Feb 2026 Lab model card
33 DeepSeek-V3.2 80.4% Dec 2025 Official leaderboard
34 MiMo-V2-Flash 80.3% Dec 2025 Lab model card
35 GPT-5.1 80.2% Nov 2025 Lab model card
36 GPT-5 80.1% Aug 2025 Lab model card
37 Nova 2 Omni Preview 80% Dec 2025 Lab model card
38 Qwen3.5-4B 79.9% Mar 2026 Lab model card
39 glm-4.7-flash 79.5% Jan 2026 Lab model card
40 Qwen3.5-122B-A10B 79.5% Feb 2026 Lab model card
41 Qwen3.5-9B 79.1% Mar 2026 Lab model card
42 Qwen3.5-27B 79% Feb 2026 Lab model card
43 ERNIE 5.0 78.8% Feb 2026 Lab model card
44 Ring-2.5-1T 78.4% Feb 2026 Lab model card
45 Ling-2.6-1T 78.4% Apr 2026 Lab model card
46 Gemma 4 31B IT 76.9% Apr 2026 Lab model card
47 Claude Haiku 4.5 76.6% Oct 2025 Lab model card
48 Nova 2 Lite 76% Dec 2025 Lab model card
49 GLM-4.6 75.9% Sep 2025 Lab model card
50 LongCat-Flash-Thinking 74% Sep 2025 Lab model card
51 K-EXAONE 73.2% Jan 2026 Lab model card
52 Nemotron 3.5 Super VL 73% Oct 2026 Lab model card
53 Tencent HY 2.0 Think 72.4% Dec 2025 Lab model card
54 Solar Pro 3 (March 2026 update) 72.3% Mar 2026 Lab model card
55 Seed1.8 72% Dec 2025 Lab model card
56 EXAONE 4.5 33B 72% Apr 2026 Lab model card
57 MAI-Code-1-Flash 71.7% Jun 2026 Lab model card
58 GPT-5 mini 70.8% Aug 2025 Lab model card
59 Gemma 4 12B 69% Jun 2026 Lab model card
60 Sarvam 105B 68.3% Mar 2026 Lab model card
61 Gemma 4 26B A4B 68.2% Apr 2026 Lab model card
62 LongCat-Flash-Chat 67.7% Aug 2025 Lab model card
63 Kimi K2 Jul 2025 66.1% Jul 2025 Lab model card
64 LFM2.5-8B-A1B 64% May 2026 Lab model card
65 Claude 3.7 Sonnet 61.8% Feb 2025 Official leaderboard
66 Nemotron 3 Super 61.1% Mar 2026 Lab model card
67 Nemotron-Cascade 2 30B-A3B 58.9% Mar 2026 Lab model card
68 Qwen3-235B-A22B-Thinking Jul 2025 58.5% Jul 2025 Lab model card
69 Qwen3-Next-80B-A3B 57.4% Sep 2025 Lab model card
70 o4-mini 56.9% Apr 2025 Official leaderboard
71 DiffusionGemma 26B A4B 56.2% Jun 2026 Lab model card
72 Solar Open 100B 55.8% Dec 2025 Lab model card
73 GPT-4.1 54.7% Apr 2025 Official leaderboard
74 GPT-4.1 mini 53% Apr 2025 Official leaderboard
75 Nemotron 3.5 Bolt E4B 52.3% Oct 2026 Lab model card
76 Nemotron 3 Nano 30B-A3B 49% Dec 2025 Lab model card
77 Qwen3.5-2B 48.8% Mar 2026 Lab model card
78 GPT-5 nano 46.3% Aug 2025 Lab model card
79 Sarvam 30B 45.7% Mar 2026 Lab model card
80 Gemma 4 E4B 42.2% Apr 2026 Lab model card
81 Nemotron 3 Nano 4B 35.4% Mar 2026 Lab model card
82 Gemma 4 E2B 24.5% Apr 2026 Lab model card
83 Qwen3.5-0.8B 11.6% Mar 2026 Lab model card

The benchmark's own leaderboard has 10 models. Switch the scores above to rank them together.

8 models have held number one in labs’ own numbers since its release in June 2025 6 models have held number one across all independent runs since its release in June 2025 10 models have held number one with claims included since its release in June 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic Moonshot AI OpenAI Qwen Google DeepMind MiniMax
Own leaderboard Passed 85% in Nov 2025, by Gemini 3 Pro
Gemini 3 Pro 85.4%
Lab claims Passed 85% in Oct 2025, by MiniMax-M2
GPT-5.4 98.9%

Sources: Official tau2-bench leaderboard and labs’ model cards

MiniMax’s own number for MiniMax-M2 passed 85% in October 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) GPT-5.4 Claude Opus 4.5 MiniMax-M2 Qwen3-Max Kimi K2 Claude 3.7 Sonnet 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) GPT-5.4 MiniMax-M2

Sources: Official tau2-bench leaderboard and labs’ model cards

4 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Moonshot AI
Qwen
OpenAI
Meituan
Anthropic
Z.ai
MiniMax
Google DeepMind
xAI
DeepSeek
Amazon
Tencent
NVIDIA
Xiaomi Corp
ByteDance Seed
LG AI Research
Upstage
StepFun
Shanghai AI Lab
Baidu
Ant Group
Sarvam AI
Arcee AI
Meta
Cohere
Kuaishou
Liquid AI
Microsoft
iFlytek
DateLaunch postScoreAs the post writes it
Oct 23, 2026 Nemotron 3.5 Bolt E4B 52.3 Tau2-Bench
Oct 22, 2026 Spark-X2.6-4B 86.9 τ²-bench
Oct 15, 2026 Nemotron 3.5 Super VL 73 Mean of tau2-bench Retail 69.24 / Airline 76.50 / Telecom 73.19
Jun 10, 2026 DiffusionGemma 26B A4B 56.2 Tau2 (average over 3)
Jun 3, 2026 Gemma 4 12B 69 Tau2 (average over 3)
Jun 2, 2026 MAI-Code-1-Flash 71.7 𝜏²-Bench telecom
May 28, 2026 LFM2.5-8B-A1B 64 Mean of Tau² Telecom 88.07 / Tau² Retail 39.82
May 25, 2026 Keye-VL-2.0-30B-A3B 82.6 TAU2-Bench
May 20, 2026 Command A+ 85 τ²-Bench Telecom only (chart label)
May 14, 2026 Ring-2.6-1T 95.3 Tau2-Bench Telecom scenario only (high)
Apr 29, 2026 Ling-2.6-1T 78.4 TAU2-Bench
Apr 23, 2026 GPT-5.5 98 Tau2-bench Telecom (original prompts)
Apr 9, 2026 EXAONE 4.5 33B 69.1 Mean of τ2-Bench Retail 77.9 / Airline 56.5 / Telecom 73.0
Apr 8, 2026 Muse Spark 91.5 τ²-Bench Telecom (Agentic Tool Use (Artificial Analysis))
Apr 2, 2026 Gemma 4 E4B 42.2 Tau2 (average over 3)
Apr 2, 2026 Gemma 4 E2B 24.5 Tau2 (average over 3)
Apr 2, 2026 Gemma 4 26B A4B 68.2 Tau2 (average over 3)
Apr 2, 2026 Gemma 4 31B IT 76.9 Tau2 (average over 3)
Apr 1, 2026 Trinity Large Thinking 91.3 Mean of Tau2-Airline 88.0 / Tau2-Telecom 94.7
Mar 30, 2026 Qwen3.5-Omni-Plus 81 TAU2Bench
Mar 24, 2026 Solar Pro 3 (March 2026 update) 72.3 Tau2-all
Mar 19, 2026 Nemotron-Cascade 2 30B-A3B 58.9 τ²-Bench
Mar 18, 2026 MiMo-V2-Pro 96.8 τ2-bench (Telecom)
Mar 17, 2026 GPT-5.4 Nano 92.5 τ2-bench (telecom)
Mar 17, 2026 GPT-5.4 Mini 93.4 τ²-bench telecom
Mar 16, 2026 Nemotron 3 Nano 4B 35.4 Mean of Tau2 Airline 33.3 / Retail 39.8 / Telecom 33 (reasoning on)
Mar 11, 2026 Nemotron 3 Super 61.1 TauBench V2 Average of Airline 56.25 / Retail 62.83 / Telecom 64.36
Mar 6, 2026 Sarvam 30B 45.7 Tau2 (avg.)
Mar 6, 2026 Sarvam 105B 68.3 Tau2 (avg.)
Mar 5, 2026 GPT-5.4 98.9 Tau2-bench Telecom (xhigh; body chart shows 64.3% without reasoning)
Mar 2, 2026 Qwen3.5-0.8B 11.6 TAU2-Bench
Mar 2, 2026 Qwen3.5-2B 48.8 TAU2-Bench
Mar 2, 2026 Qwen3.5-4B 79.9 TAU2-Bench
Mar 2, 2026 Qwen3.5-9B 79.1 TAU2-Bench
Feb 24, 2026 Qwen3.5-35B-A3B 81.2 TAU2-Bench
Feb 24, 2026 Qwen3.5-27B 79 TAU2-Bench
Feb 24, 2026 Qwen3.5-122B-A10B 79.5 TAU2-Bench (airline fixes from Claude Opus 4.5 system card)
Feb 19, 2026 Gemini 3.1 Pro Mean of τ2-bench, Retail / τ2-bench, Telecom
Feb 17, 2026 Claude Sonnet 4.6 94.8 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Telecom
Feb 16, 2026 Qwen3.5 397B-A17B 86.7 TAU2-Bench
Feb 14, 2026 Seed2.0 Pro 92.3 Mean of τ2-Bench (retail) 90.4 / τ2-Bench (telecom) 94.2
Feb 12, 2026 MiniMax-M2.5 97.8 τ² Telecom
Feb 11, 2026 GLM-5 89.7 τ²-Bench
Feb 10, 2026 Ring-2.5-1T 78.4 Tau2-Bench (Mean@3)
Feb 6, 2026 ERNIE 5.0 78.8 TAU2-Bench
Feb 5, 2026 Claude Opus 4.6 95.6 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Telecom
Feb 2, 2026 Intern-S1-Pro 80.9 Tau2-Bench
Feb 2, 2026 Step 3.5 Flash 88.2 τ²-Bench
Jan 23, 2026 Qwen3-Max 82.1 Tau² Bench
Jan 19, 2026 glm-4.7-flash 79.5 τ²-Bench
Jan 14, 2026 LongCat-Flash-Thinking-2601 88.2 τ²-Avg (Avg@4): Retail 88.6 / Airline 76.5 / Telecom 99.3
Dec 31, 2025 Solar Open 100B 55.8 Mean of Tau² Airline 52.4 / Telecom 55.6 / Retail 59.3
Dec 31, 2025 K-EXAONE 70.8 Mean of τ2-Bench Retail 78.6 / Airline 60.4 / Telecom 73.5
Dec 23, 2025 MiniMax-M2.1 87 𝜏²-Bench Telecom
Dec 22, 2025 GLM-4.7 87.4 τ²-Bench
Dec 18, 2025 Seed1.8 72 τ2-Bench
Dec 17, 2025 Gemini 3 Flash 90.2 τ2-bench
Dec 16, 2025 MiMo-V2-Flash 80.3 τ²-Bench
Dec 15, 2025 Nemotron 3 Nano 30B-A3B 49 TauBench V2 (Average) of Airline 48.0 / Retail 56.9 / Telecom 42.2
Dec 11, 2025 GPT-5.2 90.3 Mean of Tau2-bench Telecom Tool use in customer support / Tau2-bench Retail Tool use in customer support
Dec 5, 2025 Tencent HY 2.0 Think 72.4 Tau2-Bench
Dec 2, 2025 Nova 2 Omni Preview 80 τ2-bench Telecom only (as benchmarked by Artificial Analysis)
Dec 2, 2025 Nova 2 Lite 76 τ2-bench Telecom only (as benchmarked by Artificial Analysis)
Dec 2, 2025 Nova 2.0 Pro Preview 92.7 τ2-bench Telecom only (as benchmarked by Artificial Analysis)
Dec 1, 2025 DeepSeek-V3.2 80.3 τ2-Bench (Pass@1)
Nov 24, 2025 Claude Opus 4.5 93.6 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Telecom
Nov 19, 2025 Grok 4.1 Fast τ²-bench Telecom (chart)
Nov 18, 2025 Gemini 3 Pro 85.4 τ2-bench
Nov 13, 2025 GPT-5.1 80.2 Mean of Tau2-bench Airline / Tau2-bench Telecom* / Tau2-bench Retail
Oct 27, 2025 MiniMax-M2 87 Best of τ²-Bench / τ²-Bench-Telecom (AA)
Oct 15, 2025 Claude Haiku 4.5 76.6 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Airline / Agentic tool use τ2-bench Telecom
Sep 30, 2025 GLM-4.6 75.9 τ²-Bench (Weighted)
Sep 29, 2025 Claude Sonnet 4.5 84.7 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Airline / Agentic tool use τ2-bench Telecom
Sep 24, 2025 Qwen3-Max 74.8 Tau2-Bench (Weighted)
Sep 21, 2025 LongCat-Flash-Thinking 74 Mean of τ²-Bench-Retail 71.5 / τ²-Bench-Airline 67.5 / τ²-Bench-Telecom 83.1 (Mean@4)
Sep 11, 2025 Qwen3-Next-80B-A3B 57.4 Mean of TAU2-Retail 67.8 / TAU2-Airline 60.5 / TAU2-Telecom 43.9
Aug 29, 2025 LongCat-Flash-Chat 67.7 Mean of τ²-Bench (telecom) 73.68 / (airline) 58.00 / (retail) 71.27 (avg@4)
Aug 7, 2025 GPT-5 nano 46.3 Mean of Tau2-bench airline 41.0 / retail 62.3 / telecom 35.5
Aug 7, 2025 GPT-5 mini 70.8 Mean of Tau2-bench airline 60.0 / retail 78.3 / telecom 74.1
Aug 7, 2025 GPT-5 80.1 Mean of Tau2-bench Function calling (airline) / Tau2-bench Function calling (retail) / Tau2-bench Function calling (telecom)
Jul 25, 2025 Qwen3-235B-A22B-Thinking 58.5 Mean of TAU2-Retail / TAU2-Airline / TAU2-Telecom
Jul 11, 2025 Kimi K2 64.3 Mean of Tau2 retail, Avg@4 / Tau2 airline, Avg@4 / Tau2 telecom, Avg@4

40 system cards report it, from 21 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 23, 2026 Nemotron 3.5 Bolt E4B technical report 52.3 tau2-Bench, released Q4_K_M imatrix checkpoint (Table 5). BF16 55.1 (Table 3) Capabilities
Oct 19, 2026 Muse Glimmer technical report TauBench, three splits with no overall, high: Airline 78.0, Retail 87.1, Telecom 94.1. The telecom split suggests tau2-bench, though the text cites Yao et al. 2024 (Table 5) Capabilities
Oct 15, 2026 Nemotron 3.5 Super VL technical report Tau2 Retail 69.24, Airline 76.50, Telecom 73.19, no overall score (Table 10) Capabilities
Oct 15, 2026 Nemotron 3.5 Lightning 30B-A3B technical report Tau2 Telecom only, BF16 acc. 59.65 (Table 22) Capabilities
Jul 31, 2026 DiffusionGemma 26B A4B technical report Tau2, three domains with no overall score: Retail 71.5, Airline 69.0, Telecom 28.1, TD (Table 3) Capabilities
Jul 24, 2026 Gemma 4 31B IT technical report Tau2, three domains with no overall score: airline 75.0, retail 86.4, telecom 69.3 (Table 5) Capabilities
Jul 24, 2026 Gemma 4 26B A4B technical report Tau2, three domains with no overall score: airline 76.0, retail 85.5, telecom 43.0 (Table 5) Capabilities
Jun 13, 2026 Ring-2.6-1T technical report 83.4 τ2-Average (Acc), xhigh, Retail 77.85, Airline 77.50, Telecom 94.96 (Table 6). High gives Average 84.26, Telecom 96.71 Capabilities
Jun 13, 2026 Ling-2.6-1T technical report 78.4 τ2-bench, Mean@4, user model gpt-5.2, average of Retail, Airline and Telecom (Table 5) Capabilities
Jun 9, 2026 Keye-VL-2.0-30B-A3B technical report 82.6 τ2-Bench (Table 4) Capabilities
Jun 2, 2026 MAI-Code-1-Flash model card 71.7 tau2-Bench telecom, agentic tool use (Instruction following table) Capabilities
Apr 27, 2026 Nemotron 3 Nano Omni 30B-A3B technical report TauBench V2 (Telecom) only, 42.7, no overall score (Table 10) Capabilities
Apr 17, 2026 Qwen3.5-Omni-Plus technical report 81 TAU2Bench (Table 4) Capabilities
Apr 9, 2026 EXAONE 4.5 33B technical report 72 tau2-Bench weighted average stated in the 3.3 text. Retail 77.9, Airline 56.5, Telecom 73.0 (Table 3, Reasoning mode) Capabilities
Mar 26, 2026 Intern-S1-Pro technical report 80.9 τ2-Bench, official settings, thinking (Table 2) Capabilities
Mar 21, 2026 Seed1.8 model card 72 τ2-Bench (Table 4) Capabilities
Mar 19, 2026 Nemotron-Cascade 2 30B-A3B technical report 58.9 τ2-Bench, latest-turn thought retention, which the report says adds 3 to 5 points over the official policy (Table 1 and A.6) Capabilities
Mar 11, 2026 Nemotron 3 Super technical report 61.1 TauBench V2 Average of Airline 56.25, Retail 62.83, Telecom 64.36 (Table 5) Capabilities
Feb 19, 2026 Gemini 3.1 Pro model card τ2-bench, Retail 90.8% and Telecom 99.3%, no overall (results table, p.4) Capabilities
Feb 17, 2026 GLM-5 technical report 89.7 τ2-Bench, prompt-adjusted user simulator (Table 7) Capabilities
Feb 17, 2026 Claude Sonnet 4.6 system card τ²-bench, Retail 91.7 and Telecom 97.9, Airline not reported, no overall (Table 2.1.A, 2.5) Capabilities
Feb 14, 2026 Seed2.0 Pro model card τ2-Bench retail 90.4, telecom 94.2, no overall (Table 11) Capabilities
Feb 11, 2026 Step 3.5 Flash technical report 88.2 τ2-Bench, avg of 8 runs, fixed airline tasks (Table 5 and E.2.3) Capabilities
Feb 5, 2026 Claude Opus 4.6 system card τ²-bench, Retail 91.9 and Telecom 99.3, Airline not reported, no overall (Table 2.3.A, 2.7) Capabilities
Feb 4, 2026 ERNIE 5.0 technical report 78.8 TAU2-Bench (Table 2) Capabilities
Jan 23, 2026 LongCat-Flash-Thinking-2601 technical report 88.2 τ2-Avg (Avg@4), GPT-4.1 user simulator, cleaned airline subset, Retail 88.6, Airline 76.5, Telecom 99.3 (Table 2) Capabilities
Jan 11, 2026 Solar Open 100B technical report Tau2 Airline 52.4, Telecom 55.6, Retail 59.3, no overall score (Table 6) Capabilities
Jan 6, 2026 MiMo-V2-Flash technical report 80.3 τ2-Bench, DeepSeek-V3.2 as user agent (Table 9). Telecom 95.3, Retail 79.5, Airline 66.0 (4.5.2) Capabilities
Jan 5, 2026 Falcon H1R 7B technical report τ2-Telecom only, 25.4, no overall tau2-bench score (Table 5) Capabilities
Jan 5, 2026 K-EXAONE technical report 73.2 tau2-Bench weighted average stated in the 4.2 text. Retail 78.6, Airline 60.4, Telecom 73.5 (Table 3, Reasoning mode). Non-reasoning 73.2, 42.6, 44.0 (Table 4) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B technical report 49 TauBench V2 Average (Table 3). Airline 48.00, Retail 56.91, Telecom 42.21 Capabilities
Dec 2, 2025 Nova 2 Omni Preview technical report tau2-bench Telecom only 80.0, numbers as benchmarked by Artificial Analysis (Table 5) Capabilities
Dec 2, 2025 Nova 2 Lite technical report tau2-bench Telecom only 76.0, numbers as benchmarked by Artificial Analysis (Table 2) Capabilities
Dec 2, 2025 Nova 2.0 Pro Preview technical report tau2-bench Telecom only 92.7, numbers as benchmarked by Artificial Analysis (Table 2) Capabilities
Dec 1, 2025 DeepSeek-V3.2 technical report 80.3 τ2-Bench (Pass@1), average of categories: Airline 63.8, Retail 81.1, Telecom 96.2, DeepSeek-V3.2 Thinking (Table 2) Capabilities
Nov 26, 2025 Qwen3-VL-235B-A22B technical report TAU2-Retail 67.0, TAU2-Airline 62.0, TAU2-Telecom 44.7, no overall (Table 6) Capabilities
Nov 24, 2025 Claude Opus 4.5 system card τ²-Bench, Retail 88.9% and Telecom 98.2% without extended thinking, no overall (Table 2.3.A). Airline 67.9% original and 87.8% corrected per Table 2.3.A footnote 6. Table 2.8.A prints Airline original 70.1% Capabilities
Sep 23, 2025 LongCat-Flash-Thinking technical report τ2-Bench-Retail 71.5, Airline 67.5, Telecom 83.1 (Mean@4), no overall score (Table 2) Capabilities
Sep 1, 2025 LongCat-Flash-Chat technical report τ2-Bench telecom 73.68, airline 58.00, retail 71.27 (avg@4), no overall score (Table 3) Capabilities
Jul 21, 2025 Kimi K2 technical report 66.1 τ2-Bench micro Pass@1 across domains (Appendix C text). Avg@4: retail 70.6, airline 56.5, telecom 65.8 (Table 3) Capabilities

6 lab claims next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 5 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Kimi K2 Jul 2025 66.1% τ2-Bench micro Pass@1 across domains (Appendix C text). Avg@4: retail 70.6, airline 56.5, telecom 65.8 (Table 3) 64.3% Leaderboard +1.8 pts
GPT-5 80.1% Mean of Tau2-bench Function calling (airline) / Tau2-bench Function calling (retail) / Tau2-bench Function calling (telecom) 80% Leaderboard Same
Qwen3-Max 82.1% Tau² Bench 82.2% Leaderboard Same
DeepSeek-V3.2 80.3% τ2-Bench (Pass@1) 80.4% Leaderboard Same
Claude Sonnet 4.5 84.7% Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Airline / Agentic tool use τ2-bench Telecom 84.7% Leaderboard Same
Gemini 3 Pro 85.4% τ2-bench 85.4% Leaderboard Same

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic OpenAI Moonshot AI MiniMax Qwen Xiaomi Corp NVIDIA Inception Labs xAI
τ-bench 2.0 18 leaders
GPT-Live-1 79.9%
tau-bench Replaced by tau2-bench, Sep 2025
Claude Opus 4 70.5%
tau2-bench Passed 85% in Oct 2025, by MiniMax-M2
GPT-5.4 98.9%
tau3-bench Passed 85% in Jun 2026, by Nemotron 3 Ultra
Mercury 2.5 96%
τ-bench 2.0 2 leaders
GPT-Live-1 79.9%

Sources: Official leaderboards and labs’ model cards

Replaces tau-bench Replaced by tau3-bench

More in Tool use

All 144 in Tool use
  • AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state. Gemini 4 Carbon 53.9%
  • MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. Gemini 4 Argon 90.7%
  • BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes. Qwen3.7-Max 75%
  • τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. Qwen 3.8 Max 55.1%
  • Toolathlon-Verified In use 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. GLM-5.3-Flash 78.4%
  • DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%