Benchmark Almanac

Arena Expert

qwen3-235b-a22b-instruct-2507-invalid 1,223 tops its leaderboard DeepSeek-V3.2-Exp 1,143 leads the models we track
Not in model cards

Experts compare model answers to questions drawn from their professional work. Arena ranks models from their votes.

Arena publishes an expert leaderboard and a dataset of expert comparisons.

Made by
Arena
Released
November 2025
Kind
Expert-written questions
Level
Expert
Scored
Ranked by votes
Answers
Open-ended
Tasks
All public
Measures
Preference
Language
English
In and out
Text → Text
Interaction
Single turn
Items from
Written by experts
Refresh
Live
Run by
A host
In views
Kept fresh
Strongest model tested
GPT-5 #71 on the capability index
Last model added
September 2025
Scores refreshed
November 21st, 2026
90
Entries in its paper, 64 models we track

qwen3-235b-a22b-instruct-2507-invalid tops the leaderboard with 1,223 DeepSeek-V3.2-Exp leads with 1,143 among models released by September 2025

Best score per entry from the benchmark's paper, read on November 21st, 2026. A paper’s table doesn’t add newer models. Bradley-Terry ratings computed from the 5,128 expert prompt battles in the published dataset. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is DeepSeek-V3.2-Exp, at 1,143. This view shows only models we track, each at its best.

The newest model with a score here came out in September 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

90 entries. Scroll sideways for the rest. Bars start at 600.

  1. 1,223 qwen3-235b-a22b-instruct-2507-invalid
  2. 1,143 DeepSeek-V3.2-Exp
  3. 1,135 Qwen3-VL-235B-A22B
  4. 1,134 Qwen3-Max-Preview
  5. 1,123 Qwen3-235B-A22B-Thinking
  6. 1,106 Gemini 2.5 Pro
  7. 1,093 Gemini 2.5 Flash
  8. 1,093 claude-sonnet-4-5-20250929-thinking-32k
  9. 1,088 GLM-4.5
  10. 1,088 Qwen3-VL-235B-A22B-Instruct
  11. 1,082 GLM-4.5V
  12. 1,080 Grok 4 Fast
  13. 1,074 Grok 4
  14. 1,073 claude-sonnet-4-5-20250929-old
  15. 1,069 Kimi K2
  16. 1,066 Qwen3-235B-A22B-Instruct
  17. 1,063 gpt-5-old
  18. 1,062 Qwen3-Max
  19. 1,060 llama-4-maverick-03-26-experimental
  20. 1,058 Qwen3-30B-A3B-Instruct
  21. 1,057 DeepSeek-V3.1
  22. 1,057 Qwen3-Next-80B-A3B-Instruct
  23. 1,054 claude-opus-4-1-20250805-thinking-16k
  24. 1,052 Claude Opus 4.1
  25. 1,051 GPT-4.1
  26. 1,050 Ling-flash-2.0
  27. 1,045 Hunyuan-TurboS
  28. 1,041 Claude Opus 4
  29. 1,040 LongCat-Flash-Chat
  30. 1,039 GLM-4.6
  31. 1,038 GPT-5
  32. 1,033 mistral-medium-2508
  33. 1,033 Grok-3 mini
  34. 1,030 Step 3
  35. 1,030 o3
  36. 1,021 gpt-5-high-new-system-prompt
  37. 1,019 claude-sonnet-4-20250514-thinking-32k
  38. 1,018 MiniMax-M1-80k
  39. 1,012 GPT-5 mini
  40. 1,012 claude-opus-4-20250514-thinking-16k
  41. 1,009 DeepSeek-R1
  42. 1,008 chatgpt-4o-latest-20250326-old
  43. 1,007 Gemini 2.0 Flash Thinking
  44. 1,004 Qwen3-30B-A3B
  45. 1,004 Qwen3-Next-80B-A3B
  46. 1,003 Qwen-VL-Max
  47. 1,000 mai-1-preview
  48. 999 qwen3-235b-a22b-no-thinking
  49. 998 DeepSeek-V3.1-Terminus
  50. 997 GPT-5 Chat
  51. 995 o4-mini
  52. 993 GLM-4.5-Air
  53. 991 gemini-2.5-flash-lite-preview-09-2025-no-thinking
  54. 990 GPT-4o
  55. 988 qwen-max-2025-01-25
  56. 988 QwQ-32B
  57. 987 o3-mini
  58. 976 Qwen3-235B-A22B
  59. 972 gpt-oss-120b
  60. 970 Gemini 2.5 Flash-Lite
  61. 966 Claude Sonnet 4
  62. 960 gemini-2.0-flash-001
  63. 956 Hunyuan-T1
  64. 951 Grok 3
  65. 950 DeepSeek-V3
  66. 941 mistral-small-2506
  67. 941 GPT-4.1 mini
  68. 940 gpt-oss-20b
  69. 938 kimi-k2-0711-preview
  70. 936 Qwen3-Coder-480B-A35B
  71. 936 Claude 3.5 Sonnet
  72. 934 claude-3-7-sonnet-20250219-thinking-32k
  73. 928 mistral-medium-2505
  74. 919 Gemma 3 27B
  75. 914 GPT-5 nano
  76. 911 Llama 4 Maverick
  77. 904 Claude 3.7 Sonnet
  78. 890 nvidia-llama-3.3-nemotron-super-49b-v1.5
  79. 889 step-1o-turbo-202506
  80. 885 mistral-small-3.1-24b-instruct-2503
  81. 878 amazon-nova-experimental-chat-05-14
  82. 865 Llama 4 Scout
  83. 865 Claude 3.5 Haiku
  84. 862 command-a-03-2025
  85. 860 Ring-flash-2.0
  86. 835 Magistral Medium 1.0
  87. 834 Gemma 3n E4B
  88. 818 Llama 3.3 70B Instruct
  89. 796 amazon.nova-pro-v1:0
  90. 716 GPT-4o mini

Bars start at 600.

  1. 1 qwen3-235b-a22b-instruct-2507-invalid 1,223
  2. 2 DeepSeek-V3.2-Exp thinking 1,143
  3. 3 Qwen3-VL-235B-A22B thinking 1,135
  4. 4 Qwen3-Max-Preview 1,134
  5. 5 Qwen3-235B-A22B-Thinking Jul 2025 1,123
  6. 6 Gemini 2.5 Pro Jun 2025 1,106
  7. 7 Gemini 2.5 Flash Sep 2025 1,093
  8. 8 claude-sonnet-4-5-20250929-thinking-32k 1,093
  9. 9 GLM-4.5 1,088
  10. 10 Qwen3-VL-235B-A22B-Instruct 1,088
  11. 11 GLM-4.5V 1,082
  12. 12 Grok 4 Fast reasoning 1,080
  13. 13 Grok 4 1,074
  14. 14 claude-sonnet-4-5-20250929-old 1,073
  15. 15 Kimi K2 Sep 2025 1,069
  16. 16 Qwen3-235B-A22B-Instruct Jul 2025 1,066
  17. 17 gpt-5-old 1,063
  18. 18 Qwen3-Max 1,062
  19. 19 llama-4-maverick-03-26-experimental 1,060
  20. 20 Qwen3-30B-A3B-Instruct Jul 2025 1,058
  21. 21 DeepSeek-V3.1 1,057
  22. 22 Qwen3-Next-80B-A3B-Instruct 1,057
  23. 23 claude-opus-4-1-20250805-thinking-16k 1,054
  24. 24 Claude Opus 4.1 1,052
  25. 25 GPT-4.1 1,051
  26. 26 Ling-flash-2.0 1,050
  27. 27 Hunyuan-TurboS 1,045
  28. 28 Claude Opus 4 1,041
  29. 29 LongCat-Flash-Chat 1,040
  30. 30 GLM-4.6 1,039
  31. 31 GPT-5 high 1,038
  32. 32 mistral-medium-2508 1,033
  33. 33 Grok-3 mini 1,033
  34. 34 Step 3 1,030
  35. 35 o3 1,030
  36. 36 gpt-5-high-new-system-prompt 1,021
  37. 37 claude-sonnet-4-20250514-thinking-32k 1,019
  38. 38 MiniMax-M1-80k 1,018
  39. 39 GPT-5 mini high 1,012
  40. 40 claude-opus-4-20250514-thinking-16k 1,012
  41. 41 DeepSeek-R1 May 2025 1,009
  42. 42 chatgpt-4o-latest-20250326-old 1,008
  43. 43 Gemini 2.0 Flash Thinking Jan 2025 1,007
  44. 44 Qwen3-30B-A3B 1,004
  45. 45 Qwen3-Next-80B-A3B thinking 1,004
  46. 46 Qwen-VL-Max 1,003
  47. 47 mai-1-preview 1,000
  48. 48 qwen3-235b-a22b-no-thinking 999
  49. 49 DeepSeek-V3.1-Terminus thinking 998
  50. 50 GPT-5 Chat 997
  51. 51 o4-mini 995
  52. 52 GLM-4.5-Air 993
  53. 53 gemini-2.5-flash-lite-preview-09-2025-no-thinking 991
  54. 54 GPT-4o Nov 2024 990
  55. 55 qwen-max-2025-01-25 988
  56. 56 QwQ-32B 988
  57. 57 o3-mini 987
  58. 58 Qwen3-235B-A22B 976
  59. 59 gpt-oss-120b 972
  60. 60 Gemini 2.5 Flash-Lite Sep 2025, thinking 970
  61. 61 Claude Sonnet 4 966
  62. 62 gemini-2.0-flash-001 960
  63. 63 Hunyuan-T1 956
  64. 64 Grok 3 951
  65. 65 DeepSeek-V3 Mar 2025 950
  66. 66 mistral-small-2506 941
  67. 67 GPT-4.1 mini 941
  68. 68 gpt-oss-20b 940
  69. 69 kimi-k2-0711-preview 938
  70. 70 Qwen3-Coder-480B-A35B 936
  71. 71 Claude 3.5 Sonnet 936
  72. 72 claude-3-7-sonnet-20250219-thinking-32k 934
  73. 73 mistral-medium-2505 928
  74. 74 Gemma 3 27B 919
  75. 75 GPT-5 nano high 914
  76. 76 Llama 4 Maverick 911
  77. 77 Claude 3.7 Sonnet 904
  78. 78 nvidia-llama-3.3-nemotron-super-49b-v1.5 890
  79. 79 step-1o-turbo-202506 889
  80. 80 mistral-small-3.1-24b-instruct-2503 885
  81. 81 amazon-nova-experimental-chat-05-14 878
  82. 82 Llama 4 Scout 865
  83. 83 Claude 3.5 Haiku 865
  84. 84 command-a-03-2025 862
  85. 85 Ring-flash-2.0 860
  86. 86 Magistral Medium 1.0 835
  87. 87 Gemma 3n E4B 834
  88. 88 Llama 3.3 70B Instruct 818
  89. 89 amazon.nova-pro-v1:0 796
  90. 90 GPT-4o mini 716

Source: The Arena Expert paper

Bars start at 700.

  1. 1 DeepSeek-V3.2-Exp thinking 1,143
  2. 2 Qwen3-VL-235B-A22B thinking 1,135
  3. 3 Qwen3-Max-Preview 1,134
  4. 4 Qwen3-235B-A22B-Thinking Jul 2025 1,123
  5. 5 Gemini 2.5 Pro Jun 2025 1,106
  6. 6 Gemini 2.5 Flash Sep 2025 1,093
  7. 7 GLM-4.5 1,088
  8. 8 Qwen3-VL-235B-A22B-Instruct 1,088
  9. 9 GLM-4.5V 1,082
  10. 10 Grok 4 Fast reasoning 1,080
  11. 11 Grok 4 1,074
  12. 12 Kimi K2 Sep 2025 1,069
  13. 13 Qwen3-235B-A22B-Instruct Jul 2025 1,066
  14. 14 Qwen3-Max 1,062
  15. 15 Qwen3-30B-A3B-Instruct Jul 2025 1,058
  16. 16 DeepSeek-V3.1 1,057
  17. 17 Qwen3-Next-80B-A3B-Instruct 1,057
  18. 18 Claude Opus 4.1 1,052
  19. 19 GPT-4.1 1,051
  20. 20 Ling-flash-2.0 1,050
  21. 21 Hunyuan-TurboS 1,045
  22. 22 Claude Opus 4 1,041
  23. 23 LongCat-Flash-Chat 1,040
  24. 24 GLM-4.6 1,039
  25. 25 GPT-5 high 1,038
  26. 26 Grok-3 mini 1,033
  27. 27 Step 3 1,030
  28. 28 o3 1,030
  29. 29 MiniMax-M1-80k 1,018
  30. 30 GPT-5 mini high 1,012
  31. 31 DeepSeek-R1 May 2025 1,009
  32. 32 Gemini 2.0 Flash Thinking Jan 2025 1,007
  33. 33 Qwen3-30B-A3B 1,004
  34. 34 Qwen3-Next-80B-A3B thinking 1,004
  35. 35 Qwen-VL-Max 1,003
  36. 36 DeepSeek-V3.1-Terminus thinking 998
  37. 37 GPT-5 Chat 997
  38. 38 o4-mini 995
  39. 39 GLM-4.5-Air 993
  40. 40 GPT-4o Nov 2024 990
  41. 41 QwQ-32B 988
  42. 42 o3-mini 987
  43. 43 Qwen3-235B-A22B 976
  44. 44 gpt-oss-120b 972
  45. 45 Gemini 2.5 Flash-Lite Sep 2025, thinking 970
  46. 46 Claude Sonnet 4 966
  47. 47 Hunyuan-T1 956
  48. 48 Grok 3 951
  49. 49 DeepSeek-V3 Mar 2025 950
  50. 50 GPT-4.1 mini 941
  51. 51 gpt-oss-20b 940
  52. 52 Qwen3-Coder-480B-A35B 936
  53. 53 Claude 3.5 Sonnet 936
  54. 54 Gemma 3 27B 919
  55. 55 GPT-5 nano high 914
  56. 56 Llama 4 Maverick 911
  57. 57 Claude 3.7 Sonnet 904
  58. 58 Llama 4 Scout 865
  59. 59 Claude 3.5 Haiku 865
  60. 60 Ring-flash-2.0 860
  61. 61 Magistral Medium 1.0 835
  62. 62 Gemma 3n E4B 834
  63. 63 Llama 3.3 70B Instruct 818
  64. 64 GPT-4o mini 716

Source: The Arena Expert paper

#EntryScoreSettingDateSource
1 qwen3-235b-a22b-instruct-2507-invalid 1,223 Official leaderboard
2 DeepSeek-V3.2-Exp 1,143 thinking Sep 2025 Official leaderboard
3 Qwen3-VL-235B-A22B 1,135 thinking Sep 2025 Official leaderboard
4 Qwen3-Max-Preview 1,134 Sep 2025 Official leaderboard
5 Qwen3-235B-A22B-Thinking Jul 2025 1,123 Jul 2025 Official leaderboard
6 Gemini 2.5 Pro Jun 2025 1,106 Jun 2025 Official leaderboard
7 Gemini 2.5 Flash Sep 2025 1,093 Sep 2025 Official leaderboard
8 claude-sonnet-4-5-20250929-thinking-32k 1,093 Official leaderboard
9 GLM-4.5 1,088 Aug 2025 Official leaderboard
10 Qwen3-VL-235B-A22B-Instruct 1,088 Sep 2025 Official leaderboard
11 GLM-4.5V 1,082 Aug 2025 Official leaderboard
12 Grok 4 Fast 1,080 reasoning Sep 2025 Official leaderboard
13 Grok 4 1,074 Jul 2025 Official leaderboard
14 claude-sonnet-4-5-20250929-old 1,073 Official leaderboard
15 Kimi K2 Sep 2025 1,069 Sep 2025 Official leaderboard
16 Qwen3-235B-A22B-Instruct Jul 2025 1,066 Jul 2025 Official leaderboard
17 gpt-5-old 1,063 Official leaderboard
18 Qwen3-Max 1,062 Sep 2025 Official leaderboard
19 llama-4-maverick-03-26-experimental 1,060 Official leaderboard
20 Qwen3-30B-A3B-Instruct Jul 2025 1,058 Jul 2025 Official leaderboard
21 DeepSeek-V3.1 1,057 Aug 2025 Official leaderboard
22 Qwen3-Next-80B-A3B-Instruct 1,057 Sep 2025 Official leaderboard
23 claude-opus-4-1-20250805-thinking-16k 1,054 Official leaderboard
24 Claude Opus 4.1 1,052 Aug 2025 Official leaderboard
25 GPT-4.1 1,051 Apr 2025 Official leaderboard
26 Ling-flash-2.0 1,050 Sep 2025 Official leaderboard
27 Hunyuan-TurboS 1,045 Feb 2025 Official leaderboard
28 Claude Opus 4 1,041 May 2025 Official leaderboard
29 LongCat-Flash-Chat 1,040 Aug 2025 Official leaderboard
30 GLM-4.6 1,039 Sep 2025 Official leaderboard
31 GPT-5 1,038 high Aug 2025 Official leaderboard
32 mistral-medium-2508 1,033 Official leaderboard
33 Grok-3 mini 1,033 Apr 2025 Official leaderboard
34 Step 3 1,030 Jul 2025 Official leaderboard
35 o3 1,030 Apr 2025 Official leaderboard
36 gpt-5-high-new-system-prompt 1,021 Official leaderboard
37 claude-sonnet-4-20250514-thinking-32k 1,019 Official leaderboard
38 MiniMax-M1-80k 1,018 Jun 2025 Official leaderboard
39 GPT-5 mini 1,012 high Aug 2025 Official leaderboard
40 claude-opus-4-20250514-thinking-16k 1,012 Official leaderboard
41 DeepSeek-R1 May 2025 1,009 May 2025 Official leaderboard
42 chatgpt-4o-latest-20250326-old 1,008 Official leaderboard
43 Gemini 2.0 Flash Thinking Jan 2025 1,007 Jan 2025 Official leaderboard
44 Qwen3-30B-A3B 1,004 Apr 2025 Official leaderboard
45 Qwen3-Next-80B-A3B 1,004 thinking Sep 2025 Official leaderboard
46 Qwen-VL-Max 1,003 Jan 2024 Official leaderboard
47 mai-1-preview 1,000 Official leaderboard
48 qwen3-235b-a22b-no-thinking 999 Official leaderboard
49 DeepSeek-V3.1-Terminus 998 thinking Sep 2025 Official leaderboard
50 GPT-5 Chat 997 Aug 2025 Official leaderboard
51 o4-mini 995 Apr 2025 Official leaderboard
52 GLM-4.5-Air 993 Jul 2025 Official leaderboard
53 gemini-2.5-flash-lite-preview-09-2025-no-thinking 991 Official leaderboard
54 GPT-4o Nov 2024 990 Nov 2024 Official leaderboard
55 qwen-max-2025-01-25 988 Official leaderboard
56 QwQ-32B 988 Mar 2025 Official leaderboard
57 o3-mini 987 Jan 2025 Official leaderboard
58 Qwen3-235B-A22B 976 Apr 2025 Official leaderboard
59 gpt-oss-120b 972 Aug 2025 Official leaderboard
60 Gemini 2.5 Flash-Lite Sep 2025 970 thinking Sep 2025 Official leaderboard
61 Claude Sonnet 4 966 May 2025 Official leaderboard
62 gemini-2.0-flash-001 960 Official leaderboard
63 Hunyuan-T1 956 Mar 2025 Official leaderboard
64 Grok 3 951 Apr 2025 Official leaderboard
65 DeepSeek-V3 Mar 2025 950 Mar 2025 Official leaderboard
66 mistral-small-2506 941 Official leaderboard
67 GPT-4.1 mini 941 Apr 2025 Official leaderboard
68 gpt-oss-20b 940 Aug 2025 Official leaderboard
69 kimi-k2-0711-preview 938 Official leaderboard
70 Qwen3-Coder-480B-A35B 936 Jul 2025 Official leaderboard
71 Claude 3.5 Sonnet 936 Jun 2024 Official leaderboard
72 claude-3-7-sonnet-20250219-thinking-32k 934 Official leaderboard
73 mistral-medium-2505 928 Official leaderboard
74 Gemma 3 27B 919 Mar 2025 Official leaderboard
75 GPT-5 nano 914 high Aug 2025 Official leaderboard
76 Llama 4 Maverick 911 Apr 2025 Official leaderboard
77 Claude 3.7 Sonnet 904 Feb 2025 Official leaderboard
78 nvidia-llama-3.3-nemotron-super-49b-v1.5 890 Official leaderboard
79 step-1o-turbo-202506 889 Official leaderboard
80 mistral-small-3.1-24b-instruct-2503 885 Official leaderboard
81 amazon-nova-experimental-chat-05-14 878 Official leaderboard
82 Llama 4 Scout 865 Apr 2025 Official leaderboard
83 Claude 3.5 Haiku 865 Oct 2024 Official leaderboard
84 command-a-03-2025 862 Official leaderboard
85 Ring-flash-2.0 860 Sep 2025 Official leaderboard
86 Magistral Medium 1.0 835 Jun 2025 Official leaderboard
87 Gemma 3n E4B 834 Jun 2025 Official leaderboard
88 Llama 3.3 70B Instruct 818 Dec 2024 Official leaderboard
89 amazon.nova-pro-v1:0 796 Official leaderboard
90 GPT-4o mini 716 Jul 2024 Official leaderboard
#ModelScoreSettingDateSource
1 DeepSeek-V3.2-Exp 1,143 thinking Sep 2025 Official leaderboard
2 Qwen3-VL-235B-A22B 1,135 thinking Sep 2025 Official leaderboard
3 Qwen3-Max-Preview 1,134 Sep 2025 Official leaderboard
4 Qwen3-235B-A22B-Thinking Jul 2025 1,123 Jul 2025 Official leaderboard
5 Gemini 2.5 Pro Jun 2025 1,106 Jun 2025 Official leaderboard
6 Gemini 2.5 Flash Sep 2025 1,093 Sep 2025 Official leaderboard
7 GLM-4.5 1,088 Aug 2025 Official leaderboard
8 Qwen3-VL-235B-A22B-Instruct 1,088 Sep 2025 Official leaderboard
9 GLM-4.5V 1,082 Aug 2025 Official leaderboard
10 Grok 4 Fast 1,080 reasoning Sep 2025 Official leaderboard
11 Grok 4 1,074 Jul 2025 Official leaderboard
12 Kimi K2 Sep 2025 1,069 Sep 2025 Official leaderboard
13 Qwen3-235B-A22B-Instruct Jul 2025 1,066 Jul 2025 Official leaderboard
14 Qwen3-Max 1,062 Sep 2025 Official leaderboard
15 Qwen3-30B-A3B-Instruct Jul 2025 1,058 Jul 2025 Official leaderboard
16 DeepSeek-V3.1 1,057 Aug 2025 Official leaderboard
17 Qwen3-Next-80B-A3B-Instruct 1,057 Sep 2025 Official leaderboard
18 Claude Opus 4.1 1,052 Aug 2025 Official leaderboard
19 GPT-4.1 1,051 Apr 2025 Official leaderboard
20 Ling-flash-2.0 1,050 Sep 2025 Official leaderboard
21 Hunyuan-TurboS 1,045 Feb 2025 Official leaderboard
22 Claude Opus 4 1,041 May 2025 Official leaderboard
23 LongCat-Flash-Chat 1,040 Aug 2025 Official leaderboard
24 GLM-4.6 1,039 Sep 2025 Official leaderboard
25 GPT-5 1,038 high Aug 2025 Official leaderboard
26 Grok-3 mini 1,033 Apr 2025 Official leaderboard
27 Step 3 1,030 Jul 2025 Official leaderboard
28 o3 1,030 Apr 2025 Official leaderboard
29 MiniMax-M1-80k 1,018 Jun 2025 Official leaderboard
30 GPT-5 mini 1,012 high Aug 2025 Official leaderboard
31 DeepSeek-R1 May 2025 1,009 May 2025 Official leaderboard
32 Gemini 2.0 Flash Thinking Jan 2025 1,007 Jan 2025 Official leaderboard
33 Qwen3-30B-A3B 1,004 Apr 2025 Official leaderboard
34 Qwen3-Next-80B-A3B 1,004 thinking Sep 2025 Official leaderboard
35 Qwen-VL-Max 1,003 Jan 2024 Official leaderboard
36 DeepSeek-V3.1-Terminus 998 thinking Sep 2025 Official leaderboard
37 GPT-5 Chat 997 Aug 2025 Official leaderboard
38 o4-mini 995 Apr 2025 Official leaderboard
39 GLM-4.5-Air 993 Jul 2025 Official leaderboard
40 GPT-4o Nov 2024 990 Nov 2024 Official leaderboard
41 QwQ-32B 988 Mar 2025 Official leaderboard
42 o3-mini 987 Jan 2025 Official leaderboard
43 Qwen3-235B-A22B 976 Apr 2025 Official leaderboard
44 gpt-oss-120b 972 Aug 2025 Official leaderboard
45 Gemini 2.5 Flash-Lite Sep 2025 970 thinking Sep 2025 Official leaderboard
46 Claude Sonnet 4 966 May 2025 Official leaderboard
47 Hunyuan-T1 956 Mar 2025 Official leaderboard
48 Grok 3 951 Apr 2025 Official leaderboard
49 DeepSeek-V3 Mar 2025 950 Mar 2025 Official leaderboard
50 GPT-4.1 mini 941 Apr 2025 Official leaderboard
51 gpt-oss-20b 940 Aug 2025 Official leaderboard
52 Qwen3-Coder-480B-A35B 936 Jul 2025 Official leaderboard
53 Claude 3.5 Sonnet 936 Jun 2024 Official leaderboard
54 Gemma 3 27B 919 Mar 2025 Official leaderboard
55 GPT-5 nano 914 high Aug 2025 Official leaderboard
56 Llama 4 Maverick 911 Apr 2025 Official leaderboard
57 Claude 3.7 Sonnet 904 Feb 2025 Official leaderboard
58 Llama 4 Scout 865 Apr 2025 Official leaderboard
59 Claude 3.5 Haiku 865 Oct 2024 Official leaderboard
60 Ring-flash-2.0 860 Sep 2025 Official leaderboard
61 Magistral Medium 1.0 835 Jun 2025 Official leaderboard
62 Gemma 3n E4B 834 Jun 2025 Official leaderboard
63 Llama 3.3 70B Instruct 818 Dec 2024 Official leaderboard
64 GPT-4o mini 716 Jul 2024 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
DeepSeek-V3.2-Exp
thinking
1,143
Grok 4 Fast
reasoning
1,080
Grok-3 mini
high
1,033
DeepSeek-V3.1-Terminus
thinking
998

Source: The Arena Expert paper

ModelEffortHarnessScoreCost per taskOutput tokens
DeepSeek-V3.2-Exp thinking 1,143
Qwen3-VL-235B-A22B thinking 1,135
Qwen3-Max-Preview 1,134
Qwen3-235B-A22B-Thinking Jul 2025 1,123
Gemini 2.5 Pro Jun 2025 1,106
Gemini 2.5 Pro Jun 2025 1,106
Gemini 2.5 Flash Sep 2025 1,093
GLM-4.5 1,088
Qwen3-VL-235B-A22B-Instruct 1,088
GLM-4.5V 1,082
Grok 4 Fast reasoning 1,080
Grok 4 1,074
DeepSeek-V3.2-Exp 1,074
Kimi K2 Sep 2025 1,069
Qwen3-235B-A22B-Instruct Jul 2025 1,066
Qwen3-Max 1,062
Qwen3-30B-A3B-Instruct Jul 2025 1,058
DeepSeek-V3.1 1,057
Qwen3-Next-80B-A3B-Instruct 1,057
Claude Opus 4.1 1,052
GPT-4.1 1,051
Ling-flash-2.0 1,050
Hunyuan-TurboS 1,045
Claude Opus 4 1,041
LongCat-Flash-Chat 1,040
GLM-4.6 1,039
GPT-5 high 1,038
Grok-3 mini 1,033
Step 3 1,030
o3 1,030
DeepSeek-V3.1 1,026
Grok-3 mini high 1,024
MiniMax-M1-80k 1,018
GPT-5 mini high 1,012
DeepSeek-R1 May 2025 1,009
Gemini 2.0 Flash Thinking Jan 2025 1,007
Qwen3-30B-A3B 1,004
Qwen3-Next-80B-A3B thinking 1,004
Qwen-VL-Max 1,003
Grok 4 Fast 1,001
Gemini 2.5 Flash Sep 2025 999
DeepSeek-V3.1-Terminus thinking 998
GPT-5 Chat 997
o4-mini 995
GLM-4.5-Air 993
GPT-4o Nov 2024 990
QwQ-32B 988
o3-mini 987
Qwen3-Max 985
Qwen3-235B-A22B 976
gpt-oss-120b 972
DeepSeek-V3.1-Terminus 971
Gemini 2.5 Flash-Lite Sep 2025 thinking 970
Claude Sonnet 4 966
Hunyuan-T1 956
Grok 3 951
DeepSeek-V3 Mar 2025 950
GPT-4.1 mini 941
gpt-oss-20b 940
Qwen3-Coder-480B-A35B 936
Claude 3.5 Sonnet 936
Gemma 3 27B 919
GPT-5 nano high 914
Llama 4 Maverick 911
Claude 3.7 Sonnet 904
Llama 4 Scout 865
Claude 3.5 Haiku 865
Ring-flash-2.0 860
Magistral Medium 1.0 835
Gemma 3n E4B 834
Llama 3.3 70B Instruct 818
GPT-4o mini 716

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
DeepSeek-V3.2-Exp
thinking
1,143
Grok 4 Fast
reasoning
1,080
Grok-3 mini
high
1,033
DeepSeek-V3.1-Terminus
thinking
998

Source: The Arena Expert paper

ModelEffortHarnessScoreCost per taskOutput tokens
DeepSeek-V3.2-Exp thinking 1,143
Qwen3-VL-235B-A22B thinking 1,135
Qwen3-Max-Preview 1,134
Qwen3-235B-A22B-Thinking Jul 2025 1,123
Gemini 2.5 Pro Jun 2025 1,106
Gemini 2.5 Pro Jun 2025 1,106
Gemini 2.5 Flash Sep 2025 1,093
GLM-4.5 1,088
Qwen3-VL-235B-A22B-Instruct 1,088
GLM-4.5V 1,082
Grok 4 Fast reasoning 1,080
Grok 4 1,074
DeepSeek-V3.2-Exp 1,074
Kimi K2 Sep 2025 1,069
Qwen3-235B-A22B-Instruct Jul 2025 1,066
Qwen3-Max 1,062
Qwen3-30B-A3B-Instruct Jul 2025 1,058
DeepSeek-V3.1 1,057
Qwen3-Next-80B-A3B-Instruct 1,057
Claude Opus 4.1 1,052
GPT-4.1 1,051
Ling-flash-2.0 1,050
Hunyuan-TurboS 1,045
Claude Opus 4 1,041
LongCat-Flash-Chat 1,040
GLM-4.6 1,039
GPT-5 high 1,038
Grok-3 mini 1,033
Step 3 1,030
o3 1,030
DeepSeek-V3.1 1,026
Grok-3 mini high 1,024
MiniMax-M1-80k 1,018
GPT-5 mini high 1,012
DeepSeek-R1 May 2025 1,009
Gemini 2.0 Flash Thinking Jan 2025 1,007
Qwen3-30B-A3B 1,004
Qwen3-Next-80B-A3B thinking 1,004
Qwen-VL-Max 1,003
Grok 4 Fast 1,001
Gemini 2.5 Flash Sep 2025 999
DeepSeek-V3.1-Terminus thinking 998
GPT-5 Chat 997
o4-mini 995
GLM-4.5-Air 993
GPT-4o Nov 2024 990
QwQ-32B 988
o3-mini 987
Qwen3-Max 985
Qwen3-235B-A22B 976
gpt-oss-120b 972
DeepSeek-V3.1-Terminus 971
Gemini 2.5 Flash-Lite Sep 2025 thinking 970
Claude Sonnet 4 966
Hunyuan-T1 956
Grok 3 951
DeepSeek-V3 Mar 2025 950
GPT-4.1 mini 941
gpt-oss-20b 940
Qwen3-Coder-480B-A35B 936
Claude 3.5 Sonnet 936
Gemma 3 27B 919
GPT-5 nano high 914
Llama 4 Maverick 911
Claude 3.7 Sonnet 904
Llama 4 Scout 865
Claude 3.5 Haiku 865
Ring-flash-2.0 860
Magistral Medium 1.0 835
Gemma 3n E4B 834
Llama 3.3 70B Instruct 818
GPT-4o mini 716

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

DeepSeek
Benchmark paper Not tested since Nov 2025

Source: The Arena Expert paper

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
In the benchmark’s paper
700 800 900 1,000 1,100 1,200 2024 2025 700 800 900 1,000 1,100 1,200 2024 2025

Source: Official Arena Expert leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Knowledge

All 332 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • MMLU In use 57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy. Seed1.8 92.3%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • HLE-Diamond In use Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers. Claude Fable 5.2 77.9%

Data audit

8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 14 months ago, in September 2025, for GLM-4.6. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
  • Pass Independent runs Scores from the benchmark’s paper (90 entries).
  • Pass Models scored 64 models we track, plus 26 other entries on its leaderboard.
  • Warning Newest score The newest came 14 months ago, in September 2025, for GLM-4.6.
  • Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5 (#71).
  • Note Headroom Scored as a rating, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on arena.ai: 72 rows for models we track and 26 other entries at the last read.
  • Pass Last read November 21st, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check