Benchmark Almanac

TeleQnA

OTel-LLM-8.3B-QnA 91.2% tops its leaderboard Gemini 3.1 Pro 85.2% leads the models we track
Not in model cards

Models answer multiple-choice questions about telecommunications terminology, research, and technical standards.

The authors compare models on 10,000 questions and test how much relevant context helps.

Made by
Ali Maatouk and colleagues
Released
October 2023
Kind
Expert-written questions
Level
Expert
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Claude Opus 4.6 #39 on the capability index
Last model added
February 2026
Scores refreshed
November 20th, 2026
90
Entries on its own leaderboard, 69 models we track

OTel-LLM-8.3B-QnA tops the leaderboard with 91.2% Gemini 3.1 Pro leads with 85.2% among models released by February 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Gemini 3.1 Pro, at 85.2%. This view shows only models we track, each at its best.

The newest model with a score here came out in February 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

90 entries. Scroll sideways for the rest.

  1. 91.2 OTel-LLM-8.3B-QnA
  2. 89.4 TeleLLM
  3. 87.8 CHT-TeleX-8B
  4. 85.2 Gemini 3.1 Pro
  5. 84.5 Claude Opus 4.5
  6. 84.4 Claude Opus 4.6
  7. 83.8 GPT-5
  8. 83.6 Grok 4 Fast
  9. 83.6 Kimi K2.5
  10. 83.4 o3
  11. 83.2 Gemini 3 Flash
  12. 83.0 Claude Sonnet 4
  13. 82.9 Gemini 3 Pro
  14. 82.7 GPT-5.1
  15. 82.7 GPT-5.2
  16. 82.4 Claude Opus 4
  17. 82.3 Claude Opus 4.1
  18. 82.3 Claude Sonnet 4.5
  19. 82.1 netLLM
  20. 82.1 Claude 3.5 Sonnet
  21. 81.9 LTM
  22. 81.9 o1
  23. 81.8 Gemini 2.5 Pro
  24. 81.7 GPT-4.1
  25. 81.6 Claude 3.7 Sonnet
  26. 81.6 DeepSeek-V3
  27. 81.3 Grok 4.1 Fast
  28. 80.8 o4-mini
  29. 80.6 TSLAM-150B
  30. 80.6 Grok-3 mini
  31. 80.5 Gemini 2.0 Flash
  32. 80.2 Llama 4 Maverick
  33. 79.9 gpt-oss-120b
  34. 79.7 Qwen3-235B-A22B-Instruct
  35. 79.5 GPT-4.1 mini
  36. 79.1 MiniMax-M2.5
  37. 78.9 Claude Haiku 4.5
  38. 78.8 Gemini 2.5 Flash
  39. 78.7 TSLAM-30B
  40. 78.6 MiMo-V2-Flash
  41. 78.5 grok-code-fast-1
  42. 78.4 Qwen3-32B
  43. 77.5 Cohere Command A
  44. 77.3 Llama 3.3 70B Instruct
  45. 76.8 GPT-5 nano
  46. 76.7 GPT-4
  47. 76.5 Qwen2.5-72B
  48. 75.9 telco-slm-4B
  49. 75.1 GPT-4o mini
  50. 75.0 Qwen2.5-32B
  51. 74.5 Qwen3-8B
  52. 74.0 Claude 3.5 Haiku
  53. 73.7 mistral-small-24b
  54. 73.1 Phi-4
  55. 72.6 GPT-4.1 nano
  56. 72.6 DeepSeek-R1
  57. 72.5 Qwen2.5-14B
  58. 72.2 Claude 3 Haiku
  59. 71.8 Gemma 2 27B
  60. 71.3 Gemma 3 27B
  61. 71.3 GPT-3.5 Turbo
  62. 71.3 DeepSeek-V3.2
  63. 70.2 Qwen2.5-7B
  64. 69.4 Gemma 2 9B
  65. 68.8 Gemma 3 12B
  66. 68.7 falcon3-10b
  67. 67.5 granite3.2-8b
  68. 67.2 internlm2.5-20b
  69. 67.0 mistral-small-22b
  70. 66.9 Qwen2.5-Coder (7B)
  71. 66.9 internlm2.5-7b
  72. 66.4 Qwen2.5-3B
  73. 65.8 granite3.3-8b
  74. 65.2 Gemma 3 4B
  75. 64.9 Phi-4 Mini
  76. 64.2 mistral-nemo-12b
  77. 62.2 Phi-3.5-mini
  78. 61.0 Qwen2.5-1.5B
  79. 60.8 falcon3-3b
  80. 60.5 gemma2-2b
  81. 51.7 Command R7B
  82. 51.1 deepseek-r1-32b
  83. 48.8 Mixtral 8x7B
  84. 46.4 Qwen2.5-0.5B
  85. 45.7 Gemma 3 1B
  86. 45.3 Mistral 7B v0.1
  87. 44.8 Claude Sonnet 4.6
  88. 44.7 GPT-5 mini
  89. 37.0 falcon3-1b
  90. 16.1 falcon3-7b
  1. 1 OTel-LLM-8.3B-QnA 91.2%
  2. 2 TeleLLM 89.4%
  3. 3 CHT-TeleX-8B 87.8%
  4. 4 Gemini 3.1 Pro 85.2%
  5. 5 Claude Opus 4.5 84.5%
  6. 6 Claude Opus 4.6 84.4%
  7. 7 GPT-5 83.8%
  8. 8 Grok 4 Fast 83.6%
  9. 9 Kimi K2.5 83.6%
  10. 10 o3 83.4%
  11. 11 Gemini 3 Flash 83.2%
  12. 12 Claude Sonnet 4 83%
  13. 13 Gemini 3 Pro 82.9%
  14. 14 GPT-5.1 82.7%
  15. 15 GPT-5.2 82.7%
  16. 16 Claude Opus 4 82.4%
  17. 17 Claude Opus 4.1 82.3%
  18. 18 Claude Sonnet 4.5 82.3%
  19. 19 netLLM 82.1%
  20. 20 Claude 3.5 Sonnet 82.1%
  21. 21 LTM 81.9%
  22. 22 o1 81.9%
  23. 23 Gemini 2.5 Pro Jun 2025 81.8%
  24. 24 GPT-4.1 81.7%
  25. 25 Claude 3.7 Sonnet 81.6%
  26. 26 DeepSeek-V3 Mar 2025 81.6%
  27. 27 Grok 4.1 Fast 81.3%
  28. 28 o4-mini 80.8%
  29. 29 TSLAM-150B 80.6%
  30. 30 Grok-3 mini 80.6%
  31. 31 Gemini 2.0 Flash Feb 2025 80.5%
  32. 32 Llama 4 Maverick 80.2%
  33. 33 gpt-oss-120b 79.9%
  34. 34 Qwen3-235B-A22B-Instruct Jul 2025 79.7%
  35. 35 GPT-4.1 mini 79.5%
  36. 36 MiniMax-M2.5 79.1%
  37. 37 Claude Haiku 4.5 78.9%
  38. 38 Gemini 2.5 Flash Sep 2025 78.8%
  39. 39 TSLAM-30B 78.7%
  40. 40 MiMo-V2-Flash 78.6%
  41. 41 grok-code-fast-1 78.5%
  42. 42 Qwen3-32B 78.4%
  43. 43 Cohere Command A 77.5%
  44. 44 Llama 3.3 70B Instruct 77.3%
  45. 45 GPT-5 nano 76.8%
  46. 46 GPT-4 Jun 2023 76.7%
  47. 47 Qwen2.5-72B 76.5%
  48. 48 telco-slm-4B 75.9%
  49. 49 GPT-4o mini 75.1%
  50. 50 Qwen2.5-32B 75%
  51. 51 Qwen3-8B 74.5%
  52. 52 Claude 3.5 Haiku 74%
  53. 53 mistral-small-24b 73.7%
  54. 54 Phi-4 73.1%
  55. 55 GPT-4.1 nano 72.6%
  56. 56 DeepSeek-R1 May 2025 72.6%
  57. 57 Qwen2.5-14B 72.5%
  58. 58 Claude 3 Haiku 72.2%
  59. 59 Gemma 2 27B 71.8%
  60. 60 Gemma 3 27B 71.3%
  61. 61 GPT-3.5 Turbo Jan 2024 71.3%
  62. 62 DeepSeek-V3.2 71.3%
  63. 63 Qwen2.5-7B 70.2%
  64. 64 Gemma 2 9B 69.4%
  65. 65 Gemma 3 12B 68.8%
  66. 66 falcon3-10b 68.7%
  67. 67 granite3.2-8b 67.5%
  68. 68 internlm2.5-20b 67.2%
  69. 69 mistral-small-22b 67%
  70. 70 Qwen2.5-Coder (7B) 66.9%
  71. 71 internlm2.5-7b 66.9%
  72. 72 Qwen2.5-3B 66.4%
  73. 73 granite3.3-8b 65.8%
  74. 74 Gemma 3 4B 65.2%
  75. 75 Phi-4 Mini 64.9%
  76. 76 mistral-nemo-12b 64.2%
  77. 77 Phi-3.5-mini 62.2%
  78. 78 Qwen2.5-1.5B 61%
  79. 79 falcon3-3b 60.8%
  80. 80 gemma2-2b 60.5%
  81. 81 Command R7B 51.7%
  82. 82 deepseek-r1-32b 51.1%
  83. 83 Mixtral 8x7B 48.8%
  84. 84 Qwen2.5-0.5B 46.4%
  85. 85 Gemma 3 1B 45.7%
  86. 86 Mistral 7B v0.1 45.3%
  87. 87 Claude Sonnet 4.6 44.8%
  88. 88 GPT-5 mini 44.7%
  89. 89 falcon3-1b 37%
  90. 90 falcon3-7b 16.1%

Source: Official TeleQnA leaderboard

  1. 1 Gemini 3.1 Pro 85.2%
  2. 2 Claude Opus 4.5 84.5%
  3. 3 Claude Opus 4.6 84.4%
  4. 4 GPT-5 83.8%
  5. 5 Grok 4 Fast 83.6%
  6. 6 Kimi K2.5 83.6%
  7. 7 o3 83.4%
  8. 8 Gemini 3 Flash 83.2%
  9. 9 Claude Sonnet 4 83%
  10. 10 Gemini 3 Pro 82.9%
  11. 11 GPT-5.1 82.7%
  12. 12 GPT-5.2 82.7%
  13. 13 Claude Opus 4 82.4%
  14. 14 Claude Opus 4.1 82.3%
  15. 15 Claude Sonnet 4.5 82.3%
  16. 16 Claude 3.5 Sonnet 82.1%
  17. 17 o1 81.9%
  18. 18 Gemini 2.5 Pro Jun 2025 81.8%
  19. 19 GPT-4.1 81.7%
  20. 20 Claude 3.7 Sonnet 81.6%
  21. 21 DeepSeek-V3 Mar 2025 81.6%
  22. 22 Grok 4.1 Fast 81.3%
  23. 23 o4-mini 80.8%
  24. 24 Grok-3 mini 80.6%
  25. 25 Gemini 2.0 Flash Feb 2025 80.5%
  26. 26 Llama 4 Maverick 80.2%
  27. 27 gpt-oss-120b 79.9%
  28. 28 Qwen3-235B-A22B-Instruct Jul 2025 79.7%
  29. 29 GPT-4.1 mini 79.5%
  30. 30 MiniMax-M2.5 79.1%
  31. 31 Claude Haiku 4.5 78.9%
  32. 32 Gemini 2.5 Flash Sep 2025 78.8%
  33. 33 MiMo-V2-Flash 78.6%
  34. 34 grok-code-fast-1 78.5%
  35. 35 Qwen3-32B 78.4%
  36. 36 Cohere Command A 77.5%
  37. 37 Llama 3.3 70B Instruct 77.3%
  38. 38 GPT-5 nano 76.8%
  39. 39 GPT-4 Jun 2023 76.7%
  40. 40 Qwen2.5-72B 76.5%
  41. 41 GPT-4o mini 75.1%
  42. 42 Qwen2.5-32B 75%
  43. 43 Qwen3-8B 74.5%
  44. 44 Claude 3.5 Haiku 74%
  45. 45 Phi-4 73.1%
  46. 46 GPT-4.1 nano 72.6%
  47. 47 DeepSeek-R1 May 2025 72.6%
  48. 48 Qwen2.5-14B 72.5%
  49. 49 Claude 3 Haiku 72.2%
  50. 50 Gemma 2 27B 71.8%
  51. 51 Gemma 3 27B 71.3%
  52. 52 GPT-3.5 Turbo Jan 2024 71.3%
  53. 53 DeepSeek-V3.2 71.3%
  54. 54 Qwen2.5-7B 70.2%
  55. 55 Gemma 2 9B 69.4%
  56. 56 Gemma 3 12B 68.8%
  57. 57 Qwen2.5-Coder (7B) 66.9%
  58. 58 Qwen2.5-3B 66.4%
  59. 59 Gemma 3 4B 65.2%
  60. 60 Phi-4 Mini 64.9%
  61. 61 Phi-3.5-mini 62.2%
  62. 62 Qwen2.5-1.5B 61%
  63. 63 Command R7B 51.7%
  64. 64 Mixtral 8x7B 48.8%
  65. 65 Qwen2.5-0.5B 46.4%
  66. 66 Gemma 3 1B 45.7%
  67. 67 Mistral 7B v0.1 45.3%
  68. 68 Claude Sonnet 4.6 44.8%
  69. 69 GPT-5 mini 44.7%

Source: Official TeleQnA leaderboard

#EntryScore ±SettingDateSource
1 OTel-LLM-8.3B-QnA 91.2% ±0.3 Official leaderboard
2 TeleLLM 89.4% ±0.3 Official leaderboard
3 CHT-TeleX-8B 87.8% ±0.3 Official leaderboard
4 Gemini 3.1 Pro 85.2% ±1.1 Feb 2026 Official leaderboard
5 Claude Opus 4.5 84.5% ±1.1 Nov 2025 Official leaderboard
6 Claude Opus 4.6 84.4% ±1.1 Feb 2026 Official leaderboard
7 GPT-5 83.8% ±1.2 Aug 2025 Official leaderboard
8 Grok 4 Fast 83.6% ±1.2 Sep 2025 Official leaderboard
9 Kimi K2.5 83.6% ±1.2 Jan 2026 Official leaderboard
10 o3 83.4% ±1.2 Apr 2025 Official leaderboard
11 Gemini 3 Flash 83.2% ±1.2 Dec 2025 Official leaderboard
12 Claude Sonnet 4 83% ±1.2 May 2025 Official leaderboard
13 Gemini 3 Pro 82.9% ±1.2 Nov 2025 Official leaderboard
14 GPT-5.1 82.7% ±1.2 Nov 2025 Official leaderboard
15 GPT-5.2 82.7% ±1.2 Dec 2025 Official leaderboard
16 Claude Opus 4 82.4% ±1.2 May 2025 Official leaderboard
17 Claude Opus 4.1 82.3% ±1.2 Aug 2025 Official leaderboard
18 Claude Sonnet 4.5 82.3% ±1.2 Sep 2025 Official leaderboard
19 netLLM 82.1% ±0.4 Official leaderboard
20 Claude 3.5 Sonnet 82.1% ±1.2 Jun 2024 Official leaderboard
21 LTM 81.9% ±0.4 Official leaderboard
22 o1 81.9% ±1.2 Dec 2024 Official leaderboard
23 Gemini 2.5 Pro Jun 2025 81.8% ±1.2 Jun 2025 Official leaderboard
24 GPT-4.1 81.7% ±1.2 Apr 2025 Official leaderboard
25 Claude 3.7 Sonnet 81.6% ±1.2 Feb 2025 Official leaderboard
26 DeepSeek-V3 Mar 2025 81.6% ±1.2 Mar 2025 Official leaderboard
27 Grok 4.1 Fast 81.3% ±1.2 Nov 2025 Official leaderboard
28 o4-mini 80.8% ±1.2 Apr 2025 Official leaderboard
29 TSLAM-150B 80.6% ±1.3 Official leaderboard
30 Grok-3 mini 80.6% ±1.3 Apr 2025 Official leaderboard
31 Gemini 2.0 Flash Feb 2025 80.5% ±1.3 Feb 2025 Official leaderboard
32 Llama 4 Maverick 80.2% ±1.3 Apr 2025 Official leaderboard
33 gpt-oss-120b 79.9% ±1.3 Aug 2025 Official leaderboard
34 Qwen3-235B-A22B-Instruct Jul 2025 79.7% ±1.3 Jul 2025 Official leaderboard
35 GPT-4.1 mini 79.5% ±1.3 Apr 2025 Official leaderboard
36 MiniMax-M2.5 79.1% ±1.3 Feb 2026 Official leaderboard
37 Claude Haiku 4.5 78.9% ±1.3 Oct 2025 Official leaderboard
38 Gemini 2.5 Flash Sep 2025 78.8% ±1.3 Sep 2025 Official leaderboard
39 TSLAM-30B 78.7% ±1.3 Official leaderboard
40 MiMo-V2-Flash 78.6% ±1.3 Dec 2025 Official leaderboard
41 grok-code-fast-1 78.5% ±1.3 Aug 2025 Official leaderboard
42 Qwen3-32B 78.4% ±1.3 Apr 2025 Official leaderboard
43 Cohere Command A 77.5% ±1.3 Mar 2025 Official leaderboard
44 Llama 3.3 70B Instruct 77.3% ±1.3 Dec 2024 Official leaderboard
45 GPT-5 nano 76.8% ±1.3 Aug 2025 Official leaderboard
46 GPT-4 Jun 2023 76.7% ±1.3 Jun 2023 Official leaderboard
47 Qwen2.5-72B 76.5% ±0.4 Sep 2024 Official leaderboard
48 telco-slm-4B 75.9% ±0.4 Official leaderboard
49 GPT-4o mini 75.1% ±1.4 Jul 2024 Official leaderboard
50 Qwen2.5-32B 75% ±0.4 Sep 2024 Official leaderboard
51 Qwen3-8B 74.5% ±1.4 Apr 2025 Official leaderboard
52 Claude 3.5 Haiku 74% ±1.4 Oct 2024 Official leaderboard
53 mistral-small-24b 73.7% ±0.4 Official leaderboard
54 Phi-4 73.1% ±0.4 Dec 2024 Official leaderboard
55 GPT-4.1 nano 72.6% ±1.4 Apr 2025 Official leaderboard
56 DeepSeek-R1 May 2025 72.6% ±1.4 May 2025 Official leaderboard
57 Qwen2.5-14B 72.5% ±0.5 Sep 2024 Official leaderboard
58 Claude 3 Haiku 72.2% ±1.4 Mar 2024 Official leaderboard
59 Gemma 2 27B 71.8% ±0.5 Jun 2024 Official leaderboard
60 Gemma 3 27B 71.3% ±0.5 Mar 2025 Official leaderboard
61 GPT-3.5 Turbo Jan 2024 71.3% ±1.4 Jan 2024 Official leaderboard
62 DeepSeek-V3.2 71.3% ±1.4 Dec 2025 Official leaderboard
63 Qwen2.5-7B 70.2% ±0.5 Sep 2024 Official leaderboard
64 Gemma 2 9B 69.4% ±0.5 Jun 2024 Official leaderboard
65 Gemma 3 12B 68.8% ±0.5 Mar 2025 Official leaderboard
66 falcon3-10b 68.7% ±0.5 Official leaderboard
67 granite3.2-8b 67.5% ±0.5 Official leaderboard
68 internlm2.5-20b 67.2% ±0.5 Official leaderboard
69 mistral-small-22b 67% ±0.5 Official leaderboard
70 Qwen2.5-Coder (7B) 66.9% ±0.5 Sep 2024 Official leaderboard
71 internlm2.5-7b 66.9% ±0.5 Official leaderboard
72 Qwen2.5-3B 66.4% ±0.5 Sep 2024 Official leaderboard
73 granite3.3-8b 65.8% ±0.5 Official leaderboard
74 Gemma 3 4B 65.2% ±1.5 Mar 2025 Official leaderboard
75 Phi-4 Mini 64.9% ±0.5 Feb 2025 Official leaderboard
76 mistral-nemo-12b 64.2% ±0.5 Official leaderboard
77 Phi-3.5-mini 62.2% ±0.5 Aug 2024 Official leaderboard
78 Qwen2.5-1.5B 61% ±0.5 Sep 2024 Official leaderboard
79 falcon3-3b 60.8% ±0.5 Official leaderboard
80 gemma2-2b 60.5% ±0.5 Official leaderboard
81 Command R7B 51.7% ±1.6 Dec 2024 Official leaderboard
82 deepseek-r1-32b 51.1% ±1.5 Official leaderboard
83 Mixtral 8x7B 48.8% ±0.5 Dec 2023 Official leaderboard
84 Qwen2.5-0.5B 46.4% ±0.5 Sep 2024 Official leaderboard
85 Gemma 3 1B 45.7% ±0.5 Mar 2025 Official leaderboard
86 Mistral 7B v0.1 45.3% ±0.5 Sep 2023 Official leaderboard
87 Claude Sonnet 4.6 44.8% ±1.6 Feb 2026 Official leaderboard
88 GPT-5 mini 44.7% ±1.6 Aug 2025 Official leaderboard
89 falcon3-1b 37% ±0.5 Official leaderboard
90 falcon3-7b 16.1% ±0.4 Official leaderboard
#ModelScore ±SettingDateSource
1 Gemini 3.1 Pro 85.2% ±1.1 Feb 2026 Official leaderboard
2 Claude Opus 4.5 84.5% ±1.1 Nov 2025 Official leaderboard
3 Claude Opus 4.6 84.4% ±1.1 Feb 2026 Official leaderboard
4 GPT-5 83.8% ±1.2 Aug 2025 Official leaderboard
5 Grok 4 Fast 83.6% ±1.2 Sep 2025 Official leaderboard
6 Kimi K2.5 83.6% ±1.2 Jan 2026 Official leaderboard
7 o3 83.4% ±1.2 Apr 2025 Official leaderboard
8 Gemini 3 Flash 83.2% ±1.2 Dec 2025 Official leaderboard
9 Claude Sonnet 4 83% ±1.2 May 2025 Official leaderboard
10 Gemini 3 Pro 82.9% ±1.2 Nov 2025 Official leaderboard
11 GPT-5.1 82.7% ±1.2 Nov 2025 Official leaderboard
12 GPT-5.2 82.7% ±1.2 Dec 2025 Official leaderboard
13 Claude Opus 4 82.4% ±1.2 May 2025 Official leaderboard
14 Claude Opus 4.1 82.3% ±1.2 Aug 2025 Official leaderboard
15 Claude Sonnet 4.5 82.3% ±1.2 Sep 2025 Official leaderboard
16 Claude 3.5 Sonnet 82.1% ±1.2 Jun 2024 Official leaderboard
17 o1 81.9% ±1.2 Dec 2024 Official leaderboard
18 Gemini 2.5 Pro Jun 2025 81.8% ±1.2 Jun 2025 Official leaderboard
19 GPT-4.1 81.7% ±1.2 Apr 2025 Official leaderboard
20 Claude 3.7 Sonnet 81.6% ±1.2 Feb 2025 Official leaderboard
21 DeepSeek-V3 Mar 2025 81.6% ±1.2 Mar 2025 Official leaderboard
22 Grok 4.1 Fast 81.3% ±1.2 Nov 2025 Official leaderboard
23 o4-mini 80.8% ±1.2 Apr 2025 Official leaderboard
24 Grok-3 mini 80.6% ±1.3 Apr 2025 Official leaderboard
25 Gemini 2.0 Flash Feb 2025 80.5% ±1.3 Feb 2025 Official leaderboard
26 Llama 4 Maverick 80.2% ±1.3 Apr 2025 Official leaderboard
27 gpt-oss-120b 79.9% ±1.3 Aug 2025 Official leaderboard
28 Qwen3-235B-A22B-Instruct Jul 2025 79.7% ±1.3 Jul 2025 Official leaderboard
29 GPT-4.1 mini 79.5% ±1.3 Apr 2025 Official leaderboard
30 MiniMax-M2.5 79.1% ±1.3 Feb 2026 Official leaderboard
31 Claude Haiku 4.5 78.9% ±1.3 Oct 2025 Official leaderboard
32 Gemini 2.5 Flash Sep 2025 78.8% ±1.3 Sep 2025 Official leaderboard
33 MiMo-V2-Flash 78.6% ±1.3 Dec 2025 Official leaderboard
34 grok-code-fast-1 78.5% ±1.3 Aug 2025 Official leaderboard
35 Qwen3-32B 78.4% ±1.3 Apr 2025 Official leaderboard
36 Cohere Command A 77.5% ±1.3 Mar 2025 Official leaderboard
37 Llama 3.3 70B Instruct 77.3% ±1.3 Dec 2024 Official leaderboard
38 GPT-5 nano 76.8% ±1.3 Aug 2025 Official leaderboard
39 GPT-4 Jun 2023 76.7% ±1.3 Jun 2023 Official leaderboard
40 Qwen2.5-72B 76.5% ±0.4 Sep 2024 Official leaderboard
41 GPT-4o mini 75.1% ±1.4 Jul 2024 Official leaderboard
42 Qwen2.5-32B 75% ±0.4 Sep 2024 Official leaderboard
43 Qwen3-8B 74.5% ±1.4 Apr 2025 Official leaderboard
44 Claude 3.5 Haiku 74% ±1.4 Oct 2024 Official leaderboard
45 Phi-4 73.1% ±0.4 Dec 2024 Official leaderboard
46 GPT-4.1 nano 72.6% ±1.4 Apr 2025 Official leaderboard
47 DeepSeek-R1 May 2025 72.6% ±1.4 May 2025 Official leaderboard
48 Qwen2.5-14B 72.5% ±0.5 Sep 2024 Official leaderboard
49 Claude 3 Haiku 72.2% ±1.4 Mar 2024 Official leaderboard
50 Gemma 2 27B 71.8% ±0.5 Jun 2024 Official leaderboard
51 Gemma 3 27B 71.3% ±0.5 Mar 2025 Official leaderboard
52 GPT-3.5 Turbo Jan 2024 71.3% ±1.4 Jan 2024 Official leaderboard
53 DeepSeek-V3.2 71.3% ±1.4 Dec 2025 Official leaderboard
54 Qwen2.5-7B 70.2% ±0.5 Sep 2024 Official leaderboard
55 Gemma 2 9B 69.4% ±0.5 Jun 2024 Official leaderboard
56 Gemma 3 12B 68.8% ±0.5 Mar 2025 Official leaderboard
57 Qwen2.5-Coder (7B) 66.9% ±0.5 Sep 2024 Official leaderboard
58 Qwen2.5-3B 66.4% ±0.5 Sep 2024 Official leaderboard
59 Gemma 3 4B 65.2% ±1.5 Mar 2025 Official leaderboard
60 Phi-4 Mini 64.9% ±0.5 Feb 2025 Official leaderboard
61 Phi-3.5-mini 62.2% ±0.5 Aug 2024 Official leaderboard
62 Qwen2.5-1.5B 61% ±0.5 Sep 2024 Official leaderboard
63 Command R7B 51.7% ±1.6 Dec 2024 Official leaderboard
64 Mixtral 8x7B 48.8% ±0.5 Dec 2023 Official leaderboard
65 Qwen2.5-0.5B 46.4% ±0.5 Sep 2024 Official leaderboard
66 Gemma 3 1B 45.7% ±0.5 Mar 2025 Official leaderboard
67 Mistral 7B v0.1 45.3% ±0.5 Sep 2023 Official leaderboard
68 Claude Sonnet 4.6 44.8% ±1.6 Feb 2026 Official leaderboard
69 GPT-5 mini 44.7% ±1.6 Aug 2025 Official leaderboard

6 models have held number one on the official leaderboard since its release in October 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Anthropic Google DeepMind
Own leaderboard Passed 85% in Feb 2026, by Gemini 3.1 Pro

Source: Official TeleQnA leaderboard

Gemini 3.1 Pro on the official leaderboard passed 85% in February 2026

That was 28 months after its release. By the same age, 48% of the benchmarks from 2023 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) Gemini 3.1 Pro GPT-5 Claude 3.5 Sonnet GPT-4 0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) Gemini 3.1 Pro Claude 3.5 Sonnet

Source: Official TeleQnA leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Knowledge

All 332 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • MMLU In use 57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy. Seed1.8 92.3%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • HLE-Diamond In use Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers. Claude Fable 5.2 77.9%

Data audit

All 8 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
  • Pass Independent runs Scores from the official leaderboard (90 entries).
  • Pass Models scored 69 models we track, plus 21 other entries on its leaderboard.
  • Note Newest score The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro. That’s expected for a saturated test.
  • Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39).
  • Note Headroom Saturated. The best is 85.2%, past 85% since February 2026.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 70 rows for models we track and 21 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check