Benchmark Almanac

Open Hebrew LLM Leaderboard

mistralai/Mistral-Large-Instruct-2407 75.7 tops its leaderboard Gemma 4 31B IT 74.3 leads the models we track
Not in model cards

Models answer questions, resolve pronouns, classify sentiment and inference, translate passages, and answer trivia in Hebrew.

The default public table contains 201 distinct model identifiers with numeric averages across six columns. The launch announcement described four tasks.

Released
March 2024
Kind
Knowledge in many languages
Level
School
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Measures
Capability
Language
Hebrew, English
In and out
Text → Text
Interaction
Single turn
Items from
Another benchmark
Refresh
One fixed set
Run by
Anyone
Strongest model tested
Gemma 4 31B IT #101 on the capability index
Last model added
April 2026
Scores refreshed
November 20th, 2026
201
Entries on its own leaderboard, 88 models we track

mistralai/Mistral-Large-Instruct-2407 tops the leaderboard with 75.7 Gemma 4 31B IT leads with 74.3 among models released by April 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Gemma 4 31B IT, at 74.3. This view shows only models we track, each at its best.

The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

201 entries. Scroll sideways for the rest.

  1. 75.7 mistralai/Mistral-Large-Instruct-2407
  2. 74.3 Gemma 4 31B IT
  3. 73.4 Jamba 1.6 Large
  4. 72.7 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16
  5. 72.6 Llama 3.1-70B
  6. 72.5 dicta-il/DictaLM-3.0-24B-Base
  7. 72.4 CohereForAI/c4ai-command-r-plus-08-2024
  8. 71.8 CohereForAI/c4ai-command-r-plus
  9. 71.4 Llama 3-70B
  10. 70.5 Llama 3.3 70B Instruct
  11. 70.2 Aya Expanse 32B
  12. 70.2 Qwen2.5-72B
  13. 69.9 Llama-3.1-Nemotron-70B-Instruct
  14. 69.5 meta-llama/Llama-3.1-70B-Instruct
  15. 69.5 Gemma 3 27B PT
  16. 69.2 Qwen3.5-35B-A3B-Base
  17. 68.7 meta-llama/Llama-4-Scout-17B-16E
  18. 67.7 c4ai-command-r-08-2024
  19. 67.6 mistralai/Mistral-Small-24B-Instruct-2501
  20. 66.5 dicta-il/DictaLM-3.0-Nemotron-12B-Base
  21. 66.4 ibm-granite/granite-4.0-h-small-base
  22. 66.0 QwQ-32B-Preview
  23. 66.0 mistralai/Mistral-Small-3.1-24B-Base-2503
  24. 65.9 Llama-3.3-Nemotron-Super-49B-v1.5
  25. 65.7 nvidia/Nemotron-H-56B-Base-8K
  26. 65.5 zai-org/GLM-4.5-Air-Base
  27. 65.4 Gemma 4 26B A4B
  28. 65.4 CohereForAI/aya-23-35B
  29. 65.1 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16
  30. 64.8 618AI/dictalm2-it-qa-fine-tune
  31. 64.3 mistralai/Mistral-Small-24B-Base-2501
  32. 64.1 rombodawg/Rombos-LLM-V2.5-Qwen-32b
  33. 64.0 Gemma 3 12B PT
  34. 63.8 google/gemma-2-9b-it
  35. 63.6 Qwen3-32B
  36. 63.2 UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
  37. 63.1 Qwen2.5-14B
  38. 63.0 Qwen3-14B-Base
  39. 63.0 Qwen3-30B-A3B-Instruct
  40. 62.9 Jamba 1.6 Mini
  41. 62.8 Qwen3-30B-A3B-Thinking
  42. 62.7 ai21labs/AI21-Jamba-1.5-Large
  43. 62.6 dicta-il/dictalm2.0
  44. 62.3 DeepSeek-R1-Distill-Qwen-32B
  45. 61.9 Qwen3-30B-A3B-Base
  46. 61.5 Phi-4
  47. 61.5 dicta-il/dictalm2.0-instruct
  48. 61.4 Qwen3-14B
  49. 61.1 Qwen2.5-32B
  50. 61.0 Command R (03-2024)
  51. 60.7 Gemma 4 E4B
  52. 60.2 CohereLabs/c4ai-command-r7b-arabic-02-2025
  53. 60.0 SicariusSicariiStuff/Impish_Nemo_12B
  54. 59.9 SicariusSicariiStuff/Hebrew_Nemo
  55. 59.4 ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew
  56. 59.4 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned
  57. 59.4 Qwen3-8B-Base
  58. 59.2 mistralai/Mistral-Nemo-Instruct-2407
  59. 59.0 SeaLLMs/SeaLLM-7B-v2.5
  60. 58.8 Gemma 2 9B
  61. 58.5 Aya Expanse 8B
  62. 57.6 Qwen3-8B
  63. 57.6 SicariusSicariiStuff/Zion_Alpha
  64. 57.6 DeepSeek-R1-Distill-Qwen-14B
  65. 57.4 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP
  66. 57.0 ai21labs/AI21-Jamba-1.5-Mini
  67. 56.5 Gemma 2 27B
  68. 56.3 Qwen2.5-7B
  69. 55.6 Mistral-Nemo-Base-2407
  70. 55.6 ronigold/dictalm2.0-instruct-fine-tuned
  71. 55.2 mlabonne/NeuralDaredevil-8B-abliterated
  72. 54.9 mistralai/Codestral-22B-v0.1
  73. 54.8 meta-llama/Meta-Llama-3.1-8B-Instruct
  74. 54.5 NousResearch/Hermes-3-Llama-3.1-8B
  75. 54.1 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B
  76. 54.1 CohereForAI/aya-23-8B
  77. 53.8 mistralai/Mistral-Small-Instruct-2409
  78. 53.8 Qwen1.5-32B
  79. 53.7 nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base
  80. 53.3 mistralai/Ministral-8B-Instruct-2410
  81. 53.1 Qwen3-4B-Base
  82. 52.9 MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct
  83. 52.3 Llama 3.1-8B
  84. 52.2 Danielbrdz/Barcenas-Llama3-8b-ORPO
  85. 52.1 allenai/Olmo-3-1125-32B
  86. 51.8 yam-peleg/Hebrew-Mistral-7B
  87. 51.7 Gemma 7B
  88. 51.5 dicta-il/DictaLM-3.0-1.7B-Base
  89. 51.3 Qwen3-4B
  90. 50.6 yam-peleg/Hebrew-Gemma-11B-Instruct
  91. 50.6 DeepMount00/Llama-3-8b-Ita
  92. 50.2 lightblue/suzume-llama-3-8B-multilingual
  93. 50.2 ibm-granite/granite-4.0-h-tiny-base
  94. 50.0 Gemma 4 E2B
  95. 49.9 uygarkurt/llama-3-merged-linear
  96. 49.9 abacusai/Llama-3-Smaug-8B
  97. 49.8 Qwen2-7B
  98. 49.6 VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
  99. 49.3 Gemma 2 2B IT
  100. 49.2 Gemma 1.1 7B IT
  101. 49.1 nvidia/Mistral-NeMo-Minitron-8B-Base
  102. 48.9 NousResearch/Hermes-2-Pro-Llama-3-8B
  103. 48.5 natong19/Qwen2-7B-Instruct-abliterated
  104. 48.5 tiiuae/Falcon-H1-34B-Base
  105. 48.4 meta-llama/Meta-Llama-3-8B-Instruct
  106. 48.2 Qwen2.5-3B
  107. 48.2 yam-peleg/Hebrew-Gemma-11B-V2
  108. 48.1 ibm-granite/granite-4.0-h-micro-base
  109. 48.1 CohereLabs/tiny-aya-base
  110. 48.1 ibm-granite/granite-4.0-micro-base
  111. 48.1 Gemma 3 4B PT
  112. 48.0 Mixtral 8x7B
  113. 47.3 Llama 3-8B
  114. 46.9 SOLAR-10.7B-Instruct-v1.0
  115. 46.6 kekmodel/StopCarbon-10.7B-v5
  116. 46.4 jeonsworld/CarbonVillain-en-10.7B-v4
  117. 46.2 invalid-coder/Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp
  118. 45.3 Qwen1.5-14B
  119. 45.3 MAmmoTH2-8B-Plus
  120. 45.2 unsloth/Phi-3.5-mini-instruct
  121. 44.8 01-ai/Yi-1.5-34B-32K
  122. 44.6 shadowml/BeagSake-7B
  123. 43.8 google/gemma-7b-it
  124. 43.8 Qwen1.5-14B-Chat
  125. 43.6 zhengr/MixTAO-7Bx2-MoE-v8.1
  126. 43.4 eldogbbhed/Peagle-9b
  127. 43.3 vicgalle/CarbonBeagle-11B-truthy
  128. 43.3 Qwen/Qwen3-1.7B-Base
  129. 43.2 mlabonne/NeuralMonarch-7B
  130. 43.2 mlabonne/AlphaMonarch-7B
  131. 43.0 Yi-1.5-9B-Chat
  132. 42.7 maywell/EXAONE-3.0-7.8B-Instruct-Llamafied
  133. 42.7 Bingsu/exaone-3.0-7.8b-it
  134. 42.7 Kukedlc/NeuralSynthesis-7B-v0.1
  135. 42.5 google/gemma-2-2b
  136. 42.0 deepseek-ai/DeepSeek-R1-Distill-Llama-8B
  137. 42.0 HuggingFaceTB/SmolLM3-3B-Base
  138. 41.7 Granite-3.1-8B-Instruct
  139. 41.5 yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B
  140. 40.9 nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base
  141. 40.8 Gemma 2B
  142. 40.5 HuggingFaceTB/SmolLM3-3B
  143. 40.3 yunconglong/DARE_TIES_13B
  144. 39.3 Qwen2.5-1.5B
  145. 39.1 Granite-3.0-8B-Instruct
  146. 39.0 Mistral 7B v0.3
  147. 38.6 Falcon3-10B-Instruct
  148. 38.6 Granite-3.0-8B-Base
  149. 38.5 Mistral 7B v0.1
  150. 38.3 MAmmoTH2-7B-Plus
  151. 37.9 tiiuae/Falcon-H1-7B-Base
  152. 37.4 lex-hue/Delexa-7b
  153. 36.6 Zephyr 7B Beta
  154. 36.1 Qwen2-1.5B
  155. 35.8 meta-llama/Llama-2-7b-hf
  156. 35.8 Falcon3-7B-Instruct
  157. 35.6 royallab/ZephRP-m7b
  158. 35.6 Granite-3.1-8B-Base
  159. 35.4 tiiuae/Falcon3-10B-Base
  160. 34.8 Llama 3.2 1B
  161. 34.8 lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half
  162. 34.5 Qwen/Qwen3-0.6B-Base
  163. 34.3 DeepSeek-R1-Distill-Qwen-7B
  164. 34.3 XythicK/Hebrew-GPT
  165. 34.3 haoranxu/ALMA-13B-R
  166. 33.6 phi-3-mini 3.8B
  167. 33.3 shanchen/llama3-8B-slerp-med-chinese
  168. 33.3 tiiuae/Falcon3-7B-Base
  169. 32.9 Granite-3.0-2B-Instruct
  170. 32.6 Granite-3.0-2B-Base
  171. 31.8 Granite-3.1-2B-Base
  172. 31.5 Granite-3.1-3B-A800M-Instruct
  173. 30.9 ibm-granite/granite-3.0-3b-a800m-instruct
  174. 30.7 Qwen2.5-0.5B
  175. 30.6 ibm-granite/granite-3.0-3b-a800m-base
  176. 30.4 CohereForAI/c4ai-command-a-03-2025
  177. 30.2 Llama 3.2 3B
  178. 30.1 QwQ-32B
  179. 29.8 Granite-3.1-3B-A800M-Base
  180. 29.8 Falcon3-3B-Instruct
  181. 29.1 SmolLM2-1.7B
  182. 29.1 tiiuae/Falcon3-3B-Base
  183. 28.8 ibm-granite/granite-4.0-h-350m-base
  184. 27.7 Granite-3.1-1B-A400M-Instruct
  185. 27.7 Qwen2-0.5B
  186. 27.6 tiiuae/Falcon-H1-1.5B-Base
  187. 27.3 ibm-granite/granite-4.0-350m-base
  188. 26.5 AIDC-AI/Marco-LLM-GLO
  189. 26.0 tiiuae/Falcon-H1-1.5B-Deep-Base
  190. 25.9 Falcon3-1B-Instruct
  191. 25.9 ibm-granite/granite-3.0-1b-a400m-base
  192. 25.9 Granite-3.1-1B-A400M-Base
  193. 24.7 SmolLM-1.7B
  194. 23.7 allenai/OLMo-1B-0724-hf
  195. 23.3 Gemma 3 270M
  196. 22.9 tiiuae/Falcon-H1-0.5B-Base
  197. 22.7 upstage/SOLAR-10.7B-v1.0
  198. 13.4 Gemma 3 27B
  199. 13.0 nvidia/Llama-3.1-Nemotron-Nano-8B-v1
  200. 2.22 LiquidAI/LFM2-2.6B
  201. 1.35 LFM2-1.2B
  1. 1 mistralai/Mistral-Large-Instruct-2407 bfloat16 75.7
  2. 2 Gemma 4 31B IT bfloat16 74.3
  3. 3 Jamba 1.6 Large bfloat16 73.4
  4. 4 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16 bfloat16 72.7
  5. 5 Llama 3.1-70B bfloat16 72.6
  6. 6 dicta-il/DictaLM-3.0-24B-Base bfloat16 72.5
  7. 7 CohereForAI/c4ai-command-r-plus-08-2024 bfloat16 72.4
  8. 8 CohereForAI/c4ai-command-r-plus bfloat16 71.8
  9. 9 Llama 3-70B bfloat16 71.4
  10. 10 Llama 3.3 70B Instruct bfloat16 70.5
  11. 11 Aya Expanse 32B bfloat16 70.2
  12. 12 Qwen2.5-72B bfloat16 70.2
  13. 13 Llama-3.1-Nemotron-70B-Instruct bfloat16 69.9
  14. 14 meta-llama/Llama-3.1-70B-Instruct bfloat16 69.5
  15. 15 Gemma 3 27B PT bfloat16 69.5
  16. 16 Qwen3.5-35B-A3B-Base bfloat16 69.2
  17. 17 meta-llama/Llama-4-Scout-17B-16E bfloat16 68.7
  18. 18 c4ai-command-r-08-2024 bfloat16 67.7
  19. 19 mistralai/Mistral-Small-24B-Instruct-2501 bfloat16 67.6
  20. 20 dicta-il/DictaLM-3.0-Nemotron-12B-Base bfloat16 66.5
  21. 21 ibm-granite/granite-4.0-h-small-base bfloat16 66.4
  22. 22 QwQ-32B-Preview bfloat16 66.0
  23. 23 mistralai/Mistral-Small-3.1-24B-Base-2503 bfloat16 66.0
  24. 24 Llama-3.3-Nemotron-Super-49B-v1.5 bfloat16 65.9
  25. 25 nvidia/Nemotron-H-56B-Base-8K bfloat16 65.7
  26. 26 zai-org/GLM-4.5-Air-Base bfloat16 65.5
  27. 27 Gemma 4 26B A4B bfloat16 65.4
  28. 28 CohereForAI/aya-23-35B bfloat16 65.4
  29. 29 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 bfloat16 65.1
  30. 30 618AI/dictalm2-it-qa-fine-tune bfloat16 64.8
  31. 31 mistralai/Mistral-Small-24B-Base-2501 bfloat16 64.3
  32. 32 rombodawg/Rombos-LLM-V2.5-Qwen-32b bfloat16 64.1
  33. 33 Gemma 3 12B PT bfloat16 64.0
  34. 34 google/gemma-2-9b-it bfloat16 63.8
  35. 35 Qwen3-32B float16 63.6
  36. 36 UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 bfloat16 63.2
  37. 37 Qwen2.5-14B bfloat16 63.1
  38. 38 Qwen3-14B-Base bfloat16 63.0
  39. 39 Qwen3-30B-A3B-Instruct Jul 2025, float16 63.0
  40. 40 Jamba 1.6 Mini bfloat16 62.9
  41. 41 Qwen3-30B-A3B-Thinking Jul 2025, float16 62.8
  42. 42 ai21labs/AI21-Jamba-1.5-Large 8bit 62.7
  43. 43 dicta-il/dictalm2.0 bfloat16 62.6
  44. 44 DeepSeek-R1-Distill-Qwen-32B bfloat16 62.3
  45. 45 Qwen3-30B-A3B-Base bfloat16 61.9
  46. 46 Phi-4 bfloat16 61.5
  47. 47 dicta-il/dictalm2.0-instruct bfloat16 61.5
  48. 48 Qwen3-14B float16 61.4
  49. 49 Qwen2.5-32B bfloat16 61.1
  50. 50 Command R (03-2024) bfloat16 61.0
  51. 51 Gemma 4 E4B bfloat16 60.7
  52. 52 CohereLabs/c4ai-command-r7b-arabic-02-2025 bfloat16 60.2
  53. 53 SicariusSicariiStuff/Impish_Nemo_12B bfloat16 60.0
  54. 54 SicariusSicariiStuff/Hebrew_Nemo bfloat16 59.9
  55. 55 ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew float16 59.4
  56. 56 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned bfloat16 59.4
  57. 57 Qwen3-8B-Base bfloat16 59.4
  58. 58 mistralai/Mistral-Nemo-Instruct-2407 bfloat16 59.2
  59. 59 SeaLLMs/SeaLLM-7B-v2.5 bfloat16 59.0
  60. 60 Gemma 2 9B bfloat16 58.8
  61. 61 Aya Expanse 8B bfloat16 58.5
  62. 62 Qwen3-8B float16 57.6
  63. 63 SicariusSicariiStuff/Zion_Alpha bfloat16 57.6
  64. 64 DeepSeek-R1-Distill-Qwen-14B float16 57.6
  65. 65 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP bfloat16 57.4
  66. 66 ai21labs/AI21-Jamba-1.5-Mini bfloat16 57.0
  67. 67 Gemma 2 27B bfloat16 56.5
  68. 68 Qwen2.5-7B bfloat16 56.3
  69. 69 Mistral-Nemo-Base-2407 bfloat16 55.6
  70. 70 ronigold/dictalm2.0-instruct-fine-tuned float16 55.6
  71. 71 mlabonne/NeuralDaredevil-8B-abliterated float16 55.2
  72. 72 mistralai/Codestral-22B-v0.1 bfloat16 54.9
  73. 73 meta-llama/Meta-Llama-3.1-8B-Instruct bfloat16 54.8
  74. 74 NousResearch/Hermes-3-Llama-3.1-8B bfloat16 54.5
  75. 75 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B float16 54.1
  76. 76 CohereForAI/aya-23-8B bfloat16 54.1
  77. 77 mistralai/Mistral-Small-Instruct-2409 bfloat16 53.8
  78. 78 Qwen1.5-32B bfloat16 53.8
  79. 79 nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base bfloat16 53.7
  80. 80 mistralai/Ministral-8B-Instruct-2410 bfloat16 53.3
  81. 81 Qwen3-4B-Base bfloat16 53.1
  82. 82 MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct bfloat16 52.9
  83. 83 Llama 3.1-8B bfloat16 52.3
  84. 84 Danielbrdz/Barcenas-Llama3-8b-ORPO float16 52.2
  85. 85 allenai/Olmo-3-1125-32B bfloat16 52.1
  86. 86 yam-peleg/Hebrew-Mistral-7B bfloat16 51.8
  87. 87 Gemma 7B bfloat16 51.7
  88. 88 dicta-il/DictaLM-3.0-1.7B-Base bfloat16 51.5
  89. 89 Qwen3-4B float16 51.3
  90. 90 yam-peleg/Hebrew-Gemma-11B-Instruct float16 50.6
  91. 91 DeepMount00/Llama-3-8b-Ita bfloat16 50.6
  92. 92 lightblue/suzume-llama-3-8B-multilingual bfloat16 50.2
  93. 93 ibm-granite/granite-4.0-h-tiny-base bfloat16 50.2
  94. 94 Gemma 4 E2B bfloat16 50.0
  95. 95 uygarkurt/llama-3-merged-linear float16 49.9
  96. 96 abacusai/Llama-3-Smaug-8B bfloat16 49.9
  97. 97 Qwen2-7B bfloat16 49.8
  98. 98 VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct bfloat16 49.6
  99. 99 Gemma 2 2B IT bfloat16 49.3
  100. 100 Gemma 1.1 7B IT float16 49.2
  101. 101 nvidia/Mistral-NeMo-Minitron-8B-Base bfloat16 49.1
  102. 102 NousResearch/Hermes-2-Pro-Llama-3-8B bfloat16 48.9
  103. 103 natong19/Qwen2-7B-Instruct-abliterated bfloat16 48.5
  104. 104 tiiuae/Falcon-H1-34B-Base bfloat16 48.5
  105. 105 meta-llama/Meta-Llama-3-8B-Instruct bfloat16 48.4
  106. 106 Qwen2.5-3B bfloat16 48.2
  107. 107 yam-peleg/Hebrew-Gemma-11B-V2 bfloat16 48.2
  108. 108 ibm-granite/granite-4.0-h-micro-base bfloat16 48.1
  109. 109 CohereLabs/tiny-aya-base bfloat16 48.1
  110. 110 ibm-granite/granite-4.0-micro-base bfloat16 48.1
  111. 111 Gemma 3 4B PT bfloat16 48.1
  112. 112 Mixtral 8x7B bfloat16 48.0
  113. 113 Llama 3-8B bfloat16 47.3
  114. 114 SOLAR-10.7B-Instruct-v1.0 float16 46.9
  115. 115 kekmodel/StopCarbon-10.7B-v5 float16 46.6
  116. 116 jeonsworld/CarbonVillain-en-10.7B-v4 float16 46.4
  117. 117 invalid-coder/Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp float16 46.2
  118. 118 Qwen1.5-14B bfloat16 45.3
  119. 119 MAmmoTH2-8B-Plus bfloat16 45.3
  120. 120 unsloth/Phi-3.5-mini-instruct bfloat16 45.2
  121. 121 01-ai/Yi-1.5-34B-32K bfloat16 44.8
  122. 122 shadowml/BeagSake-7B bfloat16 44.6
  123. 123 google/gemma-7b-it bfloat16 43.8
  124. 124 Qwen1.5-14B-Chat bfloat16 43.8
  125. 125 zhengr/MixTAO-7Bx2-MoE-v8.1 bfloat16 43.6
  126. 126 eldogbbhed/Peagle-9b float16 43.4
  127. 127 vicgalle/CarbonBeagle-11B-truthy float16 43.3
  128. 128 Qwen/Qwen3-1.7B-Base bfloat16 43.3
  129. 129 mlabonne/NeuralMonarch-7B float16 43.2
  130. 130 mlabonne/AlphaMonarch-7B float16 43.2
  131. 131 Yi-1.5-9B-Chat bfloat16 43.0
  132. 132 maywell/EXAONE-3.0-7.8B-Instruct-Llamafied ? 42.7
  133. 133 Bingsu/exaone-3.0-7.8b-it ? 42.7
  134. 134 Kukedlc/NeuralSynthesis-7B-v0.1 bfloat16 42.7
  135. 135 google/gemma-2-2b bfloat16 42.5
  136. 136 deepseek-ai/DeepSeek-R1-Distill-Llama-8B float16 42.0
  137. 137 HuggingFaceTB/SmolLM3-3B-Base bfloat16 42.0
  138. 138 Granite-3.1-8B-Instruct bfloat16 41.7
  139. 139 yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B bfloat16 41.5
  140. 140 nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base bfloat16 40.9
  141. 141 Gemma 2B bfloat16 40.8
  142. 142 HuggingFaceTB/SmolLM3-3B float16 40.5
  143. 143 yunconglong/DARE_TIES_13B bfloat16 40.3
  144. 144 Qwen2.5-1.5B bfloat16 39.3
  145. 145 Granite-3.0-8B-Instruct bfloat16 39.1
  146. 146 Mistral 7B v0.3 bfloat16 39.0
  147. 147 Falcon3-10B-Instruct bfloat16 38.6
  148. 148 Granite-3.0-8B-Base bfloat16 38.6
  149. 149 Mistral 7B v0.1 bfloat16 38.5
  150. 150 MAmmoTH2-7B-Plus bfloat16 38.3
  151. 151 tiiuae/Falcon-H1-7B-Base bfloat16 37.9
  152. 152 lex-hue/Delexa-7b float16 37.4
  153. 153 Zephyr 7B Beta bfloat16 36.6
  154. 154 Qwen2-1.5B bfloat16 36.1
  155. 155 meta-llama/Llama-2-7b-hf bfloat16 35.8
  156. 156 Falcon3-7B-Instruct bfloat16 35.8
  157. 157 royallab/ZephRP-m7b bfloat16 35.6
  158. 158 Granite-3.1-8B-Base bfloat16 35.6
  159. 159 tiiuae/Falcon3-10B-Base bfloat16 35.4
  160. 160 Llama 3.2 1B bfloat16 34.8
  161. 161 lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half bfloat16 34.8
  162. 162 Qwen/Qwen3-0.6B-Base bfloat16 34.5
  163. 163 DeepSeek-R1-Distill-Qwen-7B float16 34.3
  164. 164 XythicK/Hebrew-GPT bfloat16 34.3
  165. 165 haoranxu/ALMA-13B-R bfloat16 34.3
  166. 166 phi-3-mini 3.8B bfloat16 33.6
  167. 167 shanchen/llama3-8B-slerp-med-chinese bfloat16 33.3
  168. 168 tiiuae/Falcon3-7B-Base bfloat16 33.3
  169. 169 Granite-3.0-2B-Instruct bfloat16 32.9
  170. 170 Granite-3.0-2B-Base bfloat16 32.6
  171. 171 Granite-3.1-2B-Base bfloat16 31.8
  172. 172 Granite-3.1-3B-A800M-Instruct bfloat16 31.5
  173. 173 ibm-granite/granite-3.0-3b-a800m-instruct bfloat16 30.9
  174. 174 Qwen2.5-0.5B bfloat16 30.7
  175. 175 ibm-granite/granite-3.0-3b-a800m-base bfloat16 30.6
  176. 176 CohereForAI/c4ai-command-a-03-2025 bfloat16 30.4
  177. 177 Llama 3.2 3B bfloat16 30.2
  178. 178 QwQ-32B bfloat16 30.1
  179. 179 Granite-3.1-3B-A800M-Base bfloat16 29.8
  180. 180 Falcon3-3B-Instruct bfloat16 29.8
  181. 181 SmolLM2-1.7B bfloat16 29.1
  182. 182 tiiuae/Falcon3-3B-Base bfloat16 29.1
  183. 183 ibm-granite/granite-4.0-h-350m-base bfloat16 28.8
  184. 184 Granite-3.1-1B-A400M-Instruct bfloat16 27.7
  185. 185 Qwen2-0.5B bfloat16 27.7
  186. 186 tiiuae/Falcon-H1-1.5B-Base bfloat16 27.6
  187. 187 ibm-granite/granite-4.0-350m-base bfloat16 27.3
  188. 188 AIDC-AI/Marco-LLM-GLO bfloat16 26.5
  189. 189 tiiuae/Falcon-H1-1.5B-Deep-Base bfloat16 26.0
  190. 190 Falcon3-1B-Instruct bfloat16 25.9
  191. 191 ibm-granite/granite-3.0-1b-a400m-base bfloat16 25.9
  192. 192 Granite-3.1-1B-A400M-Base bfloat16 25.9
  193. 193 SmolLM-1.7B bfloat16 24.7
  194. 194 allenai/OLMo-1B-0724-hf bfloat16 23.7
  195. 195 Gemma 3 270M bfloat16 23.3
  196. 196 tiiuae/Falcon-H1-0.5B-Base bfloat16 22.9
  197. 197 upstage/SOLAR-10.7B-v1.0 float16 22.7
  198. 198 Gemma 3 27B bfloat16 13.4
  199. 199 nvidia/Llama-3.1-Nemotron-Nano-8B-v1 bfloat16 13.0
  200. 200 LiquidAI/LFM2-2.6B bfloat16 2.22
  201. 201 LFM2-1.2B bfloat16 1.35

Source: Official Open Hebrew LLM Leaderboard leaderboard

88 models. Scroll sideways for the rest.

  1. 74.3 Gemma 4 31B IT
  2. 73.4 Jamba 1.6 Large
  3. 72.6 Llama 3.1-70B
  4. 71.4 Llama 3-70B
  5. 70.5 Llama 3.3 70B Instruct
  6. 70.2 Aya Expanse 32B
  7. 70.2 Qwen2.5-72B
  8. 69.9 Llama-3.1-Nemotron-70B-Instruct
  9. 69.5 Gemma 3 27B PT
  10. 69.2 Qwen3.5-35B-A3B-Base
  11. 67.7 c4ai-command-r-08-2024
  12. 66.0 QwQ-32B-Preview
  13. 65.9 Llama-3.3-Nemotron-Super-49B-v1.5
  14. 65.4 Gemma 4 26B A4B
  15. 64.0 Gemma 3 12B PT
  16. 63.6 Qwen3-32B
  17. 63.1 Qwen2.5-14B
  18. 63.0 Qwen3-14B-Base
  19. 63.0 Qwen3-30B-A3B-Instruct
  20. 62.9 Jamba 1.6 Mini
  21. 62.8 Qwen3-30B-A3B-Thinking
  22. 62.3 DeepSeek-R1-Distill-Qwen-32B
  23. 61.9 Qwen3-30B-A3B-Base
  24. 61.5 Phi-4
  25. 61.4 Qwen3-14B
  26. 61.1 Qwen2.5-32B
  27. 61.0 Command R (03-2024)
  28. 60.7 Gemma 4 E4B
  29. 59.4 Qwen3-8B-Base
  30. 58.8 Gemma 2 9B
  31. 58.5 Aya Expanse 8B
  32. 57.6 Qwen3-8B
  33. 57.6 DeepSeek-R1-Distill-Qwen-14B
  34. 56.5 Gemma 2 27B
  35. 56.3 Qwen2.5-7B
  36. 55.6 Mistral-Nemo-Base-2407
  37. 53.8 Qwen1.5-32B
  38. 53.1 Qwen3-4B-Base
  39. 52.3 Llama 3.1-8B
  40. 51.7 Gemma 7B
  41. 51.3 Qwen3-4B
  42. 50.0 Gemma 4 E2B
  43. 49.8 Qwen2-7B
  44. 49.3 Gemma 2 2B IT
  45. 49.2 Gemma 1.1 7B IT
  46. 48.2 Qwen2.5-3B
  47. 48.1 Gemma 3 4B PT
  48. 48.0 Mixtral 8x7B
  49. 47.3 Llama 3-8B
  50. 46.9 SOLAR-10.7B-Instruct-v1.0
  51. 45.3 Qwen1.5-14B
  52. 45.3 MAmmoTH2-8B-Plus
  53. 43.8 Qwen1.5-14B-Chat
  54. 43.0 Yi-1.5-9B-Chat
  55. 41.7 Granite-3.1-8B-Instruct
  56. 40.8 Gemma 2B
  57. 39.3 Qwen2.5-1.5B
  58. 39.1 Granite-3.0-8B-Instruct
  59. 39.0 Mistral 7B v0.3
  60. 38.6 Falcon3-10B-Instruct
  61. 38.6 Granite-3.0-8B-Base
  62. 38.5 Mistral 7B v0.1
  63. 38.3 MAmmoTH2-7B-Plus
  64. 36.6 Zephyr 7B Beta
  65. 36.1 Qwen2-1.5B
  66. 35.8 Falcon3-7B-Instruct
  67. 35.6 Granite-3.1-8B-Base
  68. 34.8 Llama 3.2 1B
  69. 34.3 DeepSeek-R1-Distill-Qwen-7B
  70. 33.6 phi-3-mini 3.8B
  71. 32.9 Granite-3.0-2B-Instruct
  72. 32.6 Granite-3.0-2B-Base
  73. 31.8 Granite-3.1-2B-Base
  74. 31.5 Granite-3.1-3B-A800M-Instruct
  75. 30.7 Qwen2.5-0.5B
  76. 30.2 Llama 3.2 3B
  77. 30.1 QwQ-32B
  78. 29.8 Granite-3.1-3B-A800M-Base
  79. 29.8 Falcon3-3B-Instruct
  80. 29.1 SmolLM2-1.7B
  81. 27.7 Granite-3.1-1B-A400M-Instruct
  82. 27.7 Qwen2-0.5B
  83. 25.9 Falcon3-1B-Instruct
  84. 25.9 Granite-3.1-1B-A400M-Base
  85. 24.7 SmolLM-1.7B
  86. 23.3 Gemma 3 270M
  87. 13.4 Gemma 3 27B
  88. 1.35 LFM2-1.2B
  1. 1 Gemma 4 31B IT bfloat16 74.3
  2. 2 Jamba 1.6 Large bfloat16 73.4
  3. 3 Llama 3.1-70B bfloat16 72.6
  4. 4 Llama 3-70B bfloat16 71.4
  5. 5 Llama 3.3 70B Instruct bfloat16 70.5
  6. 6 Aya Expanse 32B bfloat16 70.2
  7. 7 Qwen2.5-72B bfloat16 70.2
  8. 8 Llama-3.1-Nemotron-70B-Instruct bfloat16 69.9
  9. 9 Gemma 3 27B PT bfloat16 69.5
  10. 10 Qwen3.5-35B-A3B-Base bfloat16 69.2
  11. 11 c4ai-command-r-08-2024 bfloat16 67.7
  12. 12 QwQ-32B-Preview bfloat16 66.0
  13. 13 Llama-3.3-Nemotron-Super-49B-v1.5 bfloat16 65.9
  14. 14 Gemma 4 26B A4B bfloat16 65.4
  15. 15 Gemma 3 12B PT bfloat16 64.0
  16. 16 Qwen3-32B float16 63.6
  17. 17 Qwen2.5-14B bfloat16 63.1
  18. 18 Qwen3-14B-Base bfloat16 63.0
  19. 19 Qwen3-30B-A3B-Instruct Jul 2025, float16 63.0
  20. 20 Jamba 1.6 Mini bfloat16 62.9
  21. 21 Qwen3-30B-A3B-Thinking Jul 2025, float16 62.8
  22. 22 DeepSeek-R1-Distill-Qwen-32B bfloat16 62.3
  23. 23 Qwen3-30B-A3B-Base bfloat16 61.9
  24. 24 Phi-4 bfloat16 61.5
  25. 25 Qwen3-14B float16 61.4
  26. 26 Qwen2.5-32B bfloat16 61.1
  27. 27 Command R (03-2024) bfloat16 61.0
  28. 28 Gemma 4 E4B bfloat16 60.7
  29. 29 Qwen3-8B-Base bfloat16 59.4
  30. 30 Gemma 2 9B bfloat16 58.8
  31. 31 Aya Expanse 8B bfloat16 58.5
  32. 32 Qwen3-8B float16 57.6
  33. 33 DeepSeek-R1-Distill-Qwen-14B float16 57.6
  34. 34 Gemma 2 27B bfloat16 56.5
  35. 35 Qwen2.5-7B bfloat16 56.3
  36. 36 Mistral-Nemo-Base-2407 bfloat16 55.6
  37. 37 Qwen1.5-32B bfloat16 53.8
  38. 38 Qwen3-4B-Base bfloat16 53.1
  39. 39 Llama 3.1-8B bfloat16 52.3
  40. 40 Gemma 7B bfloat16 51.7
  41. 41 Qwen3-4B float16 51.3
  42. 42 Gemma 4 E2B bfloat16 50.0
  43. 43 Qwen2-7B bfloat16 49.8
  44. 44 Gemma 2 2B IT bfloat16 49.3
  45. 45 Gemma 1.1 7B IT float16 49.2
  46. 46 Qwen2.5-3B bfloat16 48.2
  47. 47 Gemma 3 4B PT bfloat16 48.1
  48. 48 Mixtral 8x7B bfloat16 48.0
  49. 49 Llama 3-8B bfloat16 47.3
  50. 50 SOLAR-10.7B-Instruct-v1.0 float16 46.9
  51. 51 Qwen1.5-14B bfloat16 45.3
  52. 52 MAmmoTH2-8B-Plus bfloat16 45.3
  53. 53 Qwen1.5-14B-Chat bfloat16 43.8
  54. 54 Yi-1.5-9B-Chat bfloat16 43.0
  55. 55 Granite-3.1-8B-Instruct bfloat16 41.7
  56. 56 Gemma 2B bfloat16 40.8
  57. 57 Qwen2.5-1.5B bfloat16 39.3
  58. 58 Granite-3.0-8B-Instruct bfloat16 39.1
  59. 59 Mistral 7B v0.3 bfloat16 39.0
  60. 60 Falcon3-10B-Instruct bfloat16 38.6
  61. 61 Granite-3.0-8B-Base bfloat16 38.6
  62. 62 Mistral 7B v0.1 bfloat16 38.5
  63. 63 MAmmoTH2-7B-Plus bfloat16 38.3
  64. 64 Zephyr 7B Beta bfloat16 36.6
  65. 65 Qwen2-1.5B bfloat16 36.1
  66. 66 Falcon3-7B-Instruct bfloat16 35.8
  67. 67 Granite-3.1-8B-Base bfloat16 35.6
  68. 68 Llama 3.2 1B bfloat16 34.8
  69. 69 DeepSeek-R1-Distill-Qwen-7B float16 34.3
  70. 70 phi-3-mini 3.8B bfloat16 33.6
  71. 71 Granite-3.0-2B-Instruct bfloat16 32.9
  72. 72 Granite-3.0-2B-Base bfloat16 32.6
  73. 73 Granite-3.1-2B-Base bfloat16 31.8
  74. 74 Granite-3.1-3B-A800M-Instruct bfloat16 31.5
  75. 75 Qwen2.5-0.5B bfloat16 30.7
  76. 76 Llama 3.2 3B bfloat16 30.2
  77. 77 QwQ-32B bfloat16 30.1
  78. 78 Granite-3.1-3B-A800M-Base bfloat16 29.8
  79. 79 Falcon3-3B-Instruct bfloat16 29.8
  80. 80 SmolLM2-1.7B bfloat16 29.1
  81. 81 Granite-3.1-1B-A400M-Instruct bfloat16 27.7
  82. 82 Qwen2-0.5B bfloat16 27.7
  83. 83 Falcon3-1B-Instruct bfloat16 25.9
  84. 84 Granite-3.1-1B-A400M-Base bfloat16 25.9
  85. 85 SmolLM-1.7B bfloat16 24.7
  86. 86 Gemma 3 270M bfloat16 23.3
  87. 87 Gemma 3 27B bfloat16 13.4
  88. 88 LFM2-1.2B bfloat16 1.35

Source: Official Open Hebrew LLM Leaderboard leaderboard

#EntryScoreSettingDateLinkSource
1 mistralai/Mistral-Large-Instruct-2407 122.6B 75.7 bfloat16 huggingface.co Official leaderboard
2 Gemma 4 31B IT 74.3 bfloat16 Apr 2026 huggingface.co Official leaderboard
3 Jamba 1.6 Large 73.4 bfloat16 Mar 2025 huggingface.co Official leaderboard
4 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16 124B 72.7 bfloat16 huggingface.co Official leaderboard
5 Llama 3.1-70B 72.6 bfloat16 Jul 2024 huggingface.co Official leaderboard
6 dicta-il/DictaLM-3.0-24B-Base 23.6B 72.5 bfloat16 huggingface.co Official leaderboard
7 CohereForAI/c4ai-command-r-plus-08-2024 103.8B 72.4 bfloat16 huggingface.co Official leaderboard
8 CohereForAI/c4ai-command-r-plus 103.8B 71.8 bfloat16 huggingface.co Official leaderboard
9 Llama 3-70B 71.4 bfloat16 Apr 2024 huggingface.co Official leaderboard
10 Llama 3.3 70B Instruct 70.5 bfloat16 Dec 2024 huggingface.co Official leaderboard
11 Aya Expanse 32B 70.2 bfloat16 Oct 2024 huggingface.co Official leaderboard
12 Qwen2.5-72B 70.2 bfloat16 Sep 2024 huggingface.co Official leaderboard
13 Llama-3.1-Nemotron-70B-Instruct 69.9 bfloat16 Oct 2024 huggingface.co Official leaderboard
14 meta-llama/Llama-3.1-70B-Instruct 70.6B 69.5 bfloat16 huggingface.co Official leaderboard
15 Gemma 3 27B PT 69.5 bfloat16 Mar 2025 huggingface.co Official leaderboard
16 Qwen3.5-35B-A3B-Base 69.2 bfloat16 Feb 2026 huggingface.co Official leaderboard
17 meta-llama/Llama-4-Scout-17B-16E 108.6B 68.7 bfloat16 huggingface.co Official leaderboard
18 c4ai-command-r-08-2024 67.7 bfloat16 Aug 2024 huggingface.co Official leaderboard
19 mistralai/Mistral-Small-24B-Instruct-2501 23.6B 67.6 bfloat16 huggingface.co Official leaderboard
20 dicta-il/DictaLM-3.0-Nemotron-12B-Base 12.3B 66.5 bfloat16 huggingface.co Official leaderboard
21 ibm-granite/granite-4.0-h-small-base 32.2B 66.4 bfloat16 huggingface.co Official leaderboard
22 QwQ-32B-Preview 66.0 bfloat16 Nov 2024 huggingface.co Official leaderboard
23 mistralai/Mistral-Small-3.1-24B-Base-2503 24B 66.0 bfloat16 huggingface.co Official leaderboard
24 Llama-3.3-Nemotron-Super-49B-v1.5 65.9 bfloat16 Jul 2025 huggingface.co Official leaderboard
25 nvidia/Nemotron-H-56B-Base-8K 56.3B 65.7 bfloat16 huggingface.co Official leaderboard
26 zai-org/GLM-4.5-Air-Base 110.5B 65.5 bfloat16 huggingface.co Official leaderboard
27 Gemma 4 26B A4B 65.4 bfloat16 Apr 2026 huggingface.co Official leaderboard
28 CohereForAI/aya-23-35B 35B 65.4 bfloat16 huggingface.co Official leaderboard
29 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 31.6B 65.1 bfloat16 huggingface.co Official leaderboard
30 618AI/dictalm2-it-qa-fine-tune 7.3B 64.8 bfloat16 huggingface.co Official leaderboard
31 mistralai/Mistral-Small-24B-Base-2501 23.6B 64.3 bfloat16 huggingface.co Official leaderboard
32 rombodawg/Rombos-LLM-V2.5-Qwen-32b 32.8B 64.1 bfloat16 huggingface.co Official leaderboard
33 Gemma 3 12B PT 64.0 bfloat16 Mar 2025 huggingface.co Official leaderboard
34 google/gemma-2-9b-it 9.2B 63.8 bfloat16 huggingface.co Official leaderboard
35 Qwen3-32B 63.6 float16 Apr 2025 huggingface.co Official leaderboard
36 UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 9.2B 63.2 bfloat16 huggingface.co Official leaderboard
37 Qwen2.5-14B 63.1 bfloat16 Sep 2024 huggingface.co Official leaderboard
38 Qwen3-14B-Base 63.0 bfloat16 Apr 2025 huggingface.co Official leaderboard
39 Qwen3-30B-A3B-Instruct Jul 2025 63.0 float16 Jul 2025 huggingface.co Official leaderboard
40 Jamba 1.6 Mini 62.9 bfloat16 Mar 2025 huggingface.co Official leaderboard
41 Qwen3-30B-A3B-Thinking Jul 2025 62.8 float16 Jul 2025 huggingface.co Official leaderboard
42 ai21labs/AI21-Jamba-1.5-Large 398.6B 62.7 8bit huggingface.co Official leaderboard
43 dicta-il/dictalm2.0 7.3B 62.6 bfloat16 huggingface.co Official leaderboard
44 DeepSeek-R1-Distill-Qwen-32B 62.3 bfloat16 Jan 2025 huggingface.co Official leaderboard
45 Qwen3-30B-A3B-Base 61.9 bfloat16 Apr 2025 huggingface.co Official leaderboard
46 Phi-4 61.5 bfloat16 Dec 2024 huggingface.co Official leaderboard
47 dicta-il/dictalm2.0-instruct 7.3B 61.5 bfloat16 huggingface.co Official leaderboard
48 Qwen3-14B 61.4 float16 Apr 2025 huggingface.co Official leaderboard
49 Qwen2.5-32B 61.1 bfloat16 Sep 2024 huggingface.co Official leaderboard
50 Command R (03-2024) 61.0 bfloat16 Mar 2024 huggingface.co Official leaderboard
51 Gemma 4 E4B 60.7 bfloat16 Mar 2026 huggingface.co Official leaderboard
52 CohereLabs/c4ai-command-r7b-arabic-02-2025 8B 60.2 bfloat16 huggingface.co Official leaderboard
53 SicariusSicariiStuff/Impish_Nemo_12B 60.0 bfloat16 huggingface.co Official leaderboard
54 SicariusSicariiStuff/Hebrew_Nemo 12.3B 59.9 bfloat16 huggingface.co Official leaderboard
55 ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew 7.3B 59.4 float16 huggingface.co Official leaderboard
56 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned 7.2B 59.4 bfloat16 huggingface.co Official leaderboard
57 Qwen3-8B-Base 59.4 bfloat16 Apr 2025 huggingface.co Official leaderboard
58 mistralai/Mistral-Nemo-Instruct-2407 12.3B 59.2 bfloat16 huggingface.co Official leaderboard
59 SeaLLMs/SeaLLM-7B-v2.5 8.5B 59.0 bfloat16 huggingface.co Official leaderboard
60 Gemma 2 9B 58.8 bfloat16 Jun 2024 huggingface.co Official leaderboard
61 Aya Expanse 8B 58.5 bfloat16 Oct 2024 huggingface.co Official leaderboard
62 Qwen3-8B 57.6 float16 Apr 2025 huggingface.co Official leaderboard
63 SicariusSicariiStuff/Zion_Alpha 7.2B 57.6 bfloat16 huggingface.co Official leaderboard
64 DeepSeek-R1-Distill-Qwen-14B 57.6 float16 Jan 2025 huggingface.co Official leaderboard
65 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP 7.2B 57.4 bfloat16 huggingface.co Official leaderboard
66 ai21labs/AI21-Jamba-1.5-Mini 51.6B 57.0 bfloat16 huggingface.co Official leaderboard
67 Gemma 2 27B 56.5 bfloat16 Jun 2024 huggingface.co Official leaderboard
68 Qwen2.5-7B 56.3 bfloat16 Sep 2024 huggingface.co Official leaderboard
69 Mistral-Nemo-Base-2407 55.6 bfloat16 Jul 2024 huggingface.co Official leaderboard
70 ronigold/dictalm2.0-instruct-fine-tuned 7.3B 55.6 float16 huggingface.co Official leaderboard
71 mlabonne/NeuralDaredevil-8B-abliterated 8B 55.2 float16 huggingface.co Official leaderboard
72 mistralai/Codestral-22B-v0.1 22.3B 54.9 bfloat16 huggingface.co Official leaderboard
73 meta-llama/Meta-Llama-3.1-8B-Instruct 8B 54.8 bfloat16 huggingface.co Official leaderboard
74 NousResearch/Hermes-3-Llama-3.1-8B 8B 54.5 bfloat16 huggingface.co Official leaderboard
75 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B 8B 54.1 float16 huggingface.co Official leaderboard
76 CohereForAI/aya-23-8B 8B 54.1 bfloat16 huggingface.co Official leaderboard
77 mistralai/Mistral-Small-Instruct-2409 22.3B 53.8 bfloat16 huggingface.co Official leaderboard
78 Qwen1.5-32B 53.8 bfloat16 Feb 2024 huggingface.co Official leaderboard
79 nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base 12.3B 53.7 bfloat16 huggingface.co Official leaderboard
80 mistralai/Ministral-8B-Instruct-2410 8B 53.3 bfloat16 huggingface.co Official leaderboard
81 Qwen3-4B-Base 53.1 bfloat16 Apr 2025 huggingface.co Official leaderboard
82 MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct 8B 52.9 bfloat16 huggingface.co Official leaderboard
83 Llama 3.1-8B 52.3 bfloat16 Jul 2024 huggingface.co Official leaderboard
84 Danielbrdz/Barcenas-Llama3-8b-ORPO 8B 52.2 float16 huggingface.co Official leaderboard
85 allenai/Olmo-3-1125-32B 32.2B 52.1 bfloat16 huggingface.co Official leaderboard
86 yam-peleg/Hebrew-Mistral-7B 7.5B 51.8 bfloat16 huggingface.co Official leaderboard
87 Gemma 7B 51.7 bfloat16 Feb 2024 huggingface.co Official leaderboard
88 dicta-il/DictaLM-3.0-1.7B-Base 1.7B 51.5 bfloat16 huggingface.co Official leaderboard
89 Qwen3-4B 51.3 float16 Apr 2025 huggingface.co Official leaderboard
90 yam-peleg/Hebrew-Gemma-11B-Instruct 10.5B 50.6 float16 huggingface.co Official leaderboard
91 DeepMount00/Llama-3-8b-Ita 8B 50.6 bfloat16 huggingface.co Official leaderboard
92 lightblue/suzume-llama-3-8B-multilingual 8B 50.2 bfloat16 huggingface.co Official leaderboard
93 ibm-granite/granite-4.0-h-tiny-base 6.9B 50.2 bfloat16 huggingface.co Official leaderboard
94 Gemma 4 E2B 50.0 bfloat16 Mar 2026 huggingface.co Official leaderboard
95 uygarkurt/llama-3-merged-linear 8B 49.9 float16 huggingface.co Official leaderboard
96 abacusai/Llama-3-Smaug-8B 8B 49.9 bfloat16 huggingface.co Official leaderboard
97 Qwen2-7B 49.8 bfloat16 Jun 2024 huggingface.co Official leaderboard
98 VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct 8B 49.6 bfloat16 huggingface.co Official leaderboard
99 Gemma 2 2B IT 49.3 bfloat16 Jul 2024 huggingface.co Official leaderboard
100 Gemma 1.1 7B IT 49.2 float16 Apr 2024 huggingface.co Official leaderboard
101 nvidia/Mistral-NeMo-Minitron-8B-Base 8.4B 49.1 bfloat16 huggingface.co Official leaderboard
102 NousResearch/Hermes-2-Pro-Llama-3-8B 8B 48.9 bfloat16 huggingface.co Official leaderboard
103 natong19/Qwen2-7B-Instruct-abliterated 7.6B 48.5 bfloat16 huggingface.co Official leaderboard
104 tiiuae/Falcon-H1-34B-Base 33.6B 48.5 bfloat16 huggingface.co Official leaderboard
105 meta-llama/Meta-Llama-3-8B-Instruct 8B 48.4 bfloat16 huggingface.co Official leaderboard
106 Qwen2.5-3B 48.2 bfloat16 Sep 2024 huggingface.co Official leaderboard
107 yam-peleg/Hebrew-Gemma-11B-V2 10.5B 48.2 bfloat16 huggingface.co Official leaderboard
108 ibm-granite/granite-4.0-h-micro-base 3.2B 48.1 bfloat16 huggingface.co Official leaderboard
109 CohereLabs/tiny-aya-base 3.4B 48.1 bfloat16 huggingface.co Official leaderboard
110 ibm-granite/granite-4.0-micro-base 3.4B 48.1 bfloat16 huggingface.co Official leaderboard
111 Gemma 3 4B PT 48.1 bfloat16 Mar 2025 huggingface.co Official leaderboard
112 Mixtral 8x7B 48.0 bfloat16 Dec 2023 huggingface.co Official leaderboard
113 Llama 3-8B 47.3 bfloat16 Apr 2024 huggingface.co Official leaderboard
114 SOLAR-10.7B-Instruct-v1.0 46.9 float16 Dec 2023 huggingface.co Official leaderboard
115 kekmodel/StopCarbon-10.7B-v5 10.7B 46.6 float16 huggingface.co Official leaderboard
116 jeonsworld/CarbonVillain-en-10.7B-v4 10.7B 46.4 float16 huggingface.co Official leaderboard
117 invalid-coder/Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp 10.7B 46.2 float16 huggingface.co Official leaderboard
118 Qwen1.5-14B 45.3 bfloat16 Feb 2024 huggingface.co Official leaderboard
119 MAmmoTH2-8B-Plus 45.3 bfloat16 May 2024 huggingface.co Official leaderboard
120 unsloth/Phi-3.5-mini-instruct 3.8B 45.2 bfloat16 huggingface.co Official leaderboard
121 01-ai/Yi-1.5-34B-32K 34.4B 44.8 bfloat16 huggingface.co Official leaderboard
122 shadowml/BeagSake-7B 7.2B 44.6 bfloat16 huggingface.co Official leaderboard
123 google/gemma-7b-it 8.5B 43.8 bfloat16 huggingface.co Official leaderboard
124 Qwen1.5-14B-Chat 43.8 bfloat16 Feb 2024 huggingface.co Official leaderboard
125 zhengr/MixTAO-7Bx2-MoE-v8.1 12.9B 43.6 bfloat16 huggingface.co Official leaderboard
126 eldogbbhed/Peagle-9b 9B 43.4 float16 huggingface.co Official leaderboard
127 vicgalle/CarbonBeagle-11B-truthy 10.7B 43.3 float16 huggingface.co Official leaderboard
128 Qwen/Qwen3-1.7B-Base 1.7B 43.3 bfloat16 huggingface.co Official leaderboard
129 mlabonne/NeuralMonarch-7B 7.2B 43.2 float16 huggingface.co Official leaderboard
130 mlabonne/AlphaMonarch-7B 7.2B 43.2 float16 huggingface.co Official leaderboard
131 Yi-1.5-9B-Chat 43.0 bfloat16 May 2024 huggingface.co Official leaderboard
132 maywell/EXAONE-3.0-7.8B-Instruct-Llamafied 42.7 ? huggingface.co Official leaderboard
133 Bingsu/exaone-3.0-7.8b-it 42.7 ? huggingface.co Official leaderboard
134 Kukedlc/NeuralSynthesis-7B-v0.1 7.2B 42.7 bfloat16 huggingface.co Official leaderboard
135 google/gemma-2-2b 2.6B 42.5 bfloat16 huggingface.co Official leaderboard
136 deepseek-ai/DeepSeek-R1-Distill-Llama-8B 8B 42.0 float16 huggingface.co Official leaderboard
137 HuggingFaceTB/SmolLM3-3B-Base 3.1B 42.0 bfloat16 huggingface.co Official leaderboard
138 Granite-3.1-8B-Instruct 41.7 bfloat16 Dec 2024 huggingface.co Official leaderboard
139 yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B 12.9B 41.5 bfloat16 huggingface.co Official leaderboard
140 nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base 8.9B 40.9 bfloat16 huggingface.co Official leaderboard
141 Gemma 2B 40.8 bfloat16 Feb 2024 huggingface.co Official leaderboard
142 HuggingFaceTB/SmolLM3-3B 3.1B 40.5 float16 huggingface.co Official leaderboard
143 yunconglong/DARE_TIES_13B 12.9B 40.3 bfloat16 huggingface.co Official leaderboard
144 Qwen2.5-1.5B 39.3 bfloat16 Sep 2024 huggingface.co Official leaderboard
145 Granite-3.0-8B-Instruct 39.1 bfloat16 Oct 2024 huggingface.co Official leaderboard
146 Mistral 7B v0.3 39.0 bfloat16 Sep 2023 huggingface.co Official leaderboard
147 Falcon3-10B-Instruct 38.6 bfloat16 Dec 2024 huggingface.co Official leaderboard
148 Granite-3.0-8B-Base 38.6 bfloat16 Oct 2024 huggingface.co Official leaderboard
149 Mistral 7B v0.1 38.5 bfloat16 Sep 2023 huggingface.co Official leaderboard
150 MAmmoTH2-7B-Plus 38.3 bfloat16 May 2024 huggingface.co Official leaderboard
151 tiiuae/Falcon-H1-7B-Base 7.6B 37.9 bfloat16 huggingface.co Official leaderboard
152 lex-hue/Delexa-7b 7.2B 37.4 float16 huggingface.co Official leaderboard
153 Zephyr 7B Beta 36.6 bfloat16 Oct 2023 huggingface.co Official leaderboard
154 Qwen2-1.5B 36.1 bfloat16 Jun 2024 huggingface.co Official leaderboard
155 meta-llama/Llama-2-7b-hf 6.7B 35.8 bfloat16 huggingface.co Official leaderboard
156 Falcon3-7B-Instruct 35.8 bfloat16 Dec 2024 huggingface.co Official leaderboard
157 royallab/ZephRP-m7b 7.2B 35.6 bfloat16 huggingface.co Official leaderboard
158 Granite-3.1-8B-Base 35.6 bfloat16 Dec 2024 huggingface.co Official leaderboard
159 tiiuae/Falcon3-10B-Base 10.3B 35.4 bfloat16 huggingface.co Official leaderboard
160 Llama 3.2 1B 34.8 bfloat16 Sep 2024 huggingface.co Official leaderboard
161 lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half 8B 34.8 bfloat16 huggingface.co Official leaderboard
162 Qwen/Qwen3-0.6B-Base 0.6B 34.5 bfloat16 huggingface.co Official leaderboard
163 DeepSeek-R1-Distill-Qwen-7B 34.3 float16 Jan 2025 huggingface.co Official leaderboard
164 XythicK/Hebrew-GPT 1.2B 34.3 bfloat16 huggingface.co Official leaderboard
165 haoranxu/ALMA-13B-R 13B 34.3 bfloat16 huggingface.co Official leaderboard
166 phi-3-mini 3.8B 33.6 bfloat16 Apr 2024 huggingface.co Official leaderboard
167 shanchen/llama3-8B-slerp-med-chinese 8B 33.3 bfloat16 huggingface.co Official leaderboard
168 tiiuae/Falcon3-7B-Base 7.5B 33.3 bfloat16 huggingface.co Official leaderboard
169 Granite-3.0-2B-Instruct 32.9 bfloat16 Oct 2024 huggingface.co Official leaderboard
170 Granite-3.0-2B-Base 32.6 bfloat16 Oct 2024 huggingface.co Official leaderboard
171 Granite-3.1-2B-Base 31.8 bfloat16 Dec 2024 huggingface.co Official leaderboard
172 Granite-3.1-3B-A800M-Instruct 31.5 bfloat16 Dec 2024 huggingface.co Official leaderboard
173 ibm-granite/granite-3.0-3b-a800m-instruct 3.4B 30.9 bfloat16 huggingface.co Official leaderboard
174 Qwen2.5-0.5B 30.7 bfloat16 Sep 2024 huggingface.co Official leaderboard
175 ibm-granite/granite-3.0-3b-a800m-base 3.4B 30.6 bfloat16 huggingface.co Official leaderboard
176 CohereForAI/c4ai-command-a-03-2025 111.1B 30.4 bfloat16 huggingface.co Official leaderboard
177 Llama 3.2 3B 30.2 bfloat16 Sep 2024 huggingface.co Official leaderboard
178 QwQ-32B 30.1 bfloat16 Mar 2025 huggingface.co Official leaderboard
179 Granite-3.1-3B-A800M-Base 29.8 bfloat16 Dec 2024 huggingface.co Official leaderboard
180 Falcon3-3B-Instruct 29.8 bfloat16 Dec 2024 huggingface.co Official leaderboard
181 SmolLM2-1.7B 29.1 bfloat16 Oct 2024 huggingface.co Official leaderboard
182 tiiuae/Falcon3-3B-Base 3.2B 29.1 bfloat16 huggingface.co Official leaderboard
183 ibm-granite/granite-4.0-h-350m-base 0.3B 28.8 bfloat16 huggingface.co Official leaderboard
184 Granite-3.1-1B-A400M-Instruct 27.7 bfloat16 Dec 2024 huggingface.co Official leaderboard
185 Qwen2-0.5B 27.7 bfloat16 Jun 2024 huggingface.co Official leaderboard
186 tiiuae/Falcon-H1-1.5B-Base 1.6B 27.6 bfloat16 huggingface.co Official leaderboard
187 ibm-granite/granite-4.0-350m-base 0.4B 27.3 bfloat16 huggingface.co Official leaderboard
188 AIDC-AI/Marco-LLM-GLO 7.6B 26.5 bfloat16 huggingface.co Official leaderboard
189 tiiuae/Falcon-H1-1.5B-Deep-Base 1.6B 26.0 bfloat16 huggingface.co Official leaderboard
190 Falcon3-1B-Instruct 25.9 bfloat16 Dec 2024 huggingface.co Official leaderboard
191 ibm-granite/granite-3.0-1b-a400m-base 1.4B 25.9 bfloat16 huggingface.co Official leaderboard
192 Granite-3.1-1B-A400M-Base 25.9 bfloat16 Dec 2024 huggingface.co Official leaderboard
193 SmolLM-1.7B 24.7 bfloat16 Jul 2024 huggingface.co Official leaderboard
194 allenai/OLMo-1B-0724-hf 1.3B 23.7 bfloat16 huggingface.co Official leaderboard
195 Gemma 3 270M 23.3 bfloat16 Aug 2025 huggingface.co Official leaderboard
196 tiiuae/Falcon-H1-0.5B-Base 0.5B 22.9 bfloat16 huggingface.co Official leaderboard
197 upstage/SOLAR-10.7B-v1.0 10.7B 22.7 float16 huggingface.co Official leaderboard
198 Gemma 3 27B 13.4 bfloat16 Mar 2025 huggingface.co Official leaderboard
199 nvidia/Llama-3.1-Nemotron-Nano-8B-v1 8B 13.0 bfloat16 huggingface.co Official leaderboard
200 LiquidAI/LFM2-2.6B 2.6B 2.22 bfloat16 huggingface.co Official leaderboard
201 LFM2-1.2B 1.35 bfloat16 Jul 2025 huggingface.co Official leaderboard
#ModelScoreSettingDateLinkSource
1 Gemma 4 31B IT 74.3 bfloat16 Apr 2026 huggingface.co Official leaderboard
2 Jamba 1.6 Large 73.4 bfloat16 Mar 2025 huggingface.co Official leaderboard
3 Llama 3.1-70B 72.6 bfloat16 Jul 2024 huggingface.co Official leaderboard
4 Llama 3-70B 71.4 bfloat16 Apr 2024 huggingface.co Official leaderboard
5 Llama 3.3 70B Instruct 70.5 bfloat16 Dec 2024 huggingface.co Official leaderboard
6 Aya Expanse 32B 70.2 bfloat16 Oct 2024 huggingface.co Official leaderboard
7 Qwen2.5-72B 70.2 bfloat16 Sep 2024 huggingface.co Official leaderboard
8 Llama-3.1-Nemotron-70B-Instruct 69.9 bfloat16 Oct 2024 huggingface.co Official leaderboard
9 Gemma 3 27B PT 69.5 bfloat16 Mar 2025 huggingface.co Official leaderboard
10 Qwen3.5-35B-A3B-Base 69.2 bfloat16 Feb 2026 huggingface.co Official leaderboard
11 c4ai-command-r-08-2024 67.7 bfloat16 Aug 2024 huggingface.co Official leaderboard
12 QwQ-32B-Preview 66.0 bfloat16 Nov 2024 huggingface.co Official leaderboard
13 Llama-3.3-Nemotron-Super-49B-v1.5 65.9 bfloat16 Jul 2025 huggingface.co Official leaderboard
14 Gemma 4 26B A4B 65.4 bfloat16 Apr 2026 huggingface.co Official leaderboard
15 Gemma 3 12B PT 64.0 bfloat16 Mar 2025 huggingface.co Official leaderboard
16 Qwen3-32B 63.6 float16 Apr 2025 huggingface.co Official leaderboard
17 Qwen2.5-14B 63.1 bfloat16 Sep 2024 huggingface.co Official leaderboard
18 Qwen3-14B-Base 63.0 bfloat16 Apr 2025 huggingface.co Official leaderboard
19 Qwen3-30B-A3B-Instruct Jul 2025 63.0 float16 Jul 2025 huggingface.co Official leaderboard
20 Jamba 1.6 Mini 62.9 bfloat16 Mar 2025 huggingface.co Official leaderboard
21 Qwen3-30B-A3B-Thinking Jul 2025 62.8 float16 Jul 2025 huggingface.co Official leaderboard
22 DeepSeek-R1-Distill-Qwen-32B 62.3 bfloat16 Jan 2025 huggingface.co Official leaderboard
23 Qwen3-30B-A3B-Base 61.9 bfloat16 Apr 2025 huggingface.co Official leaderboard
24 Phi-4 61.5 bfloat16 Dec 2024 huggingface.co Official leaderboard
25 Qwen3-14B 61.4 float16 Apr 2025 huggingface.co Official leaderboard
26 Qwen2.5-32B 61.1 bfloat16 Sep 2024 huggingface.co Official leaderboard
27 Command R (03-2024) 61.0 bfloat16 Mar 2024 huggingface.co Official leaderboard
28 Gemma 4 E4B 60.7 bfloat16 Mar 2026 huggingface.co Official leaderboard
29 Qwen3-8B-Base 59.4 bfloat16 Apr 2025 huggingface.co Official leaderboard
30 Gemma 2 9B 58.8 bfloat16 Jun 2024 huggingface.co Official leaderboard
31 Aya Expanse 8B 58.5 bfloat16 Oct 2024 huggingface.co Official leaderboard
32 Qwen3-8B 57.6 float16 Apr 2025 huggingface.co Official leaderboard
33 DeepSeek-R1-Distill-Qwen-14B 57.6 float16 Jan 2025 huggingface.co Official leaderboard
34 Gemma 2 27B 56.5 bfloat16 Jun 2024 huggingface.co Official leaderboard
35 Qwen2.5-7B 56.3 bfloat16 Sep 2024 huggingface.co Official leaderboard
36 Mistral-Nemo-Base-2407 55.6 bfloat16 Jul 2024 huggingface.co Official leaderboard
37 Qwen1.5-32B 53.8 bfloat16 Feb 2024 huggingface.co Official leaderboard
38 Qwen3-4B-Base 53.1 bfloat16 Apr 2025 huggingface.co Official leaderboard
39 Llama 3.1-8B 52.3 bfloat16 Jul 2024 huggingface.co Official leaderboard
40 Gemma 7B 51.7 bfloat16 Feb 2024 huggingface.co Official leaderboard
41 Qwen3-4B 51.3 float16 Apr 2025 huggingface.co Official leaderboard
42 Gemma 4 E2B 50.0 bfloat16 Mar 2026 huggingface.co Official leaderboard
43 Qwen2-7B 49.8 bfloat16 Jun 2024 huggingface.co Official leaderboard
44 Gemma 2 2B IT 49.3 bfloat16 Jul 2024 huggingface.co Official leaderboard
45 Gemma 1.1 7B IT 49.2 float16 Apr 2024 huggingface.co Official leaderboard
46 Qwen2.5-3B 48.2 bfloat16 Sep 2024 huggingface.co Official leaderboard
47 Gemma 3 4B PT 48.1 bfloat16 Mar 2025 huggingface.co Official leaderboard
48 Mixtral 8x7B 48.0 bfloat16 Dec 2023 huggingface.co Official leaderboard
49 Llama 3-8B 47.3 bfloat16 Apr 2024 huggingface.co Official leaderboard
50 SOLAR-10.7B-Instruct-v1.0 46.9 float16 Dec 2023 huggingface.co Official leaderboard
51 Qwen1.5-14B 45.3 bfloat16 Feb 2024 huggingface.co Official leaderboard
52 MAmmoTH2-8B-Plus 45.3 bfloat16 May 2024 huggingface.co Official leaderboard
53 Qwen1.5-14B-Chat 43.8 bfloat16 Feb 2024 huggingface.co Official leaderboard
54 Yi-1.5-9B-Chat 43.0 bfloat16 May 2024 huggingface.co Official leaderboard
55 Granite-3.1-8B-Instruct 41.7 bfloat16 Dec 2024 huggingface.co Official leaderboard
56 Gemma 2B 40.8 bfloat16 Feb 2024 huggingface.co Official leaderboard
57 Qwen2.5-1.5B 39.3 bfloat16 Sep 2024 huggingface.co Official leaderboard
58 Granite-3.0-8B-Instruct 39.1 bfloat16 Oct 2024 huggingface.co Official leaderboard
59 Mistral 7B v0.3 39.0 bfloat16 Sep 2023 huggingface.co Official leaderboard
60 Falcon3-10B-Instruct 38.6 bfloat16 Dec 2024 huggingface.co Official leaderboard
61 Granite-3.0-8B-Base 38.6 bfloat16 Oct 2024 huggingface.co Official leaderboard
62 Mistral 7B v0.1 38.5 bfloat16 Sep 2023 huggingface.co Official leaderboard
63 MAmmoTH2-7B-Plus 38.3 bfloat16 May 2024 huggingface.co Official leaderboard
64 Zephyr 7B Beta 36.6 bfloat16 Oct 2023 huggingface.co Official leaderboard
65 Qwen2-1.5B 36.1 bfloat16 Jun 2024 huggingface.co Official leaderboard
66 Falcon3-7B-Instruct 35.8 bfloat16 Dec 2024 huggingface.co Official leaderboard
67 Granite-3.1-8B-Base 35.6 bfloat16 Dec 2024 huggingface.co Official leaderboard
68 Llama 3.2 1B 34.8 bfloat16 Sep 2024 huggingface.co Official leaderboard
69 DeepSeek-R1-Distill-Qwen-7B 34.3 float16 Jan 2025 huggingface.co Official leaderboard
70 phi-3-mini 3.8B 33.6 bfloat16 Apr 2024 huggingface.co Official leaderboard
71 Granite-3.0-2B-Instruct 32.9 bfloat16 Oct 2024 huggingface.co Official leaderboard
72 Granite-3.0-2B-Base 32.6 bfloat16 Oct 2024 huggingface.co Official leaderboard
73 Granite-3.1-2B-Base 31.8 bfloat16 Dec 2024 huggingface.co Official leaderboard
74 Granite-3.1-3B-A800M-Instruct 31.5 bfloat16 Dec 2024 huggingface.co Official leaderboard
75 Qwen2.5-0.5B 30.7 bfloat16 Sep 2024 huggingface.co Official leaderboard
76 Llama 3.2 3B 30.2 bfloat16 Sep 2024 huggingface.co Official leaderboard
77 QwQ-32B 30.1 bfloat16 Mar 2025 huggingface.co Official leaderboard
78 Granite-3.1-3B-A800M-Base 29.8 bfloat16 Dec 2024 huggingface.co Official leaderboard
79 Falcon3-3B-Instruct 29.8 bfloat16 Dec 2024 huggingface.co Official leaderboard
80 SmolLM2-1.7B 29.1 bfloat16 Oct 2024 huggingface.co Official leaderboard
81 Granite-3.1-1B-A400M-Instruct 27.7 bfloat16 Dec 2024 huggingface.co Official leaderboard
82 Qwen2-0.5B 27.7 bfloat16 Jun 2024 huggingface.co Official leaderboard
83 Falcon3-1B-Instruct 25.9 bfloat16 Dec 2024 huggingface.co Official leaderboard
84 Granite-3.1-1B-A400M-Base 25.9 bfloat16 Dec 2024 huggingface.co Official leaderboard
85 SmolLM-1.7B 24.7 bfloat16 Jul 2024 huggingface.co Official leaderboard
86 Gemma 3 270M 23.3 bfloat16 Aug 2025 huggingface.co Official leaderboard
87 Gemma 3 27B 13.4 bfloat16 Mar 2025 huggingface.co Official leaderboard
88 LFM2-1.2B 1.35 bfloat16 Jul 2025 huggingface.co Official leaderboard

Rankings by part

We read 6 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as SNLI Accuracy, QA TLNLS (HeQ) and Sentiment Acc (Mafat). Gemma 4 31B IT leads 2 of them, and 4 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

SNLI Accuracy

88 models we track have a score here.

  1. 1 Qwen3.5-35B-A3B-Base ↑9 96.2
  2. 2 Qwen2.5-72B ↑5 95.5
  3. 3 Qwen3-30B-A3B-Base ↑20 93.3
  4. 4 Jamba 1.6 Large ↓2 93.1
  5. 5 Aya Expanse 32B ↑1 92.1
  6. 6 Gemma 4 31B IT ↓5 92.1
  7. 7 Qwen3-30B-A3B-Instruct Jul 2025 ↑12 90.5
  8. 8 Qwen2.5-32B ↑18 90.0
  9. 9 Qwen2.5-14B ↑8 90.0
  10. 10 Qwen3-30B-A3B-Thinking Jul 2025 ↑11 89.0

6 models have held number one on the official leaderboard since its release in March 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Qwen Cohere Meta AI21 Labs Google DeepMind
Own leaderboard Not tested since Apr 2026

Source: Official Open Hebrew LLM Leaderboard leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 20.0 40.0 60.0 80.0 2024 2025 2026 0.00 20.0 40.0 60.0 80.0 2024 2025 2026

Source: Official Open Hebrew LLM Leaderboard leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Multilingual

All 545 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%

Data audit

8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (201 entries).
  • Pass Models scored 88 models we track, plus 113 other entries on its leaderboard.
  • Warning Newest score The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 89 rows for models we track and 113 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check