Open Hebrew LLM Leaderboard
Models answer questions, resolve pronouns, classify sentiment and inference, translate passages, and answer trivia in Hebrew.
The default public table contains 201 distinct model identifiers with numeric averages across six columns. The launch announcement described four tasks.
- Made by
- Israeli National Program for NLP, Dicta and Webiks
- Released
- March 2024
- Kind
- Knowledge in many languages
- Level
- School
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Measures
- Capability
- Subject
- Languages and literature
- Interaction
- Single turn
- Items from
- Another benchmark
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
-
Gemma 4 31B IT #101 on the capability index
- Last model added
- April 2026
- Scores refreshed
- November 20th, 2026
mistralai/Mistral-Large-Instruct-2407 tops the leaderboard with 75.7 Gemma 4 31B IT leads with 74.3 among models released by April 2026
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Gemma 4 31B IT, at 74.3. This view shows only models we track, each at its best.
The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 75.7 mistralai/Mistral-Large-Instruct-2407
- 74.3
Gemma 4 31B IT
- 73.4 Jamba 1.6 Large
- 72.7 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16
- 72.6
Llama 3.1-70B
- 72.5 dicta-il/DictaLM-3.0-24B-Base
- 72.4 CohereForAI/c4ai-command-r-plus-08-2024
- 71.8 CohereForAI/c4ai-command-r-plus
- 71.4
Llama 3-70B
- 70.5
Llama 3.3 70B Instruct
- 70.2
Aya Expanse 32B
- 70.2
Qwen2.5-72B
- 69.9
Llama-3.1-Nemotron-70B-Instruct
- 69.5 meta-llama/Llama-3.1-70B-Instruct
- 69.5
Gemma 3 27B PT
- 69.2
Qwen3.5-35B-A3B-Base
- 68.7 meta-llama/Llama-4-Scout-17B-16E
- 67.7 c4ai-command-r-08-2024
- 67.6 mistralai/Mistral-Small-24B-Instruct-2501
- 66.5 dicta-il/DictaLM-3.0-Nemotron-12B-Base
- 66.4 ibm-granite/granite-4.0-h-small-base
- 66.0
QwQ-32B-Preview
- 66.0 mistralai/Mistral-Small-3.1-24B-Base-2503
- 65.9
Llama-3.3-Nemotron-Super-49B-v1.5
- 65.7 nvidia/Nemotron-H-56B-Base-8K
- 65.5 zai-org/GLM-4.5-Air-Base
- 65.4
Gemma 4 26B A4B
- 65.4 CohereForAI/aya-23-35B
- 65.1 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16
- 64.8 618AI/dictalm2-it-qa-fine-tune
- 64.3 mistralai/Mistral-Small-24B-Base-2501
- 64.1 rombodawg/Rombos-LLM-V2.5-Qwen-32b
- 64.0
Gemma 3 12B PT
- 63.8 google/gemma-2-9b-it
- 63.6
Qwen3-32B
- 63.2 UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
- 63.1
Qwen2.5-14B
- 63.0
Qwen3-14B-Base
- 63.0
Qwen3-30B-A3B-Instruct
- 62.9 Jamba 1.6 Mini
- 62.8
Qwen3-30B-A3B-Thinking
- 62.7 ai21labs/AI21-Jamba-1.5-Large
- 62.6 dicta-il/dictalm2.0
- 62.3
DeepSeek-R1-Distill-Qwen-32B
- 61.9
Qwen3-30B-A3B-Base
- 61.5
Phi-4
- 61.5 dicta-il/dictalm2.0-instruct
- 61.4
Qwen3-14B
- 61.1
Qwen2.5-32B
- 61.0
Command R (03-2024)
- 60.7
Gemma 4 E4B
- 60.2 CohereLabs/c4ai-command-r7b-arabic-02-2025
- 60.0 SicariusSicariiStuff/Impish_Nemo_12B
- 59.9 SicariusSicariiStuff/Hebrew_Nemo
- 59.4 ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew
- 59.4 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned
- 59.4
Qwen3-8B-Base
- 59.2 mistralai/Mistral-Nemo-Instruct-2407
- 59.0 SeaLLMs/SeaLLM-7B-v2.5
- 58.8
Gemma 2 9B
- 58.5
Aya Expanse 8B
- 57.6
Qwen3-8B
- 57.6 SicariusSicariiStuff/Zion_Alpha
- 57.6
DeepSeek-R1-Distill-Qwen-14B
- 57.4 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP
- 57.0 ai21labs/AI21-Jamba-1.5-Mini
- 56.5
Gemma 2 27B
- 56.3
Qwen2.5-7B
- 55.6
Mistral-Nemo-Base-2407
- 55.6 ronigold/dictalm2.0-instruct-fine-tuned
- 55.2 mlabonne/NeuralDaredevil-8B-abliterated
- 54.9 mistralai/Codestral-22B-v0.1
- 54.8 meta-llama/Meta-Llama-3.1-8B-Instruct
- 54.5 NousResearch/Hermes-3-Llama-3.1-8B
- 54.1 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B
- 54.1 CohereForAI/aya-23-8B
- 53.8 mistralai/Mistral-Small-Instruct-2409
- 53.8
Qwen1.5-32B
- 53.7 nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base
- 53.3 mistralai/Ministral-8B-Instruct-2410
- 53.1
Qwen3-4B-Base
- 52.9 MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct
- 52.3
Llama 3.1-8B
- 52.2 Danielbrdz/Barcenas-Llama3-8b-ORPO
- 52.1 allenai/Olmo-3-1125-32B
- 51.8 yam-peleg/Hebrew-Mistral-7B
- 51.7
Gemma 7B
- 51.5 dicta-il/DictaLM-3.0-1.7B-Base
- 51.3
Qwen3-4B
- 50.6 yam-peleg/Hebrew-Gemma-11B-Instruct
- 50.6 DeepMount00/Llama-3-8b-Ita
- 50.2 lightblue/suzume-llama-3-8B-multilingual
- 50.2 ibm-granite/granite-4.0-h-tiny-base
- 50.0
Gemma 4 E2B
- 49.9 uygarkurt/llama-3-merged-linear
- 49.9 abacusai/Llama-3-Smaug-8B
- 49.8
Qwen2-7B
- 49.6 VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
- 49.3
Gemma 2 2B IT
- 49.2
Gemma 1.1 7B IT
- 49.1 nvidia/Mistral-NeMo-Minitron-8B-Base
- 48.9 NousResearch/Hermes-2-Pro-Llama-3-8B
- 48.5 natong19/Qwen2-7B-Instruct-abliterated
- 48.5 tiiuae/Falcon-H1-34B-Base
- 48.4 meta-llama/Meta-Llama-3-8B-Instruct
- 48.2
Qwen2.5-3B
- 48.2 yam-peleg/Hebrew-Gemma-11B-V2
- 48.1 ibm-granite/granite-4.0-h-micro-base
- 48.1 CohereLabs/tiny-aya-base
- 48.1 ibm-granite/granite-4.0-micro-base
- 48.1
Gemma 3 4B PT
- 48.0
Mixtral 8x7B
- 47.3
Llama 3-8B
- 46.9
SOLAR-10.7B-Instruct-v1.0
- 46.6 kekmodel/StopCarbon-10.7B-v5
- 46.4 jeonsworld/CarbonVillain-en-10.7B-v4
- 46.2 invalid-coder/Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp
- 45.3
Qwen1.5-14B
- 45.3
MAmmoTH2-8B-Plus
- 45.2 unsloth/Phi-3.5-mini-instruct
- 44.8 01-ai/Yi-1.5-34B-32K
- 44.6 shadowml/BeagSake-7B
- 43.8 google/gemma-7b-it
- 43.8
Qwen1.5-14B-Chat
- 43.6 zhengr/MixTAO-7Bx2-MoE-v8.1
- 43.4 eldogbbhed/Peagle-9b
- 43.3 vicgalle/CarbonBeagle-11B-truthy
- 43.3 Qwen/Qwen3-1.7B-Base
- 43.2 mlabonne/NeuralMonarch-7B
- 43.2 mlabonne/AlphaMonarch-7B
- 43.0
Yi-1.5-9B-Chat
- 42.7 maywell/EXAONE-3.0-7.8B-Instruct-Llamafied
- 42.7 Bingsu/exaone-3.0-7.8b-it
- 42.7 Kukedlc/NeuralSynthesis-7B-v0.1
- 42.5 google/gemma-2-2b
- 42.0 deepseek-ai/DeepSeek-R1-Distill-Llama-8B
- 42.0 HuggingFaceTB/SmolLM3-3B-Base
- 41.7 Granite-3.1-8B-Instruct
- 41.5 yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B
- 40.9 nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base
- 40.8
Gemma 2B
- 40.5 HuggingFaceTB/SmolLM3-3B
- 40.3 yunconglong/DARE_TIES_13B
- 39.3
Qwen2.5-1.5B
- 39.1 Granite-3.0-8B-Instruct
- 39.0
Mistral 7B v0.3
- 38.6
Falcon3-10B-Instruct
- 38.6 Granite-3.0-8B-Base
- 38.5
Mistral 7B v0.1
- 38.3
MAmmoTH2-7B-Plus
- 37.9 tiiuae/Falcon-H1-7B-Base
- 37.4 lex-hue/Delexa-7b
- 36.6
Zephyr 7B Beta
- 36.1
Qwen2-1.5B
- 35.8 meta-llama/Llama-2-7b-hf
- 35.8
Falcon3-7B-Instruct
- 35.6 royallab/ZephRP-m7b
- 35.6 Granite-3.1-8B-Base
- 35.4 tiiuae/Falcon3-10B-Base
- 34.8
Llama 3.2 1B
- 34.8 lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half
- 34.5 Qwen/Qwen3-0.6B-Base
- 34.3
DeepSeek-R1-Distill-Qwen-7B
- 34.3 XythicK/Hebrew-GPT
- 34.3 haoranxu/ALMA-13B-R
- 33.6
phi-3-mini 3.8B
- 33.3 shanchen/llama3-8B-slerp-med-chinese
- 33.3 tiiuae/Falcon3-7B-Base
- 32.9 Granite-3.0-2B-Instruct
- 32.6 Granite-3.0-2B-Base
- 31.8 Granite-3.1-2B-Base
- 31.5 Granite-3.1-3B-A800M-Instruct
- 30.9 ibm-granite/granite-3.0-3b-a800m-instruct
- 30.7
Qwen2.5-0.5B
- 30.6 ibm-granite/granite-3.0-3b-a800m-base
- 30.4 CohereForAI/c4ai-command-a-03-2025
- 30.2
Llama 3.2 3B
- 30.1
QwQ-32B
- 29.8 Granite-3.1-3B-A800M-Base
- 29.8
Falcon3-3B-Instruct
- 29.1
SmolLM2-1.7B
- 29.1 tiiuae/Falcon3-3B-Base
- 28.8 ibm-granite/granite-4.0-h-350m-base
- 27.7 Granite-3.1-1B-A400M-Instruct
- 27.7
Qwen2-0.5B
- 27.6 tiiuae/Falcon-H1-1.5B-Base
- 27.3 ibm-granite/granite-4.0-350m-base
- 26.5 AIDC-AI/Marco-LLM-GLO
- 26.0 tiiuae/Falcon-H1-1.5B-Deep-Base
- 25.9
Falcon3-1B-Instruct
- 25.9 ibm-granite/granite-3.0-1b-a400m-base
- 25.9 Granite-3.1-1B-A400M-Base
- 24.7
SmolLM-1.7B
- 23.7 allenai/OLMo-1B-0724-hf
- 23.3
Gemma 3 270M
- 22.9 tiiuae/Falcon-H1-0.5B-Base
- 22.7 upstage/SOLAR-10.7B-v1.0
- 13.4
Gemma 3 27B
- 13.0 nvidia/Llama-3.1-Nemotron-Nano-8B-v1
- 2.22 LiquidAI/LFM2-2.6B
- 1.35 LFM2-1.2B
- 1 mistralai/Mistral-Large-Instruct-2407 bfloat16 75.7
- 2
Gemma 4 31B IT bfloat16 74.3
- 3 Jamba 1.6 Large bfloat16 73.4
- 4 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16 bfloat16 72.7
- 5
Llama 3.1-70B bfloat16 72.6
- 6 dicta-il/DictaLM-3.0-24B-Base bfloat16 72.5
- 7 CohereForAI/c4ai-command-r-plus-08-2024 bfloat16 72.4
- 8 CohereForAI/c4ai-command-r-plus bfloat16 71.8
- 9
Llama 3-70B bfloat16 71.4
- 10
Llama 3.3 70B Instruct bfloat16 70.5
- 11
Aya Expanse 32B bfloat16 70.2
- 12
Qwen2.5-72B bfloat16 70.2
- 13
Llama-3.1-Nemotron-70B-Instruct bfloat16 69.9
- 14 meta-llama/Llama-3.1-70B-Instruct bfloat16 69.5
- 15
Gemma 3 27B PT bfloat16 69.5
- 16
Qwen3.5-35B-A3B-Base bfloat16 69.2
- 17 meta-llama/Llama-4-Scout-17B-16E bfloat16 68.7
- 18 c4ai-command-r-08-2024 bfloat16 67.7
- 19 mistralai/Mistral-Small-24B-Instruct-2501 bfloat16 67.6
- 20 dicta-il/DictaLM-3.0-Nemotron-12B-Base bfloat16 66.5
- 21 ibm-granite/granite-4.0-h-small-base bfloat16 66.4
- 22
QwQ-32B-Preview bfloat16 66.0
- 23 mistralai/Mistral-Small-3.1-24B-Base-2503 bfloat16 66.0
- 24
Llama-3.3-Nemotron-Super-49B-v1.5 bfloat16 65.9
- 25 nvidia/Nemotron-H-56B-Base-8K bfloat16 65.7
- 26 zai-org/GLM-4.5-Air-Base bfloat16 65.5
- 27
Gemma 4 26B A4B bfloat16 65.4
- 28 CohereForAI/aya-23-35B bfloat16 65.4
- 29 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 bfloat16 65.1
- 30 618AI/dictalm2-it-qa-fine-tune bfloat16 64.8
- 31 mistralai/Mistral-Small-24B-Base-2501 bfloat16 64.3
- 32 rombodawg/Rombos-LLM-V2.5-Qwen-32b bfloat16 64.1
- 33
Gemma 3 12B PT bfloat16 64.0
- 34 google/gemma-2-9b-it bfloat16 63.8
- 35
Qwen3-32B float16 63.6
- 36 UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 bfloat16 63.2
- 37
Qwen2.5-14B bfloat16 63.1
- 38
Qwen3-14B-Base bfloat16 63.0
- 39
Qwen3-30B-A3B-Instruct Jul 2025, float16 63.0
- 40 Jamba 1.6 Mini bfloat16 62.9
- 41
Qwen3-30B-A3B-Thinking Jul 2025, float16 62.8
- 42 ai21labs/AI21-Jamba-1.5-Large 8bit 62.7
- 43 dicta-il/dictalm2.0 bfloat16 62.6
- 44
DeepSeek-R1-Distill-Qwen-32B bfloat16 62.3
- 45
Qwen3-30B-A3B-Base bfloat16 61.9
- 46
Phi-4 bfloat16 61.5
- 47 dicta-il/dictalm2.0-instruct bfloat16 61.5
- 48
Qwen3-14B float16 61.4
- 49
Qwen2.5-32B bfloat16 61.1
- 50
Command R (03-2024) bfloat16 61.0
- 51
Gemma 4 E4B bfloat16 60.7
- 52 CohereLabs/c4ai-command-r7b-arabic-02-2025 bfloat16 60.2
- 53 SicariusSicariiStuff/Impish_Nemo_12B bfloat16 60.0
- 54 SicariusSicariiStuff/Hebrew_Nemo bfloat16 59.9
- 55 ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew float16 59.4
- 56 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned bfloat16 59.4
- 57
Qwen3-8B-Base bfloat16 59.4
- 58 mistralai/Mistral-Nemo-Instruct-2407 bfloat16 59.2
- 59 SeaLLMs/SeaLLM-7B-v2.5 bfloat16 59.0
- 60
Gemma 2 9B bfloat16 58.8
- 61
Aya Expanse 8B bfloat16 58.5
- 62
Qwen3-8B float16 57.6
- 63 SicariusSicariiStuff/Zion_Alpha bfloat16 57.6
- 64
DeepSeek-R1-Distill-Qwen-14B float16 57.6
- 65 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP bfloat16 57.4
- 66 ai21labs/AI21-Jamba-1.5-Mini bfloat16 57.0
- 67
Gemma 2 27B bfloat16 56.5
- 68
Qwen2.5-7B bfloat16 56.3
- 69
Mistral-Nemo-Base-2407 bfloat16 55.6
- 70 ronigold/dictalm2.0-instruct-fine-tuned float16 55.6
- 71 mlabonne/NeuralDaredevil-8B-abliterated float16 55.2
- 72 mistralai/Codestral-22B-v0.1 bfloat16 54.9
- 73 meta-llama/Meta-Llama-3.1-8B-Instruct bfloat16 54.8
- 74 NousResearch/Hermes-3-Llama-3.1-8B bfloat16 54.5
- 75 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B float16 54.1
- 76 CohereForAI/aya-23-8B bfloat16 54.1
- 77 mistralai/Mistral-Small-Instruct-2409 bfloat16 53.8
- 78
Qwen1.5-32B bfloat16 53.8
- 79 nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base bfloat16 53.7
- 80 mistralai/Ministral-8B-Instruct-2410 bfloat16 53.3
- 81
Qwen3-4B-Base bfloat16 53.1
- 82 MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct bfloat16 52.9
- 83
Llama 3.1-8B bfloat16 52.3
- 84 Danielbrdz/Barcenas-Llama3-8b-ORPO float16 52.2
- 85 allenai/Olmo-3-1125-32B bfloat16 52.1
- 86 yam-peleg/Hebrew-Mistral-7B bfloat16 51.8
- 87
Gemma 7B bfloat16 51.7
- 88 dicta-il/DictaLM-3.0-1.7B-Base bfloat16 51.5
- 89
Qwen3-4B float16 51.3
- 90 yam-peleg/Hebrew-Gemma-11B-Instruct float16 50.6
- 91 DeepMount00/Llama-3-8b-Ita bfloat16 50.6
- 92 lightblue/suzume-llama-3-8B-multilingual bfloat16 50.2
- 93 ibm-granite/granite-4.0-h-tiny-base bfloat16 50.2
- 94
Gemma 4 E2B bfloat16 50.0
- 95 uygarkurt/llama-3-merged-linear float16 49.9
- 96 abacusai/Llama-3-Smaug-8B bfloat16 49.9
- 97
Qwen2-7B bfloat16 49.8
- 98 VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct bfloat16 49.6
- 99
Gemma 2 2B IT bfloat16 49.3
- 100
Gemma 1.1 7B IT float16 49.2
- 101 nvidia/Mistral-NeMo-Minitron-8B-Base bfloat16 49.1
- 102 NousResearch/Hermes-2-Pro-Llama-3-8B bfloat16 48.9
- 103 natong19/Qwen2-7B-Instruct-abliterated bfloat16 48.5
- 104 tiiuae/Falcon-H1-34B-Base bfloat16 48.5
- 105 meta-llama/Meta-Llama-3-8B-Instruct bfloat16 48.4
- 106
Qwen2.5-3B bfloat16 48.2
- 107 yam-peleg/Hebrew-Gemma-11B-V2 bfloat16 48.2
- 108 ibm-granite/granite-4.0-h-micro-base bfloat16 48.1
- 109 CohereLabs/tiny-aya-base bfloat16 48.1
- 110 ibm-granite/granite-4.0-micro-base bfloat16 48.1
- 111
Gemma 3 4B PT bfloat16 48.1
- 112
Mixtral 8x7B bfloat16 48.0
- 113
Llama 3-8B bfloat16 47.3
- 114
SOLAR-10.7B-Instruct-v1.0 float16 46.9
- 115 kekmodel/StopCarbon-10.7B-v5 float16 46.6
- 116 jeonsworld/CarbonVillain-en-10.7B-v4 float16 46.4
- 117 invalid-coder/Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp float16 46.2
- 118
Qwen1.5-14B bfloat16 45.3
- 119
MAmmoTH2-8B-Plus bfloat16 45.3
- 120 unsloth/Phi-3.5-mini-instruct bfloat16 45.2
- 121 01-ai/Yi-1.5-34B-32K bfloat16 44.8
- 122 shadowml/BeagSake-7B bfloat16 44.6
- 123 google/gemma-7b-it bfloat16 43.8
- 124
Qwen1.5-14B-Chat bfloat16 43.8
- 125 zhengr/MixTAO-7Bx2-MoE-v8.1 bfloat16 43.6
- 126 eldogbbhed/Peagle-9b float16 43.4
- 127 vicgalle/CarbonBeagle-11B-truthy float16 43.3
- 128 Qwen/Qwen3-1.7B-Base bfloat16 43.3
- 129 mlabonne/NeuralMonarch-7B float16 43.2
- 130 mlabonne/AlphaMonarch-7B float16 43.2
- 131
Yi-1.5-9B-Chat bfloat16 43.0
- 132 maywell/EXAONE-3.0-7.8B-Instruct-Llamafied ? 42.7
- 133 Bingsu/exaone-3.0-7.8b-it ? 42.7
- 134 Kukedlc/NeuralSynthesis-7B-v0.1 bfloat16 42.7
- 135 google/gemma-2-2b bfloat16 42.5
- 136 deepseek-ai/DeepSeek-R1-Distill-Llama-8B float16 42.0
- 137 HuggingFaceTB/SmolLM3-3B-Base bfloat16 42.0
- 138 Granite-3.1-8B-Instruct bfloat16 41.7
- 139 yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B bfloat16 41.5
- 140 nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base bfloat16 40.9
- 141
Gemma 2B bfloat16 40.8
- 142 HuggingFaceTB/SmolLM3-3B float16 40.5
- 143 yunconglong/DARE_TIES_13B bfloat16 40.3
- 144
Qwen2.5-1.5B bfloat16 39.3
- 145 Granite-3.0-8B-Instruct bfloat16 39.1
- 146
Mistral 7B v0.3 bfloat16 39.0
- 147
Falcon3-10B-Instruct bfloat16 38.6
- 148 Granite-3.0-8B-Base bfloat16 38.6
- 149
Mistral 7B v0.1 bfloat16 38.5
- 150
MAmmoTH2-7B-Plus bfloat16 38.3
- 151 tiiuae/Falcon-H1-7B-Base bfloat16 37.9
- 152 lex-hue/Delexa-7b float16 37.4
- 153
Zephyr 7B Beta bfloat16 36.6
- 154
Qwen2-1.5B bfloat16 36.1
- 155 meta-llama/Llama-2-7b-hf bfloat16 35.8
- 156
Falcon3-7B-Instruct bfloat16 35.8
- 157 royallab/ZephRP-m7b bfloat16 35.6
- 158 Granite-3.1-8B-Base bfloat16 35.6
- 159 tiiuae/Falcon3-10B-Base bfloat16 35.4
- 160
Llama 3.2 1B bfloat16 34.8
- 161 lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half bfloat16 34.8
- 162 Qwen/Qwen3-0.6B-Base bfloat16 34.5
- 163
DeepSeek-R1-Distill-Qwen-7B float16 34.3
- 164 XythicK/Hebrew-GPT bfloat16 34.3
- 165 haoranxu/ALMA-13B-R bfloat16 34.3
- 166
phi-3-mini 3.8B bfloat16 33.6
- 167 shanchen/llama3-8B-slerp-med-chinese bfloat16 33.3
- 168 tiiuae/Falcon3-7B-Base bfloat16 33.3
- 169 Granite-3.0-2B-Instruct bfloat16 32.9
- 170 Granite-3.0-2B-Base bfloat16 32.6
- 171 Granite-3.1-2B-Base bfloat16 31.8
- 172 Granite-3.1-3B-A800M-Instruct bfloat16 31.5
- 173 ibm-granite/granite-3.0-3b-a800m-instruct bfloat16 30.9
- 174
Qwen2.5-0.5B bfloat16 30.7
- 175 ibm-granite/granite-3.0-3b-a800m-base bfloat16 30.6
- 176 CohereForAI/c4ai-command-a-03-2025 bfloat16 30.4
- 177
Llama 3.2 3B bfloat16 30.2
- 178
QwQ-32B bfloat16 30.1
- 179 Granite-3.1-3B-A800M-Base bfloat16 29.8
- 180
Falcon3-3B-Instruct bfloat16 29.8
- 181
SmolLM2-1.7B bfloat16 29.1
- 182 tiiuae/Falcon3-3B-Base bfloat16 29.1
- 183 ibm-granite/granite-4.0-h-350m-base bfloat16 28.8
- 184 Granite-3.1-1B-A400M-Instruct bfloat16 27.7
- 185
Qwen2-0.5B bfloat16 27.7
- 186 tiiuae/Falcon-H1-1.5B-Base bfloat16 27.6
- 187 ibm-granite/granite-4.0-350m-base bfloat16 27.3
- 188 AIDC-AI/Marco-LLM-GLO bfloat16 26.5
- 189 tiiuae/Falcon-H1-1.5B-Deep-Base bfloat16 26.0
- 190
Falcon3-1B-Instruct bfloat16 25.9
- 191 ibm-granite/granite-3.0-1b-a400m-base bfloat16 25.9
- 192 Granite-3.1-1B-A400M-Base bfloat16 25.9
- 193
SmolLM-1.7B bfloat16 24.7
- 194 allenai/OLMo-1B-0724-hf bfloat16 23.7
- 195
Gemma 3 270M bfloat16 23.3
- 196 tiiuae/Falcon-H1-0.5B-Base bfloat16 22.9
- 197 upstage/SOLAR-10.7B-v1.0 float16 22.7
- 198
Gemma 3 27B bfloat16 13.4
- 199 nvidia/Llama-3.1-Nemotron-Nano-8B-v1 bfloat16 13.0
- 200 LiquidAI/LFM2-2.6B bfloat16 2.22
- 201 LFM2-1.2B bfloat16 1.35
Source: Official Open Hebrew LLM Leaderboard leaderboard
- 74.3
Gemma 4 31B IT
- 73.4 Jamba 1.6 Large
- 72.6
Llama 3.1-70B
- 71.4
Llama 3-70B
- 70.5
Llama 3.3 70B Instruct
- 70.2
Aya Expanse 32B
- 70.2
Qwen2.5-72B
- 69.9
Llama-3.1-Nemotron-70B-Instruct
- 69.5
Gemma 3 27B PT
- 69.2
Qwen3.5-35B-A3B-Base
- 67.7 c4ai-command-r-08-2024
- 66.0
QwQ-32B-Preview
- 65.9
Llama-3.3-Nemotron-Super-49B-v1.5
- 65.4
Gemma 4 26B A4B
- 64.0
Gemma 3 12B PT
- 63.6
Qwen3-32B
- 63.1
Qwen2.5-14B
- 63.0
Qwen3-14B-Base
- 63.0
Qwen3-30B-A3B-Instruct
- 62.9 Jamba 1.6 Mini
- 62.8
Qwen3-30B-A3B-Thinking
- 62.3
DeepSeek-R1-Distill-Qwen-32B
- 61.9
Qwen3-30B-A3B-Base
- 61.5
Phi-4
- 61.4
Qwen3-14B
- 61.1
Qwen2.5-32B
- 61.0
Command R (03-2024)
- 60.7
Gemma 4 E4B
- 59.4
Qwen3-8B-Base
- 58.8
Gemma 2 9B
- 58.5
Aya Expanse 8B
- 57.6
Qwen3-8B
- 57.6
DeepSeek-R1-Distill-Qwen-14B
- 56.5
Gemma 2 27B
- 56.3
Qwen2.5-7B
- 55.6
Mistral-Nemo-Base-2407
- 53.8
Qwen1.5-32B
- 53.1
Qwen3-4B-Base
- 52.3
Llama 3.1-8B
- 51.7
Gemma 7B
- 51.3
Qwen3-4B
- 50.0
Gemma 4 E2B
- 49.8
Qwen2-7B
- 49.3
Gemma 2 2B IT
- 49.2
Gemma 1.1 7B IT
- 48.2
Qwen2.5-3B
- 48.1
Gemma 3 4B PT
- 48.0
Mixtral 8x7B
- 47.3
Llama 3-8B
- 46.9
SOLAR-10.7B-Instruct-v1.0
- 45.3
Qwen1.5-14B
- 45.3
MAmmoTH2-8B-Plus
- 43.8
Qwen1.5-14B-Chat
- 43.0
Yi-1.5-9B-Chat
- 41.7 Granite-3.1-8B-Instruct
- 40.8
Gemma 2B
- 39.3
Qwen2.5-1.5B
- 39.1 Granite-3.0-8B-Instruct
- 39.0
Mistral 7B v0.3
- 38.6
Falcon3-10B-Instruct
- 38.6 Granite-3.0-8B-Base
- 38.5
Mistral 7B v0.1
- 38.3
MAmmoTH2-7B-Plus
- 36.6
Zephyr 7B Beta
- 36.1
Qwen2-1.5B
- 35.8
Falcon3-7B-Instruct
- 35.6 Granite-3.1-8B-Base
- 34.8
Llama 3.2 1B
- 34.3
DeepSeek-R1-Distill-Qwen-7B
- 33.6
phi-3-mini 3.8B
- 32.9 Granite-3.0-2B-Instruct
- 32.6 Granite-3.0-2B-Base
- 31.8 Granite-3.1-2B-Base
- 31.5 Granite-3.1-3B-A800M-Instruct
- 30.7
Qwen2.5-0.5B
- 30.2
Llama 3.2 3B
- 30.1
QwQ-32B
- 29.8 Granite-3.1-3B-A800M-Base
- 29.8
Falcon3-3B-Instruct
- 29.1
SmolLM2-1.7B
- 27.7 Granite-3.1-1B-A400M-Instruct
- 27.7
Qwen2-0.5B
- 25.9
Falcon3-1B-Instruct
- 25.9 Granite-3.1-1B-A400M-Base
- 24.7
SmolLM-1.7B
- 23.3
Gemma 3 270M
- 13.4
Gemma 3 27B
- 1.35 LFM2-1.2B
- 1
Gemma 4 31B IT bfloat16 74.3
- 2 Jamba 1.6 Large bfloat16 73.4
- 3
Llama 3.1-70B bfloat16 72.6
- 4
Llama 3-70B bfloat16 71.4
- 5
Llama 3.3 70B Instruct bfloat16 70.5
- 6
Aya Expanse 32B bfloat16 70.2
- 7
Qwen2.5-72B bfloat16 70.2
- 8
Llama-3.1-Nemotron-70B-Instruct bfloat16 69.9
- 9
Gemma 3 27B PT bfloat16 69.5
- 10
Qwen3.5-35B-A3B-Base bfloat16 69.2
- 11 c4ai-command-r-08-2024 bfloat16 67.7
- 12
QwQ-32B-Preview bfloat16 66.0
- 13
Llama-3.3-Nemotron-Super-49B-v1.5 bfloat16 65.9
- 14
Gemma 4 26B A4B bfloat16 65.4
- 15
Gemma 3 12B PT bfloat16 64.0
- 16
Qwen3-32B float16 63.6
- 17
Qwen2.5-14B bfloat16 63.1
- 18
Qwen3-14B-Base bfloat16 63.0
- 19
Qwen3-30B-A3B-Instruct Jul 2025, float16 63.0
- 20 Jamba 1.6 Mini bfloat16 62.9
- 21
Qwen3-30B-A3B-Thinking Jul 2025, float16 62.8
- 22
DeepSeek-R1-Distill-Qwen-32B bfloat16 62.3
- 23
Qwen3-30B-A3B-Base bfloat16 61.9
- 24
Phi-4 bfloat16 61.5
- 25
Qwen3-14B float16 61.4
- 26
Qwen2.5-32B bfloat16 61.1
- 27
Command R (03-2024) bfloat16 61.0
- 28
Gemma 4 E4B bfloat16 60.7
- 29
Qwen3-8B-Base bfloat16 59.4
- 30
Gemma 2 9B bfloat16 58.8
- 31
Aya Expanse 8B bfloat16 58.5
- 32
Qwen3-8B float16 57.6
- 33
DeepSeek-R1-Distill-Qwen-14B float16 57.6
- 34
Gemma 2 27B bfloat16 56.5
- 35
Qwen2.5-7B bfloat16 56.3
- 36
Mistral-Nemo-Base-2407 bfloat16 55.6
- 37
Qwen1.5-32B bfloat16 53.8
- 38
Qwen3-4B-Base bfloat16 53.1
- 39
Llama 3.1-8B bfloat16 52.3
- 40
Gemma 7B bfloat16 51.7
- 41
Qwen3-4B float16 51.3
- 42
Gemma 4 E2B bfloat16 50.0
- 43
Qwen2-7B bfloat16 49.8
- 44
Gemma 2 2B IT bfloat16 49.3
- 45
Gemma 1.1 7B IT float16 49.2
- 46
Qwen2.5-3B bfloat16 48.2
- 47
Gemma 3 4B PT bfloat16 48.1
- 48
Mixtral 8x7B bfloat16 48.0
- 49
Llama 3-8B bfloat16 47.3
- 50
SOLAR-10.7B-Instruct-v1.0 float16 46.9
- 51
Qwen1.5-14B bfloat16 45.3
- 52
MAmmoTH2-8B-Plus bfloat16 45.3
- 53
Qwen1.5-14B-Chat bfloat16 43.8
- 54
Yi-1.5-9B-Chat bfloat16 43.0
- 55 Granite-3.1-8B-Instruct bfloat16 41.7
- 56
Gemma 2B bfloat16 40.8
- 57
Qwen2.5-1.5B bfloat16 39.3
- 58 Granite-3.0-8B-Instruct bfloat16 39.1
- 59
Mistral 7B v0.3 bfloat16 39.0
- 60
Falcon3-10B-Instruct bfloat16 38.6
- 61 Granite-3.0-8B-Base bfloat16 38.6
- 62
Mistral 7B v0.1 bfloat16 38.5
- 63
MAmmoTH2-7B-Plus bfloat16 38.3
- 64
Zephyr 7B Beta bfloat16 36.6
- 65
Qwen2-1.5B bfloat16 36.1
- 66
Falcon3-7B-Instruct bfloat16 35.8
- 67 Granite-3.1-8B-Base bfloat16 35.6
- 68
Llama 3.2 1B bfloat16 34.8
- 69
DeepSeek-R1-Distill-Qwen-7B float16 34.3
- 70
phi-3-mini 3.8B bfloat16 33.6
- 71 Granite-3.0-2B-Instruct bfloat16 32.9
- 72 Granite-3.0-2B-Base bfloat16 32.6
- 73 Granite-3.1-2B-Base bfloat16 31.8
- 74 Granite-3.1-3B-A800M-Instruct bfloat16 31.5
- 75
Qwen2.5-0.5B bfloat16 30.7
- 76
Llama 3.2 3B bfloat16 30.2
- 77
QwQ-32B bfloat16 30.1
- 78 Granite-3.1-3B-A800M-Base bfloat16 29.8
- 79
Falcon3-3B-Instruct bfloat16 29.8
- 80
SmolLM2-1.7B bfloat16 29.1
- 81 Granite-3.1-1B-A400M-Instruct bfloat16 27.7
- 82
Qwen2-0.5B bfloat16 27.7
- 83
Falcon3-1B-Instruct bfloat16 25.9
- 84 Granite-3.1-1B-A400M-Base bfloat16 25.9
- 85
SmolLM-1.7B bfloat16 24.7
- 86
Gemma 3 270M bfloat16 23.3
- 87
Gemma 3 27B bfloat16 13.4
- 88 LFM2-1.2B bfloat16 1.35
Source: Official Open Hebrew LLM Leaderboard leaderboard
| # | Entry | Score | Setting | Date | Link | Source |
|---|---|---|---|---|---|---|
| 1 | mistralai/Mistral-Large-Instruct-2407 122.6B | 75.7 | bfloat16 | huggingface.co | Official leaderboard | |
| 2 | | 74.3 | bfloat16 | Apr 2026 | huggingface.co | Official leaderboard |
| 3 | Jamba 1.6 Large | 73.4 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 4 | nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16 124B | 72.7 | bfloat16 | huggingface.co | Official leaderboard | |
| 5 | | 72.6 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 6 | dicta-il/DictaLM-3.0-24B-Base 23.6B | 72.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 7 | CohereForAI/c4ai-command-r-plus-08-2024 103.8B | 72.4 | bfloat16 | huggingface.co | Official leaderboard | |
| 8 | CohereForAI/c4ai-command-r-plus 103.8B | 71.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 9 | | 71.4 | bfloat16 | Apr 2024 | huggingface.co | Official leaderboard |
| 10 | | 70.5 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 11 | | 70.2 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 12 | | 70.2 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 13 | | 69.9 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 14 | meta-llama/Llama-3.1-70B-Instruct 70.6B | 69.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 15 | | 69.5 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 16 | | 69.2 | bfloat16 | Feb 2026 | huggingface.co | Official leaderboard |
| 17 | meta-llama/Llama-4-Scout-17B-16E 108.6B | 68.7 | bfloat16 | huggingface.co | Official leaderboard | |
| 18 | c4ai-command-r-08-2024 | 67.7 | bfloat16 | Aug 2024 | huggingface.co | Official leaderboard |
| 19 | mistralai/Mistral-Small-24B-Instruct-2501 23.6B | 67.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 20 | dicta-il/DictaLM-3.0-Nemotron-12B-Base 12.3B | 66.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 21 | ibm-granite/granite-4.0-h-small-base 32.2B | 66.4 | bfloat16 | huggingface.co | Official leaderboard | |
| 22 | | 66.0 | bfloat16 | Nov 2024 | huggingface.co | Official leaderboard |
| 23 | mistralai/Mistral-Small-3.1-24B-Base-2503 24B | 66.0 | bfloat16 | huggingface.co | Official leaderboard | |
| 24 | | 65.9 | bfloat16 | Jul 2025 | huggingface.co | Official leaderboard |
| 25 | nvidia/Nemotron-H-56B-Base-8K 56.3B | 65.7 | bfloat16 | huggingface.co | Official leaderboard | |
| 26 | zai-org/GLM-4.5-Air-Base 110.5B | 65.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 27 | | 65.4 | bfloat16 | Apr 2026 | huggingface.co | Official leaderboard |
| 28 | CohereForAI/aya-23-35B 35B | 65.4 | bfloat16 | huggingface.co | Official leaderboard | |
| 29 | nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 31.6B | 65.1 | bfloat16 | huggingface.co | Official leaderboard | |
| 30 | 618AI/dictalm2-it-qa-fine-tune 7.3B | 64.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 31 | mistralai/Mistral-Small-24B-Base-2501 23.6B | 64.3 | bfloat16 | huggingface.co | Official leaderboard | |
| 32 | rombodawg/Rombos-LLM-V2.5-Qwen-32b 32.8B | 64.1 | bfloat16 | huggingface.co | Official leaderboard | |
| 33 | | 64.0 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 34 | google/gemma-2-9b-it 9.2B | 63.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 35 | | 63.6 | float16 | Apr 2025 | huggingface.co | Official leaderboard |
| 36 | UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 9.2B | 63.2 | bfloat16 | huggingface.co | Official leaderboard | |
| 37 | | 63.1 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 38 | | 63.0 | bfloat16 | Apr 2025 | huggingface.co | Official leaderboard |
| 39 | | 63.0 | float16 | Jul 2025 | huggingface.co | Official leaderboard |
| 40 | Jamba 1.6 Mini | 62.9 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 41 | | 62.8 | float16 | Jul 2025 | huggingface.co | Official leaderboard |
| 42 | ai21labs/AI21-Jamba-1.5-Large 398.6B | 62.7 | 8bit | huggingface.co | Official leaderboard | |
| 43 | dicta-il/dictalm2.0 7.3B | 62.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 44 | | 62.3 | bfloat16 | Jan 2025 | huggingface.co | Official leaderboard |
| 45 | | 61.9 | bfloat16 | Apr 2025 | huggingface.co | Official leaderboard |
| 46 | | 61.5 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 47 | dicta-il/dictalm2.0-instruct 7.3B | 61.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 48 | | 61.4 | float16 | Apr 2025 | huggingface.co | Official leaderboard |
| 49 | | 61.1 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 50 | | 61.0 | bfloat16 | Mar 2024 | huggingface.co | Official leaderboard |
| 51 | | 60.7 | bfloat16 | Mar 2026 | huggingface.co | Official leaderboard |
| 52 | CohereLabs/c4ai-command-r7b-arabic-02-2025 8B | 60.2 | bfloat16 | huggingface.co | Official leaderboard | |
| 53 | SicariusSicariiStuff/Impish_Nemo_12B | 60.0 | bfloat16 | huggingface.co | Official leaderboard | |
| 54 | SicariusSicariiStuff/Hebrew_Nemo 12.3B | 59.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 55 | ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew 7.3B | 59.4 | float16 | huggingface.co | Official leaderboard | |
| 56 | SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned 7.2B | 59.4 | bfloat16 | huggingface.co | Official leaderboard | |
| 57 | | 59.4 | bfloat16 | Apr 2025 | huggingface.co | Official leaderboard |
| 58 | mistralai/Mistral-Nemo-Instruct-2407 12.3B | 59.2 | bfloat16 | huggingface.co | Official leaderboard | |
| 59 | SeaLLMs/SeaLLM-7B-v2.5 8.5B | 59.0 | bfloat16 | huggingface.co | Official leaderboard | |
| 60 | | 58.8 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 61 | | 58.5 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 62 | | 57.6 | float16 | Apr 2025 | huggingface.co | Official leaderboard |
| 63 | SicariusSicariiStuff/Zion_Alpha 7.2B | 57.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 64 | | 57.6 | float16 | Jan 2025 | huggingface.co | Official leaderboard |
| 65 | SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP 7.2B | 57.4 | bfloat16 | huggingface.co | Official leaderboard | |
| 66 | ai21labs/AI21-Jamba-1.5-Mini 51.6B | 57.0 | bfloat16 | huggingface.co | Official leaderboard | |
| 67 | | 56.5 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 68 | | 56.3 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 69 | | 55.6 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 70 | ronigold/dictalm2.0-instruct-fine-tuned 7.3B | 55.6 | float16 | huggingface.co | Official leaderboard | |
| 71 | mlabonne/NeuralDaredevil-8B-abliterated 8B | 55.2 | float16 | huggingface.co | Official leaderboard | |
| 72 | mistralai/Codestral-22B-v0.1 22.3B | 54.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 73 | meta-llama/Meta-Llama-3.1-8B-Instruct 8B | 54.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 74 | NousResearch/Hermes-3-Llama-3.1-8B 8B | 54.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 75 | vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B 8B | 54.1 | float16 | huggingface.co | Official leaderboard | |
| 76 | CohereForAI/aya-23-8B 8B | 54.1 | bfloat16 | huggingface.co | Official leaderboard | |
| 77 | mistralai/Mistral-Small-Instruct-2409 22.3B | 53.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 78 | | 53.8 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 79 | nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base 12.3B | 53.7 | bfloat16 | huggingface.co | Official leaderboard | |
| 80 | mistralai/Ministral-8B-Instruct-2410 8B | 53.3 | bfloat16 | huggingface.co | Official leaderboard | |
| 81 | | 53.1 | bfloat16 | Apr 2025 | huggingface.co | Official leaderboard |
| 82 | MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct 8B | 52.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 83 | | 52.3 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 84 | Danielbrdz/Barcenas-Llama3-8b-ORPO 8B | 52.2 | float16 | huggingface.co | Official leaderboard | |
| 85 | allenai/Olmo-3-1125-32B 32.2B | 52.1 | bfloat16 | huggingface.co | Official leaderboard | |
| 86 | yam-peleg/Hebrew-Mistral-7B 7.5B | 51.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 87 | | 51.7 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 88 | dicta-il/DictaLM-3.0-1.7B-Base 1.7B | 51.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 89 | | 51.3 | float16 | Apr 2025 | huggingface.co | Official leaderboard |
| 90 | yam-peleg/Hebrew-Gemma-11B-Instruct 10.5B | 50.6 | float16 | huggingface.co | Official leaderboard | |
| 91 | DeepMount00/Llama-3-8b-Ita 8B | 50.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 92 | lightblue/suzume-llama-3-8B-multilingual 8B | 50.2 | bfloat16 | huggingface.co | Official leaderboard | |
| 93 | ibm-granite/granite-4.0-h-tiny-base 6.9B | 50.2 | bfloat16 | huggingface.co | Official leaderboard | |
| 94 | | 50.0 | bfloat16 | Mar 2026 | huggingface.co | Official leaderboard |
| 95 | uygarkurt/llama-3-merged-linear 8B | 49.9 | float16 | huggingface.co | Official leaderboard | |
| 96 | abacusai/Llama-3-Smaug-8B 8B | 49.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 97 | | 49.8 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 98 | VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct 8B | 49.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 99 | | 49.3 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 100 | | 49.2 | float16 | Apr 2024 | huggingface.co | Official leaderboard |
| 101 | nvidia/Mistral-NeMo-Minitron-8B-Base 8.4B | 49.1 | bfloat16 | huggingface.co | Official leaderboard | |
| 102 | NousResearch/Hermes-2-Pro-Llama-3-8B 8B | 48.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 103 | natong19/Qwen2-7B-Instruct-abliterated 7.6B | 48.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 104 | tiiuae/Falcon-H1-34B-Base 33.6B | 48.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 105 | meta-llama/Meta-Llama-3-8B-Instruct 8B | 48.4 | bfloat16 | huggingface.co | Official leaderboard | |
| 106 | | 48.2 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 107 | yam-peleg/Hebrew-Gemma-11B-V2 10.5B | 48.2 | bfloat16 | huggingface.co | Official leaderboard | |
| 108 | ibm-granite/granite-4.0-h-micro-base 3.2B | 48.1 | bfloat16 | huggingface.co | Official leaderboard | |
| 109 | CohereLabs/tiny-aya-base 3.4B | 48.1 | bfloat16 | huggingface.co | Official leaderboard | |
| 110 | ibm-granite/granite-4.0-micro-base 3.4B | 48.1 | bfloat16 | huggingface.co | Official leaderboard | |
| 111 | | 48.1 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 112 | | 48.0 | bfloat16 | Dec 2023 | huggingface.co | Official leaderboard |
| 113 | | 47.3 | bfloat16 | Apr 2024 | huggingface.co | Official leaderboard |
| 114 | | 46.9 | float16 | Dec 2023 | huggingface.co | Official leaderboard |
| 115 | kekmodel/StopCarbon-10.7B-v5 10.7B | 46.6 | float16 | huggingface.co | Official leaderboard | |
| 116 | jeonsworld/CarbonVillain-en-10.7B-v4 10.7B | 46.4 | float16 | huggingface.co | Official leaderboard | |
| 117 | invalid-coder/Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp 10.7B | 46.2 | float16 | huggingface.co | Official leaderboard | |
| 118 | | 45.3 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 119 | | 45.3 | bfloat16 | May 2024 | huggingface.co | Official leaderboard |
| 120 | unsloth/Phi-3.5-mini-instruct 3.8B | 45.2 | bfloat16 | huggingface.co | Official leaderboard | |
| 121 | 01-ai/Yi-1.5-34B-32K 34.4B | 44.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 122 | shadowml/BeagSake-7B 7.2B | 44.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 123 | google/gemma-7b-it 8.5B | 43.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 124 | | 43.8 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 125 | zhengr/MixTAO-7Bx2-MoE-v8.1 12.9B | 43.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 126 | eldogbbhed/Peagle-9b 9B | 43.4 | float16 | huggingface.co | Official leaderboard | |
| 127 | vicgalle/CarbonBeagle-11B-truthy 10.7B | 43.3 | float16 | huggingface.co | Official leaderboard | |
| 128 | Qwen/Qwen3-1.7B-Base 1.7B | 43.3 | bfloat16 | huggingface.co | Official leaderboard | |
| 129 | mlabonne/NeuralMonarch-7B 7.2B | 43.2 | float16 | huggingface.co | Official leaderboard | |
| 130 | mlabonne/AlphaMonarch-7B 7.2B | 43.2 | float16 | huggingface.co | Official leaderboard | |
| 131 | | 43.0 | bfloat16 | May 2024 | huggingface.co | Official leaderboard |
| 132 | maywell/EXAONE-3.0-7.8B-Instruct-Llamafied | 42.7 | ? | huggingface.co | Official leaderboard | |
| 133 | Bingsu/exaone-3.0-7.8b-it | 42.7 | ? | huggingface.co | Official leaderboard | |
| 134 | Kukedlc/NeuralSynthesis-7B-v0.1 7.2B | 42.7 | bfloat16 | huggingface.co | Official leaderboard | |
| 135 | google/gemma-2-2b 2.6B | 42.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 136 | deepseek-ai/DeepSeek-R1-Distill-Llama-8B 8B | 42.0 | float16 | huggingface.co | Official leaderboard | |
| 137 | HuggingFaceTB/SmolLM3-3B-Base 3.1B | 42.0 | bfloat16 | huggingface.co | Official leaderboard | |
| 138 | Granite-3.1-8B-Instruct | 41.7 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 139 | yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B 12.9B | 41.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 140 | nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base 8.9B | 40.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 141 | | 40.8 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 142 | HuggingFaceTB/SmolLM3-3B 3.1B | 40.5 | float16 | huggingface.co | Official leaderboard | |
| 143 | yunconglong/DARE_TIES_13B 12.9B | 40.3 | bfloat16 | huggingface.co | Official leaderboard | |
| 144 | | 39.3 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 145 | Granite-3.0-8B-Instruct | 39.1 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 146 | | 39.0 | bfloat16 | Sep 2023 | huggingface.co | Official leaderboard |
| 147 | | 38.6 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 148 | Granite-3.0-8B-Base | 38.6 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 149 | | 38.5 | bfloat16 | Sep 2023 | huggingface.co | Official leaderboard |
| 150 | | 38.3 | bfloat16 | May 2024 | huggingface.co | Official leaderboard |
| 151 | tiiuae/Falcon-H1-7B-Base 7.6B | 37.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 152 | lex-hue/Delexa-7b 7.2B | 37.4 | float16 | huggingface.co | Official leaderboard | |
| 153 | | 36.6 | bfloat16 | Oct 2023 | huggingface.co | Official leaderboard |
| 154 | | 36.1 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 155 | meta-llama/Llama-2-7b-hf 6.7B | 35.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 156 | | 35.8 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 157 | royallab/ZephRP-m7b 7.2B | 35.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 158 | Granite-3.1-8B-Base | 35.6 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 159 | tiiuae/Falcon3-10B-Base 10.3B | 35.4 | bfloat16 | huggingface.co | Official leaderboard | |
| 160 | | 34.8 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 161 | lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half 8B | 34.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 162 | Qwen/Qwen3-0.6B-Base 0.6B | 34.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 163 | | 34.3 | float16 | Jan 2025 | huggingface.co | Official leaderboard |
| 164 | XythicK/Hebrew-GPT 1.2B | 34.3 | bfloat16 | huggingface.co | Official leaderboard | |
| 165 | haoranxu/ALMA-13B-R 13B | 34.3 | bfloat16 | huggingface.co | Official leaderboard | |
| 166 | | 33.6 | bfloat16 | Apr 2024 | huggingface.co | Official leaderboard |
| 167 | shanchen/llama3-8B-slerp-med-chinese 8B | 33.3 | bfloat16 | huggingface.co | Official leaderboard | |
| 168 | tiiuae/Falcon3-7B-Base 7.5B | 33.3 | bfloat16 | huggingface.co | Official leaderboard | |
| 169 | Granite-3.0-2B-Instruct | 32.9 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 170 | Granite-3.0-2B-Base | 32.6 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 171 | Granite-3.1-2B-Base | 31.8 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 172 | Granite-3.1-3B-A800M-Instruct | 31.5 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 173 | ibm-granite/granite-3.0-3b-a800m-instruct 3.4B | 30.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 174 | | 30.7 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 175 | ibm-granite/granite-3.0-3b-a800m-base 3.4B | 30.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 176 | CohereForAI/c4ai-command-a-03-2025 111.1B | 30.4 | bfloat16 | huggingface.co | Official leaderboard | |
| 177 | | 30.2 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 178 | | 30.1 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 179 | Granite-3.1-3B-A800M-Base | 29.8 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 180 | | 29.8 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 181 | | 29.1 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 182 | tiiuae/Falcon3-3B-Base 3.2B | 29.1 | bfloat16 | huggingface.co | Official leaderboard | |
| 183 | ibm-granite/granite-4.0-h-350m-base 0.3B | 28.8 | bfloat16 | huggingface.co | Official leaderboard | |
| 184 | Granite-3.1-1B-A400M-Instruct | 27.7 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 185 | | 27.7 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 186 | tiiuae/Falcon-H1-1.5B-Base 1.6B | 27.6 | bfloat16 | huggingface.co | Official leaderboard | |
| 187 | ibm-granite/granite-4.0-350m-base 0.4B | 27.3 | bfloat16 | huggingface.co | Official leaderboard | |
| 188 | AIDC-AI/Marco-LLM-GLO 7.6B | 26.5 | bfloat16 | huggingface.co | Official leaderboard | |
| 189 | tiiuae/Falcon-H1-1.5B-Deep-Base 1.6B | 26.0 | bfloat16 | huggingface.co | Official leaderboard | |
| 190 | | 25.9 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 191 | ibm-granite/granite-3.0-1b-a400m-base 1.4B | 25.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 192 | Granite-3.1-1B-A400M-Base | 25.9 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 193 | | 24.7 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 194 | allenai/OLMo-1B-0724-hf 1.3B | 23.7 | bfloat16 | huggingface.co | Official leaderboard | |
| 195 | | 23.3 | bfloat16 | Aug 2025 | huggingface.co | Official leaderboard |
| 196 | tiiuae/Falcon-H1-0.5B-Base 0.5B | 22.9 | bfloat16 | huggingface.co | Official leaderboard | |
| 197 | upstage/SOLAR-10.7B-v1.0 10.7B | 22.7 | float16 | huggingface.co | Official leaderboard | |
| 198 | | 13.4 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 199 | nvidia/Llama-3.1-Nemotron-Nano-8B-v1 8B | 13.0 | bfloat16 | huggingface.co | Official leaderboard | |
| 200 | LiquidAI/LFM2-2.6B 2.6B | 2.22 | bfloat16 | huggingface.co | Official leaderboard | |
| 201 | LFM2-1.2B | 1.35 | bfloat16 | Jul 2025 | huggingface.co | Official leaderboard |
| # | Model | Score | Setting | Date | Link | Source |
|---|---|---|---|---|---|---|
| 1 | | 74.3 | bfloat16 | Apr 2026 | huggingface.co | Official leaderboard |
| 2 | Jamba 1.6 Large | 73.4 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 3 | | 72.6 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 4 | | 71.4 | bfloat16 | Apr 2024 | huggingface.co | Official leaderboard |
| 5 | | 70.5 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 6 | | 70.2 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 7 | | 70.2 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 8 | | 69.9 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 9 | | 69.5 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 10 | | 69.2 | bfloat16 | Feb 2026 | huggingface.co | Official leaderboard |
| 11 | c4ai-command-r-08-2024 | 67.7 | bfloat16 | Aug 2024 | huggingface.co | Official leaderboard |
| 12 | | 66.0 | bfloat16 | Nov 2024 | huggingface.co | Official leaderboard |
| 13 | | 65.9 | bfloat16 | Jul 2025 | huggingface.co | Official leaderboard |
| 14 | | 65.4 | bfloat16 | Apr 2026 | huggingface.co | Official leaderboard |
| 15 | | 64.0 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 16 | | 63.6 | float16 | Apr 2025 | huggingface.co | Official leaderboard |
| 17 | | 63.1 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 18 | | 63.0 | bfloat16 | Apr 2025 | huggingface.co | Official leaderboard |
| 19 | | 63.0 | float16 | Jul 2025 | huggingface.co | Official leaderboard |
| 20 | Jamba 1.6 Mini | 62.9 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 21 | | 62.8 | float16 | Jul 2025 | huggingface.co | Official leaderboard |
| 22 | | 62.3 | bfloat16 | Jan 2025 | huggingface.co | Official leaderboard |
| 23 | | 61.9 | bfloat16 | Apr 2025 | huggingface.co | Official leaderboard |
| 24 | | 61.5 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 25 | | 61.4 | float16 | Apr 2025 | huggingface.co | Official leaderboard |
| 26 | | 61.1 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 27 | | 61.0 | bfloat16 | Mar 2024 | huggingface.co | Official leaderboard |
| 28 | | 60.7 | bfloat16 | Mar 2026 | huggingface.co | Official leaderboard |
| 29 | | 59.4 | bfloat16 | Apr 2025 | huggingface.co | Official leaderboard |
| 30 | | 58.8 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 31 | | 58.5 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 32 | | 57.6 | float16 | Apr 2025 | huggingface.co | Official leaderboard |
| 33 | | 57.6 | float16 | Jan 2025 | huggingface.co | Official leaderboard |
| 34 | | 56.5 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 35 | | 56.3 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 36 | | 55.6 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 37 | | 53.8 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 38 | | 53.1 | bfloat16 | Apr 2025 | huggingface.co | Official leaderboard |
| 39 | | 52.3 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 40 | | 51.7 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 41 | | 51.3 | float16 | Apr 2025 | huggingface.co | Official leaderboard |
| 42 | | 50.0 | bfloat16 | Mar 2026 | huggingface.co | Official leaderboard |
| 43 | | 49.8 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 44 | | 49.3 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 45 | | 49.2 | float16 | Apr 2024 | huggingface.co | Official leaderboard |
| 46 | | 48.2 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 47 | | 48.1 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 48 | | 48.0 | bfloat16 | Dec 2023 | huggingface.co | Official leaderboard |
| 49 | | 47.3 | bfloat16 | Apr 2024 | huggingface.co | Official leaderboard |
| 50 | | 46.9 | float16 | Dec 2023 | huggingface.co | Official leaderboard |
| 51 | | 45.3 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 52 | | 45.3 | bfloat16 | May 2024 | huggingface.co | Official leaderboard |
| 53 | | 43.8 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 54 | | 43.0 | bfloat16 | May 2024 | huggingface.co | Official leaderboard |
| 55 | Granite-3.1-8B-Instruct | 41.7 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 56 | | 40.8 | bfloat16 | Feb 2024 | huggingface.co | Official leaderboard |
| 57 | | 39.3 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 58 | Granite-3.0-8B-Instruct | 39.1 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 59 | | 39.0 | bfloat16 | Sep 2023 | huggingface.co | Official leaderboard |
| 60 | | 38.6 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 61 | Granite-3.0-8B-Base | 38.6 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 62 | | 38.5 | bfloat16 | Sep 2023 | huggingface.co | Official leaderboard |
| 63 | | 38.3 | bfloat16 | May 2024 | huggingface.co | Official leaderboard |
| 64 | | 36.6 | bfloat16 | Oct 2023 | huggingface.co | Official leaderboard |
| 65 | | 36.1 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 66 | | 35.8 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 67 | Granite-3.1-8B-Base | 35.6 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 68 | | 34.8 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 69 | | 34.3 | float16 | Jan 2025 | huggingface.co | Official leaderboard |
| 70 | | 33.6 | bfloat16 | Apr 2024 | huggingface.co | Official leaderboard |
| 71 | Granite-3.0-2B-Instruct | 32.9 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 72 | Granite-3.0-2B-Base | 32.6 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 73 | Granite-3.1-2B-Base | 31.8 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 74 | Granite-3.1-3B-A800M-Instruct | 31.5 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 75 | | 30.7 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 76 | | 30.2 | bfloat16 | Sep 2024 | huggingface.co | Official leaderboard |
| 77 | | 30.1 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 78 | Granite-3.1-3B-A800M-Base | 29.8 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 79 | | 29.8 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 80 | | 29.1 | bfloat16 | Oct 2024 | huggingface.co | Official leaderboard |
| 81 | Granite-3.1-1B-A400M-Instruct | 27.7 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 82 | | 27.7 | bfloat16 | Jun 2024 | huggingface.co | Official leaderboard |
| 83 | | 25.9 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 84 | Granite-3.1-1B-A400M-Base | 25.9 | bfloat16 | Dec 2024 | huggingface.co | Official leaderboard |
| 85 | | 24.7 | bfloat16 | Jul 2024 | huggingface.co | Official leaderboard |
| 86 | | 23.3 | bfloat16 | Aug 2025 | huggingface.co | Official leaderboard |
| 87 | | 13.4 | bfloat16 | Mar 2025 | huggingface.co | Official leaderboard |
| 88 | LFM2-1.2B | 1.35 | bfloat16 | Jul 2025 | huggingface.co | Official leaderboard |
Rankings by part
We read 6 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as SNLI Accuracy, QA TLNLS (HeQ) and Sentiment Acc (Mafat). Gemma 4 31B IT leads 2 of them, and 4 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
SNLI Accuracy
88 models we track have a score here.
- 1
Qwen3.5-35B-A3B-Base ↑9 96.2
- 2
Qwen2.5-72B ↑5 95.5
- 3
Qwen3-30B-A3B-Base ↑20 93.3
- 4 Jamba 1.6 Large ↓2 93.1
- 5
Aya Expanse 32B ↑1 92.1
- 6
Gemma 4 31B IT ↓5 92.1
- 7
Qwen3-30B-A3B-Instruct Jul 2025 ↑12 90.5
- 8
Qwen2.5-32B ↑18 90.0
- 9
Qwen2.5-14B ↑8 90.0
- 10
Qwen3-30B-A3B-Thinking Jul 2025 ↑11 89.0
QA TLNLS (HeQ)
88 models we track have a score here.
- 1
Qwen2.5-72B ↑6 87.0
- 2
Gemma 4 31B IT ↓1 86.8
- 3
Qwen2.5-32B ↑23 86.5
- 4
Llama 3.3 70B Instruct ↑1 84.4
- 5
QwQ-32B ↑72 83.9
- 6
QwQ-32B-Preview ↑6 83.6
- 7
Qwen3-32B ↑9 83.5
- 8
Aya Expanse 32B ↓2 83.4
- 9
Llama-3.3-Nemotron-Super-49B-v1.5 ↑4 83.3
- 10
Llama-3.1-Nemotron-70B-Instruct ↓2 82.5
Sentiment Acc (Mafat)
88 models we track have a score here.
- 1
Llama 3.1-70B ↑2 77.5
- 2
Llama 3.3 70B Instruct ↑3 77.3
- 3
Llama 3-70B ↑1 76.7
- 4
Llama-3.1-Nemotron-70B-Instruct ↑4 75.5
- 5
Llama-3.3-Nemotron-Super-49B-v1.5 ↑8 74.4
- 6
Qwen2.5-32B ↑20 74.1
- 7
DeepSeek-R1-Distill-Qwen-32B ↑15 74.0
- 8
QwQ-32B-Preview ↑4 73.9
- 9
Qwen3-30B-A3B-Thinking Jul 2025 ↑12 72.1
- 10
Qwen2.5-14B ↑7 71.7
Winograd (Binary) Acc (V. Schwartz)
88 models we track have a score here.
- 1
Gemma 4 31B IT 87.4
- 2
Llama-3.1-Nemotron-70B-Instruct ↑6 83.8
- 3
Aya Expanse 32B ↑3 82.7
- 4 c4ai-command-r-08-2024 ↑7 82.4
- 5
Llama 3.3 70B Instruct 81.7
- 6 Jamba 1.6 Large ↓4 81.7
- 7
Gemma 3 27B PT ↑2 81.7
- 8
Llama 3-70B ↓4 81.3
- 9
Llama 3.1-70B ↓6 80.9
- 10
Qwen3.5-35B-A3B-Base 79.5
Translation BLEU
88 models we track have a score here.
- 1
Gemma 4 31B IT 37.8
- 2
Gemma 3 27B PT ↑7 36.5
- 3 Jamba 1.6 Large ↓1 36.0
- 4 c4ai-command-r-08-2024 ↑7 35.8
- 5
Gemma 4 26B A4B ↑9 35.7
- 6
Llama 3.1-70B ↓3 34.7
- 7
Aya Expanse 32B ↓1 34.7
- 8
Llama 3-70B ↓4 34.5
- 9
Gemma 3 12B PT ↑6 34.5
- 10
Llama 3.3 70B Instruct ↓5 33.9
Israeli Trivia
88 models we track have a score here.
- 1 Jamba 1.6 Large ↑1 80.1
- 2
Llama 3.1-70B ↑1 73.4
- 3
Gemma 4 31B IT ↓2 73.4
- 4
Gemma 3 27B PT ↑5 70.4
- 5
Llama 3-70B ↓1 69.4
- 6
Llama 3.3 70B Instruct ↓1 68.8
- 7
Llama-3.1-Nemotron-70B-Instruct ↑1 66.5
- 8
Gemma 4 26B A4B ↑6 65.8
- 9 c4ai-command-r-08-2024 ↑2 65.1
- 10 Jamba 1.6 Mini ↑10 62.8
6 models have held number one on the official leaderboard since its release in March 2024
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official Open Hebrew LLM Leaderboard leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official Open Hebrew LLM Leaderboard leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (201 entries).
- Pass Models scored 88 models we track, plus 113 other entries on its leaderboard.
- Warning Newest score The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B.
- Note Frontier models None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101).
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on huggingface.co: 89 rows for models we track and 113 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.