HeQ
Models extract answers from Hebrew Wikipedia and Israeli technology reporting, or identify questions that the passage cannot answer.
The Open Hebrew LLM Leaderboard reports HeQ scores for 201 distinct model identifiers in its default view.
- Made by
- Bar-Ilan University, Webiks and collaborators
- Released
- May 2023
- Kind
- Reading in many languages
- Level
- School
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Measures
- Capability
- Language
- Hebrew
- Interaction
- Single turn
- Items from
- Real work and use
- Refresh
- One fixed set
- Run by
- Anyone
- Strongest model tested
-
Gemma 4 31B IT #101 on the capability index
- Last model added
- April 2026
- Scores refreshed
- November 20th, 2026
CohereForAI/c4ai-command-a-03-2025 tops the leaderboard with 88.4% Qwen2.5-72B leads with 87% among models released by April 2026
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Qwen2.5-72B, at 87%. This view shows only models we track, each at its best.
The newest model with a score here came out in April 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 16 more
- 88.4 CohereForAI/c4ai-command-a-03-2025
- 87.2 mistralai/Mistral-Large-Instruct-2407
- 87.0
Qwen2.5-72B
- 86.8
Gemma 4 31B IT
- 86.5
Qwen2.5-32B
- 86.0 mistralai/Mistral-Small-24B-Instruct-2501
- 84.4
Llama 3.3 70B Instruct
- 83.9
QwQ-32B
- 83.9 meta-llama/Llama-3.1-70B-Instruct
- 83.7 google/gemma-2-9b-it
- 83.6
QwQ-32B-Preview
- 83.5
Qwen3-32B
- 83.4
Aya Expanse 32B
- 83.3
Llama-3.3-Nemotron-Super-49B-v1.5
- 82.7 CohereForAI/c4ai-command-r-plus
- 82.5
Llama-3.1-Nemotron-70B-Instruct
- 82.5 CohereForAI/c4ai-command-r-plus-08-2024
- 82.1
Qwen3.5-35B-A3B-Base
- 82.1
Qwen3-14B
- 81.9 nvidia/Nemotron-H-56B-Base-8K
- 81.5 Jamba 1.6 Large
- 81.1
Qwen2.5-14B
- 80.9
Qwen3-14B-Base
- 80.3
Llama 3.1-70B
- 80.3
Gemma 3 27B
- 79.7 UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
- 79.4 CohereForAI/aya-23-35B
- 78.9
Qwen3-8B-Base
- 78.8 meta-llama/Llama-4-Scout-17B-16E
- 78.5
Qwen3-8B
- 78.4
Llama 3-70B
- 78.3
Gemma 3 27B PT
- 77.9 mistralai/Mistral-Small-3.1-24B-Base-2503
- 77.8 dicta-il/DictaLM-3.0-Nemotron-12B-Base
- 77.4
Qwen3-30B-A3B-Instruct
- 76.9
DeepSeek-R1-Distill-Qwen-32B
- 76.6 dicta-il/DictaLM-3.0-24B-Base
- 76.6 mistralai/Mistral-Small-Instruct-2409
- 76.2 c4ai-command-r-08-2024
- 76.1
Qwen3-30B-A3B-Thinking
- 75.7 Jamba 1.6 Mini
- 75.5 zai-org/GLM-4.5-Air-Base
- 75.0 mistralai/Mistral-Small-24B-Base-2501
- 74.7
Phi-4
- 74.3 ibm-granite/granite-4.0-h-small-base
- 74.2
Gemma 2 27B
- 74.1 618AI/dictalm2-it-qa-fine-tune
- 73.8 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16
- 73.4 mistralai/Mistral-Nemo-Instruct-2407
- 73.0 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP
- 73.0
Qwen3-30B-A3B-Base
- 72.0 CohereLabs/tiny-aya-base
- 72.0 SeaLLMs/SeaLLM-7B-v2.5
- 71.8 CohereLabs/c4ai-command-r7b-arabic-02-2025
- 71.6 rombodawg/Rombos-LLM-V2.5-Qwen-32b
- 71.5
Gemma 3 12B PT
- 71.4
DeepSeek-R1-Distill-Qwen-14B
- 71.4 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned
- 71.3
Gemma 4 26B A4B
- 71.0 mistralai/Codestral-22B-v0.1
- 70.9
Gemma 4 E4B
- 70.9 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16
- 70.5 SicariusSicariiStuff/Hebrew_Nemo
- 70.5 meta-llama/Meta-Llama-3.1-8B-Instruct
- 69.8 mistralai/Ministral-8B-Instruct-2410
- 69.7 SicariusSicariiStuff/Impish_Nemo_12B
- 69.4 ai21labs/AI21-Jamba-1.5-Mini
- 69.3
Aya Expanse 8B
- 69.2
Gemma 2 9B
- 69.1
Command R (03-2024)
- 69.0
SOLAR-10.7B-Instruct-v1.0
- 68.8
Qwen1.5-32B
- 68.5
Mistral-Nemo-Base-2407
- 68.4 MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct
- 68.3 allenai/Olmo-3-1125-32B
- 68.0 dicta-il/dictalm2.0-instruct
- 67.8
Qwen3-4B-Base
- 67.8 dicta-il/dictalm2.0
- 67.7 SicariusSicariiStuff/Zion_Alpha
- 67.7 kekmodel/StopCarbon-10.7B-v5
- 67.5
Qwen2.5-7B
- 67.4
Gemma 7B
- 67.2
Gemma 4 E2B
- 67.1
Qwen3-4B
- 67.0 jeonsworld/CarbonVillain-en-10.7B-v4
- 67.0 invalid-coder/Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp
- 66.4
Gemma 2 2B IT
- 66.4 CohereForAI/aya-23-8B
- 66.4 lightblue/suzume-llama-3-8B-multilingual
- 66.3 mlabonne/NeuralDaredevil-8B-abliterated
- 66.3
Mixtral 8x7B
- 66.2 ai21labs/AI21-Jamba-1.5-Large
- 66.1 nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base
- 65.8
Gemma 1.1 7B IT
- 65.7
Gemma 3 4B PT
- 65.5 vicgalle/CarbonBeagle-11B-truthy
- 65.5 NousResearch/Hermes-3-Llama-3.1-8B
- 65.5 AIDC-AI/Marco-LLM-GLO
- 65.4
Llama 3.1-8B
- 65.3 tiiuae/Falcon-H1-34B-Base
- 65.2 ronigold/dictalm2.0-instruct-fine-tuned
- 65.1 ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew
- 65.0 nvidia/Mistral-NeMo-Minitron-8B-Base
- 64.4
Llama 3-8B
- 64.3 Danielbrdz/Barcenas-Llama3-8b-ORPO
- 64.2 ibm-granite/granite-4.0-micro-base
- 63.9 meta-llama/Meta-Llama-3-8B-Instruct
- 63.7 NousResearch/Hermes-2-Pro-Llama-3-8B
- 63.6 yam-peleg/Hebrew-Mistral-7B
- 63.5 ibm-granite/granite-4.0-h-tiny-base
- 63.2
MAmmoTH2-8B-Plus
- 63.1
Qwen2-7B
- 63.1 uygarkurt/llama-3-merged-linear
- 62.9 nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base
- 62.8 dicta-il/DictaLM-3.0-1.7B-Base
- 62.6
Qwen2.5-3B
- 62.0 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B
- 61.8 DeepMount00/Llama-3-8b-Ita
- 61.8 VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
- 61.4 Qwen/Qwen3-1.7B-Base
- 61.3 yam-peleg/Hebrew-Gemma-11B-Instruct
- 61.3 shadowml/BeagSake-7B
- 60.7 Granite-3.1-8B-Instruct
- 59.9 abacusai/Llama-3-Smaug-8B
- 59.7
Qwen1.5-14B
- 59.5 eldogbbhed/Peagle-9b
- 59.2 ibm-granite/granite-4.0-h-micro-base
- 58.7
Llama 3.2 3B
- 58.7 Bingsu/exaone-3.0-7.8b-it
- 58.7 maywell/EXAONE-3.0-7.8B-Instruct-Llamafied
- 57.9 deepseek-ai/DeepSeek-R1-Distill-Llama-8B
- 57.4 google/gemma-2-2b
- 57.3 unsloth/Phi-3.5-mini-instruct
- 57.0 zhengr/MixTAO-7Bx2-MoE-v8.1
- 56.7 google/gemma-7b-it
- 56.4 mlabonne/NeuralMonarch-7B
- 56.2 tiiuae/Falcon-H1-7B-Base
- 56.1 mlabonne/AlphaMonarch-7B
- 56.0 yam-peleg/Hebrew-Gemma-11B-V2
- 56.0 Granite-3.0-8B-Instruct
- 55.3
Mistral 7B v0.1
- 54.9
Gemma 2B
- 54.8 HuggingFaceTB/SmolLM3-3B
- 54.7
Falcon3-10B-Instruct
- 54.4 Kukedlc/NeuralSynthesis-7B-v0.1
- 54.0 Granite-3.0-8B-Base
- 53.7 yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B
- 52.7
MAmmoTH2-7B-Plus
- 52.5 natong19/Qwen2-7B-Instruct-abliterated
- 52.5
Mistral 7B v0.3
- 52.4
Qwen2.5-1.5B
- 52.4 tiiuae/Falcon3-10B-Base
- 52.1 HuggingFaceTB/SmolLM3-3B-Base
- 51.6 Qwen/Qwen3-0.6B-Base
- 51.5 Granite-3.1-8B-Base
- 51.1 yunconglong/DARE_TIES_13B
- 51.0 01-ai/Yi-1.5-34B-32K
- 49.5
Zephyr 7B Beta
- 49.0
Falcon3-7B-Instruct
- 46.7
Qwen1.5-14B-Chat
- 46.6
Yi-1.5-9B-Chat
- 46.2 shanchen/llama3-8B-slerp-med-chinese
- 45.2 lex-hue/Delexa-7b
- 45.2 royallab/ZephRP-m7b
- 45.1 Granite-3.0-2B-Instruct
- 44.9 Granite-3.0-2B-Base
- 44.7
Qwen2-1.5B
- 44.4
phi-3-mini 3.8B
- 44.1
DeepSeek-R1-Distill-Qwen-7B
- 42.4 haoranxu/ALMA-13B-R
- 41.6 tiiuae/Falcon3-7B-Base
- 40.5 Granite-3.1-2B-Base
- 40.0 meta-llama/Llama-2-7b-hf
- 39.6
Llama 3.2 1B
- 39.5 XythicK/Hebrew-GPT
- 37.6
Qwen2.5-0.5B
- 36.3 upstage/SOLAR-10.7B-v1.0
- 36.0 tiiuae/Falcon-H1-1.5B-Deep-Base
- 34.0 ibm-granite/granite-3.0-3b-a800m-instruct
- 32.2
Qwen2-0.5B
- 32.0 lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half
- 31.6
Falcon3-3B-Instruct
- 30.8 Granite-3.1-3B-A800M-Instruct
- 29.4
Gemma 3 270M
- 29.1 tiiuae/Falcon-H1-1.5B-Base
- 28.8 ibm-granite/granite-4.0-h-350m-base
- 28.7 ibm-granite/granite-3.0-3b-a800m-base
- 26.3 Granite-3.1-3B-A800M-Base
- 25.5
SmolLM2-1.7B
- 24.1 allenai/OLMo-1B-0724-hf
- 23.3 tiiuae/Falcon3-3B-Base
- 23.0 ibm-granite/granite-4.0-350m-base
- 20.6
Falcon3-1B-Instruct
- 19.4 Granite-3.1-1B-A400M-Instruct
- 17.4 nvidia/Llama-3.1-Nemotron-Nano-8B-v1
- 16.6 ibm-granite/granite-3.0-1b-a400m-base
- 16.5
SmolLM-1.7B
- 16.2 Granite-3.1-1B-A400M-Base
- 15.2 tiiuae/Falcon-H1-0.5B-Base
- 10.7 LiquidAI/LFM2-2.6B
- 7.64 LFM2-1.2B
- 1 CohereForAI/c4ai-command-a-03-2025 bfloat16 88.4%
- 2 mistralai/Mistral-Large-Instruct-2407 bfloat16 87.2%
- 3
Qwen2.5-72B bfloat16 87%
- 4
Gemma 4 31B IT bfloat16 86.8%
- 5
Qwen2.5-32B bfloat16 86.5%
- 6 mistralai/Mistral-Small-24B-Instruct-2501 bfloat16 86%
- 7
Llama 3.3 70B Instruct bfloat16 84.4%
- 8
QwQ-32B bfloat16 83.9%
- 9 meta-llama/Llama-3.1-70B-Instruct bfloat16 83.9%
- 10 google/gemma-2-9b-it bfloat16 83.7%
- 11
QwQ-32B-Preview bfloat16 83.6%
- 12
Qwen3-32B float16 83.5%
- 13
Aya Expanse 32B bfloat16 83.4%
- 14
Llama-3.3-Nemotron-Super-49B-v1.5 bfloat16 83.3%
- 15 CohereForAI/c4ai-command-r-plus bfloat16 82.7%
- 16
Llama-3.1-Nemotron-70B-Instruct bfloat16 82.5%
- 17 CohereForAI/c4ai-command-r-plus-08-2024 bfloat16 82.5%
- 18
Qwen3.5-35B-A3B-Base bfloat16 82.1%
- 19
Qwen3-14B float16 82.1%
- 20 nvidia/Nemotron-H-56B-Base-8K bfloat16 81.9%
- 21 Jamba 1.6 Large bfloat16 81.5%
- 22
Qwen2.5-14B bfloat16 81.1%
- 23
Qwen3-14B-Base bfloat16 80.9%
- 24
Llama 3.1-70B bfloat16 80.3%
- 25
Gemma 3 27B bfloat16 80.3%
- 26 UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 bfloat16 79.7%
- 27 CohereForAI/aya-23-35B bfloat16 79.4%
- 28
Qwen3-8B-Base bfloat16 78.9%
- 29 meta-llama/Llama-4-Scout-17B-16E bfloat16 78.8%
- 30
Qwen3-8B float16 78.5%
- 31
Llama 3-70B bfloat16 78.4%
- 32
Gemma 3 27B PT bfloat16 78.3%
- 33 mistralai/Mistral-Small-3.1-24B-Base-2503 bfloat16 77.9%
- 34 dicta-il/DictaLM-3.0-Nemotron-12B-Base bfloat16 77.8%
- 35
Qwen3-30B-A3B-Instruct Jul 2025, float16 77.4%
- 36
DeepSeek-R1-Distill-Qwen-32B bfloat16 76.9%
- 37 dicta-il/DictaLM-3.0-24B-Base bfloat16 76.6%
- 38 mistralai/Mistral-Small-Instruct-2409 bfloat16 76.6%
- 39 c4ai-command-r-08-2024 bfloat16 76.2%
- 40
Qwen3-30B-A3B-Thinking Jul 2025, float16 76.1%
- 41 Jamba 1.6 Mini bfloat16 75.7%
- 42 zai-org/GLM-4.5-Air-Base bfloat16 75.5%
- 43 mistralai/Mistral-Small-24B-Base-2501 bfloat16 75%
- 44
Phi-4 bfloat16 74.7%
- 45 ibm-granite/granite-4.0-h-small-base bfloat16 74.3%
- 46
Gemma 2 27B bfloat16 74.2%
- 47 618AI/dictalm2-it-qa-fine-tune bfloat16 74.1%
- 48 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16 bfloat16 73.8%
- 49 mistralai/Mistral-Nemo-Instruct-2407 bfloat16 73.4%
- 50 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP bfloat16 73%
- 51
Qwen3-30B-A3B-Base bfloat16 73%
- 52 CohereLabs/tiny-aya-base bfloat16 72%
- 53 SeaLLMs/SeaLLM-7B-v2.5 bfloat16 72%
- 54 CohereLabs/c4ai-command-r7b-arabic-02-2025 bfloat16 71.8%
- 55 rombodawg/Rombos-LLM-V2.5-Qwen-32b bfloat16 71.6%
- 56
Gemma 3 12B PT bfloat16 71.5%
- 57
DeepSeek-R1-Distill-Qwen-14B float16 71.4%
- 58 SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned bfloat16 71.4%
- 59
Gemma 4 26B A4B bfloat16 71.3%
- 60 mistralai/Codestral-22B-v0.1 bfloat16 71%
- 61
Gemma 4 E4B bfloat16 70.9%
- 62 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 bfloat16 70.9%
- 63 SicariusSicariiStuff/Hebrew_Nemo bfloat16 70.5%
- 64 meta-llama/Meta-Llama-3.1-8B-Instruct bfloat16 70.5%
- 65 mistralai/Ministral-8B-Instruct-2410 bfloat16 69.8%
- 66 SicariusSicariiStuff/Impish_Nemo_12B bfloat16 69.7%
- 67 ai21labs/AI21-Jamba-1.5-Mini bfloat16 69.4%
- 68
Aya Expanse 8B bfloat16 69.3%
- 69
Gemma 2 9B bfloat16 69.2%
- 70
Command R (03-2024) bfloat16 69.1%
- 71
SOLAR-10.7B-Instruct-v1.0 float16 69%
- 72
Qwen1.5-32B bfloat16 68.8%
- 73
Mistral-Nemo-Base-2407 bfloat16 68.5%
- 74 MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct bfloat16 68.4%
- 75 allenai/Olmo-3-1125-32B bfloat16 68.3%
- 76 dicta-il/dictalm2.0-instruct bfloat16 68%
- 77
Qwen3-4B-Base bfloat16 67.8%
- 78 dicta-il/dictalm2.0 bfloat16 67.8%
- 79 SicariusSicariiStuff/Zion_Alpha bfloat16 67.7%
- 80 kekmodel/StopCarbon-10.7B-v5 float16 67.7%
- 81
Qwen2.5-7B bfloat16 67.5%
- 82
Gemma 7B bfloat16 67.4%
- 83
Gemma 4 E2B bfloat16 67.2%
- 84
Qwen3-4B float16 67.1%
- 85 jeonsworld/CarbonVillain-en-10.7B-v4 float16 67%
- 86 invalid-coder/Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp float16 67%
- 87
Gemma 2 2B IT bfloat16 66.4%
- 88 CohereForAI/aya-23-8B bfloat16 66.4%
- 89 lightblue/suzume-llama-3-8B-multilingual bfloat16 66.4%
- 90 mlabonne/NeuralDaredevil-8B-abliterated float16 66.3%
- 91
Mixtral 8x7B bfloat16 66.3%
- 92 ai21labs/AI21-Jamba-1.5-Large 8bit 66.2%
- 93 nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base bfloat16 66.1%
- 94
Gemma 1.1 7B IT float16 65.8%
- 95
Gemma 3 4B PT bfloat16 65.7%
- 96 vicgalle/CarbonBeagle-11B-truthy float16 65.5%
- 97 NousResearch/Hermes-3-Llama-3.1-8B bfloat16 65.5%
- 98 AIDC-AI/Marco-LLM-GLO bfloat16 65.5%
- 99
Llama 3.1-8B bfloat16 65.4%
- 100 tiiuae/Falcon-H1-34B-Base bfloat16 65.3%
- 101 ronigold/dictalm2.0-instruct-fine-tuned float16 65.2%
- 102 ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew float16 65.1%
- 103 nvidia/Mistral-NeMo-Minitron-8B-Base bfloat16 65%
- 104
Llama 3-8B bfloat16 64.4%
- 105 Danielbrdz/Barcenas-Llama3-8b-ORPO float16 64.3%
- 106 ibm-granite/granite-4.0-micro-base bfloat16 64.2%
- 107 meta-llama/Meta-Llama-3-8B-Instruct bfloat16 63.9%
- 108 NousResearch/Hermes-2-Pro-Llama-3-8B bfloat16 63.7%
- 109 yam-peleg/Hebrew-Mistral-7B bfloat16 63.6%
- 110 ibm-granite/granite-4.0-h-tiny-base bfloat16 63.5%
- 111
MAmmoTH2-8B-Plus bfloat16 63.2%
- 112
Qwen2-7B bfloat16 63.1%
- 113 uygarkurt/llama-3-merged-linear float16 63.1%
- 114 nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base bfloat16 62.9%
- 115 dicta-il/DictaLM-3.0-1.7B-Base bfloat16 62.8%
- 116
Qwen2.5-3B bfloat16 62.6%
- 117 vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B float16 62%
- 118 DeepMount00/Llama-3-8b-Ita bfloat16 61.8%
- 119 VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct bfloat16 61.8%
- 120 Qwen/Qwen3-1.7B-Base bfloat16 61.4%
- 121 yam-peleg/Hebrew-Gemma-11B-Instruct float16 61.3%
- 122 shadowml/BeagSake-7B bfloat16 61.3%
- 123 Granite-3.1-8B-Instruct bfloat16 60.7%
- 124 abacusai/Llama-3-Smaug-8B bfloat16 59.9%
- 125
Qwen1.5-14B bfloat16 59.7%
- 126 eldogbbhed/Peagle-9b float16 59.5%
- 127 ibm-granite/granite-4.0-h-micro-base bfloat16 59.2%
- 128
Llama 3.2 3B bfloat16 58.7%
- 129 Bingsu/exaone-3.0-7.8b-it ? 58.7%
- 130 maywell/EXAONE-3.0-7.8B-Instruct-Llamafied ? 58.7%
- 131 deepseek-ai/DeepSeek-R1-Distill-Llama-8B float16 57.9%
- 132 google/gemma-2-2b bfloat16 57.4%
- 133 unsloth/Phi-3.5-mini-instruct bfloat16 57.3%
- 134 zhengr/MixTAO-7Bx2-MoE-v8.1 bfloat16 57%
- 135 google/gemma-7b-it bfloat16 56.7%
- 136 mlabonne/NeuralMonarch-7B float16 56.4%
- 137 tiiuae/Falcon-H1-7B-Base bfloat16 56.2%
- 138 mlabonne/AlphaMonarch-7B float16 56.1%
- 139 yam-peleg/Hebrew-Gemma-11B-V2 bfloat16 56%
- 140 Granite-3.0-8B-Instruct bfloat16 56%
- 141
Mistral 7B v0.1 bfloat16 55.3%
- 142
Gemma 2B bfloat16 54.9%
- 143 HuggingFaceTB/SmolLM3-3B float16 54.8%
- 144
Falcon3-10B-Instruct bfloat16 54.7%
- 145 Kukedlc/NeuralSynthesis-7B-v0.1 bfloat16 54.4%
- 146 Granite-3.0-8B-Base bfloat16 54%
- 147 yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B bfloat16 53.7%
- 148
MAmmoTH2-7B-Plus bfloat16 52.7%
- 149 natong19/Qwen2-7B-Instruct-abliterated bfloat16 52.5%
- 150
Mistral 7B v0.3 bfloat16 52.5%
- 151
Qwen2.5-1.5B bfloat16 52.4%
- 152 tiiuae/Falcon3-10B-Base bfloat16 52.4%
- 153 HuggingFaceTB/SmolLM3-3B-Base bfloat16 52.1%
- 154 Qwen/Qwen3-0.6B-Base bfloat16 51.6%
- 155 Granite-3.1-8B-Base bfloat16 51.5%
- 156 yunconglong/DARE_TIES_13B bfloat16 51.1%
- 157 01-ai/Yi-1.5-34B-32K bfloat16 51%
- 158
Zephyr 7B Beta bfloat16 49.5%
- 159
Falcon3-7B-Instruct bfloat16 49%
- 160
Qwen1.5-14B-Chat bfloat16 46.7%
- 161
Yi-1.5-9B-Chat bfloat16 46.6%
- 162 shanchen/llama3-8B-slerp-med-chinese bfloat16 46.2%
- 163 lex-hue/Delexa-7b float16 45.2%
- 164 royallab/ZephRP-m7b bfloat16 45.2%
- 165 Granite-3.0-2B-Instruct bfloat16 45.1%
- 166 Granite-3.0-2B-Base bfloat16 44.9%
- 167
Qwen2-1.5B bfloat16 44.7%
- 168
phi-3-mini 3.8B bfloat16 44.4%
- 169
DeepSeek-R1-Distill-Qwen-7B float16 44.1%
- 170 haoranxu/ALMA-13B-R bfloat16 42.4%
- 171 tiiuae/Falcon3-7B-Base bfloat16 41.6%
- 172 Granite-3.1-2B-Base bfloat16 40.5%
- 173 meta-llama/Llama-2-7b-hf bfloat16 40%
- 174
Llama 3.2 1B bfloat16 39.6%
- 175 XythicK/Hebrew-GPT bfloat16 39.5%
- 176
Qwen2.5-0.5B bfloat16 37.6%
- 177 upstage/SOLAR-10.7B-v1.0 float16 36.3%
- 178 tiiuae/Falcon-H1-1.5B-Deep-Base bfloat16 36%
- 179 ibm-granite/granite-3.0-3b-a800m-instruct bfloat16 34%
- 180
Qwen2-0.5B bfloat16 32.2%
- 181 lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half bfloat16 32%
- 182
Falcon3-3B-Instruct bfloat16 31.6%
- 183 Granite-3.1-3B-A800M-Instruct bfloat16 30.8%
- 184
Gemma 3 270M bfloat16 29.4%
- 185 tiiuae/Falcon-H1-1.5B-Base bfloat16 29.1%
- 186 ibm-granite/granite-4.0-h-350m-base bfloat16 28.8%
- 187 ibm-granite/granite-3.0-3b-a800m-base bfloat16 28.7%
- 188 Granite-3.1-3B-A800M-Base bfloat16 26.3%
- 189
SmolLM2-1.7B bfloat16 25.5%
- 190 allenai/OLMo-1B-0724-hf bfloat16 24.1%
- 191 tiiuae/Falcon3-3B-Base bfloat16 23.3%
- 192 ibm-granite/granite-4.0-350m-base bfloat16 23%
- 193
Falcon3-1B-Instruct bfloat16 20.6%
- 194 Granite-3.1-1B-A400M-Instruct bfloat16 19.4%
- 195 nvidia/Llama-3.1-Nemotron-Nano-8B-v1 bfloat16 17.4%
- 196 ibm-granite/granite-3.0-1b-a400m-base bfloat16 16.6%
- 197
SmolLM-1.7B bfloat16 16.5%
- 198 Granite-3.1-1B-A400M-Base bfloat16 16.2%
- 199 tiiuae/Falcon-H1-0.5B-Base bfloat16 15.2%
- 200 LiquidAI/LFM2-2.6B bfloat16 10.7%
- 201 LFM2-1.2B bfloat16 7.64%
Source: Official HeQ leaderboard
- 87.0
Qwen2.5-72B
- 86.8
Gemma 4 31B IT
- 86.5
Qwen2.5-32B
- 84.4
Llama 3.3 70B Instruct
- 83.9
QwQ-32B
- 83.6
QwQ-32B-Preview
- 83.5
Qwen3-32B
- 83.4
Aya Expanse 32B
- 83.3
Llama-3.3-Nemotron-Super-49B-v1.5
- 82.5
Llama-3.1-Nemotron-70B-Instruct
- 82.1
Qwen3.5-35B-A3B-Base
- 82.1
Qwen3-14B
- 81.5 Jamba 1.6 Large
- 81.1
Qwen2.5-14B
- 80.9
Qwen3-14B-Base
- 80.3
Llama 3.1-70B
- 80.3
Gemma 3 27B
- 78.9
Qwen3-8B-Base
- 78.5
Qwen3-8B
- 78.4
Llama 3-70B
- 78.3
Gemma 3 27B PT
- 77.4
Qwen3-30B-A3B-Instruct
- 76.9
DeepSeek-R1-Distill-Qwen-32B
- 76.2 c4ai-command-r-08-2024
- 76.1
Qwen3-30B-A3B-Thinking
- 75.7 Jamba 1.6 Mini
- 74.7
Phi-4
- 74.2
Gemma 2 27B
- 73.0
Qwen3-30B-A3B-Base
- 71.5
Gemma 3 12B PT
- 71.4
DeepSeek-R1-Distill-Qwen-14B
- 71.3
Gemma 4 26B A4B
- 70.9
Gemma 4 E4B
- 69.3
Aya Expanse 8B
- 69.2
Gemma 2 9B
- 69.1
Command R (03-2024)
- 69.0
SOLAR-10.7B-Instruct-v1.0
- 68.8
Qwen1.5-32B
- 68.5
Mistral-Nemo-Base-2407
- 67.8
Qwen3-4B-Base
- 67.5
Qwen2.5-7B
- 67.4
Gemma 7B
- 67.2
Gemma 4 E2B
- 67.1
Qwen3-4B
- 66.4
Gemma 2 2B IT
- 66.3
Mixtral 8x7B
- 65.8
Gemma 1.1 7B IT
- 65.7
Gemma 3 4B PT
- 65.4
Llama 3.1-8B
- 64.4
Llama 3-8B
- 63.2
MAmmoTH2-8B-Plus
- 63.1
Qwen2-7B
- 62.6
Qwen2.5-3B
- 60.7 Granite-3.1-8B-Instruct
- 59.7
Qwen1.5-14B
- 58.7
Llama 3.2 3B
- 56.0 Granite-3.0-8B-Instruct
- 55.3
Mistral 7B v0.1
- 54.9
Gemma 2B
- 54.7
Falcon3-10B-Instruct
- 54.0 Granite-3.0-8B-Base
- 52.7
MAmmoTH2-7B-Plus
- 52.5
Mistral 7B v0.3
- 52.4
Qwen2.5-1.5B
- 51.5 Granite-3.1-8B-Base
- 49.5
Zephyr 7B Beta
- 49.0
Falcon3-7B-Instruct
- 46.7
Qwen1.5-14B-Chat
- 46.6
Yi-1.5-9B-Chat
- 45.1 Granite-3.0-2B-Instruct
- 44.9 Granite-3.0-2B-Base
- 44.7
Qwen2-1.5B
- 44.4
phi-3-mini 3.8B
- 44.1
DeepSeek-R1-Distill-Qwen-7B
- 40.5 Granite-3.1-2B-Base
- 39.6
Llama 3.2 1B
- 37.6
Qwen2.5-0.5B
- 32.2
Qwen2-0.5B
- 31.6
Falcon3-3B-Instruct
- 30.8 Granite-3.1-3B-A800M-Instruct
- 29.4
Gemma 3 270M
- 26.3 Granite-3.1-3B-A800M-Base
- 25.5
SmolLM2-1.7B
- 20.6
Falcon3-1B-Instruct
- 19.4 Granite-3.1-1B-A400M-Instruct
- 16.5
SmolLM-1.7B
- 16.2 Granite-3.1-1B-A400M-Base
- 7.64 LFM2-1.2B
- 1
Qwen2.5-72B bfloat16 87%
- 2
Gemma 4 31B IT bfloat16 86.8%
- 3
Qwen2.5-32B bfloat16 86.5%
- 4
Llama 3.3 70B Instruct bfloat16 84.4%
- 5
QwQ-32B bfloat16 83.9%
- 6
QwQ-32B-Preview bfloat16 83.6%
- 7
Qwen3-32B float16 83.5%
- 8
Aya Expanse 32B bfloat16 83.4%
- 9
Llama-3.3-Nemotron-Super-49B-v1.5 bfloat16 83.3%
- 10
Llama-3.1-Nemotron-70B-Instruct bfloat16 82.5%
- 11
Qwen3.5-35B-A3B-Base bfloat16 82.1%
- 12
Qwen3-14B float16 82.1%
- 13 Jamba 1.6 Large bfloat16 81.5%
- 14
Qwen2.5-14B bfloat16 81.1%
- 15
Qwen3-14B-Base bfloat16 80.9%
- 16
Llama 3.1-70B bfloat16 80.3%
- 17
Gemma 3 27B bfloat16 80.3%
- 18
Qwen3-8B-Base bfloat16 78.9%
- 19
Qwen3-8B float16 78.5%
- 20
Llama 3-70B bfloat16 78.4%
- 21
Gemma 3 27B PT bfloat16 78.3%
- 22
Qwen3-30B-A3B-Instruct Jul 2025, float16 77.4%
- 23
DeepSeek-R1-Distill-Qwen-32B bfloat16 76.9%
- 24 c4ai-command-r-08-2024 bfloat16 76.2%
- 25
Qwen3-30B-A3B-Thinking Jul 2025, float16 76.1%
- 26 Jamba 1.6 Mini bfloat16 75.7%
- 27
Phi-4 bfloat16 74.7%
- 28
Gemma 2 27B bfloat16 74.2%
- 29
Qwen3-30B-A3B-Base bfloat16 73%
- 30
Gemma 3 12B PT bfloat16 71.5%
- 31
DeepSeek-R1-Distill-Qwen-14B float16 71.4%
- 32
Gemma 4 26B A4B bfloat16 71.3%
- 33
Gemma 4 E4B bfloat16 70.9%
- 34
Aya Expanse 8B bfloat16 69.3%
- 35
Gemma 2 9B bfloat16 69.2%
- 36
Command R (03-2024) bfloat16 69.1%
- 37
SOLAR-10.7B-Instruct-v1.0 float16 69%
- 38
Qwen1.5-32B bfloat16 68.8%
- 39
Mistral-Nemo-Base-2407 bfloat16 68.5%
- 40
Qwen3-4B-Base bfloat16 67.8%
- 41
Qwen2.5-7B bfloat16 67.5%
- 42
Gemma 7B bfloat16 67.4%
- 43
Gemma 4 E2B bfloat16 67.2%
- 44
Qwen3-4B float16 67.1%
- 45
Gemma 2 2B IT bfloat16 66.4%
- 46
Mixtral 8x7B bfloat16 66.3%
- 47
Gemma 1.1 7B IT float16 65.8%
- 48
Gemma 3 4B PT bfloat16 65.7%
- 49
Llama 3.1-8B bfloat16 65.4%
- 50
Llama 3-8B bfloat16 64.4%
- 51
MAmmoTH2-8B-Plus bfloat16 63.2%
- 52
Qwen2-7B bfloat16 63.1%
- 53
Qwen2.5-3B bfloat16 62.6%
- 54 Granite-3.1-8B-Instruct bfloat16 60.7%
- 55
Qwen1.5-14B bfloat16 59.7%
- 56
Llama 3.2 3B bfloat16 58.7%
- 57 Granite-3.0-8B-Instruct bfloat16 56%
- 58
Mistral 7B v0.1 bfloat16 55.3%
- 59
Gemma 2B bfloat16 54.9%
- 60
Falcon3-10B-Instruct bfloat16 54.7%
- 61 Granite-3.0-8B-Base bfloat16 54%
- 62
MAmmoTH2-7B-Plus bfloat16 52.7%
- 63
Mistral 7B v0.3 bfloat16 52.5%
- 64
Qwen2.5-1.5B bfloat16 52.4%
- 65 Granite-3.1-8B-Base bfloat16 51.5%
- 66
Zephyr 7B Beta bfloat16 49.5%
- 67
Falcon3-7B-Instruct bfloat16 49%
- 68
Qwen1.5-14B-Chat bfloat16 46.7%
- 69
Yi-1.5-9B-Chat bfloat16 46.6%
- 70 Granite-3.0-2B-Instruct bfloat16 45.1%
- 71 Granite-3.0-2B-Base bfloat16 44.9%
- 72
Qwen2-1.5B bfloat16 44.7%
- 73
phi-3-mini 3.8B bfloat16 44.4%
- 74
DeepSeek-R1-Distill-Qwen-7B float16 44.1%
- 75 Granite-3.1-2B-Base bfloat16 40.5%
- 76
Llama 3.2 1B bfloat16 39.6%
- 77
Qwen2.5-0.5B bfloat16 37.6%
- 78
Qwen2-0.5B bfloat16 32.2%
- 79
Falcon3-3B-Instruct bfloat16 31.6%
- 80 Granite-3.1-3B-A800M-Instruct bfloat16 30.8%
- 81
Gemma 3 270M bfloat16 29.4%
- 82 Granite-3.1-3B-A800M-Base bfloat16 26.3%
- 83
SmolLM2-1.7B bfloat16 25.5%
- 84
Falcon3-1B-Instruct bfloat16 20.6%
- 85 Granite-3.1-1B-A400M-Instruct bfloat16 19.4%
- 86
SmolLM-1.7B bfloat16 16.5%
- 87 Granite-3.1-1B-A400M-Base bfloat16 16.2%
- 88 LFM2-1.2B bfloat16 7.64%
Source: Official HeQ leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | CohereForAI/c4ai-command-a-03-2025 111.1B | 88.4% | bfloat16 | Official leaderboard | |
| 2 | mistralai/Mistral-Large-Instruct-2407 122.6B | 87.2% | bfloat16 | Official leaderboard | |
| 3 | | 87% | bfloat16 | Sep 2024 | Official leaderboard |
| 4 | | 86.8% | bfloat16 | Apr 2026 | Official leaderboard |
| 5 | | 86.5% | bfloat16 | Sep 2024 | Official leaderboard |
| 6 | mistralai/Mistral-Small-24B-Instruct-2501 23.6B | 86% | bfloat16 | Official leaderboard | |
| 7 | | 84.4% | bfloat16 | Dec 2024 | Official leaderboard |
| 8 | | 83.9% | bfloat16 | Mar 2025 | Official leaderboard |
| 9 | meta-llama/Llama-3.1-70B-Instruct 70.6B | 83.9% | bfloat16 | Official leaderboard | |
| 10 | google/gemma-2-9b-it 9.2B | 83.7% | bfloat16 | Official leaderboard | |
| 11 | | 83.6% | bfloat16 | Nov 2024 | Official leaderboard |
| 12 | | 83.5% | float16 | Apr 2025 | Official leaderboard |
| 13 | | 83.4% | bfloat16 | Oct 2024 | Official leaderboard |
| 14 | | 83.3% | bfloat16 | Jul 2025 | Official leaderboard |
| 15 | CohereForAI/c4ai-command-r-plus 103.8B | 82.7% | bfloat16 | Official leaderboard | |
| 16 | | 82.5% | bfloat16 | Oct 2024 | Official leaderboard |
| 17 | CohereForAI/c4ai-command-r-plus-08-2024 103.8B | 82.5% | bfloat16 | Official leaderboard | |
| 18 | | 82.1% | bfloat16 | Feb 2026 | Official leaderboard |
| 19 | | 82.1% | float16 | Apr 2025 | Official leaderboard |
| 20 | nvidia/Nemotron-H-56B-Base-8K 56.3B | 81.9% | bfloat16 | Official leaderboard | |
| 21 | Jamba 1.6 Large | 81.5% | bfloat16 | Mar 2025 | Official leaderboard |
| 22 | | 81.1% | bfloat16 | Sep 2024 | Official leaderboard |
| 23 | | 80.9% | bfloat16 | Apr 2025 | Official leaderboard |
| 24 | | 80.3% | bfloat16 | Jul 2024 | Official leaderboard |
| 25 | | 80.3% | bfloat16 | Mar 2025 | Official leaderboard |
| 26 | UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3 9.2B | 79.7% | bfloat16 | Official leaderboard | |
| 27 | CohereForAI/aya-23-35B 35B | 79.4% | bfloat16 | Official leaderboard | |
| 28 | | 78.9% | bfloat16 | Apr 2025 | Official leaderboard |
| 29 | meta-llama/Llama-4-Scout-17B-16E 108.6B | 78.8% | bfloat16 | Official leaderboard | |
| 30 | | 78.5% | float16 | Apr 2025 | Official leaderboard |
| 31 | | 78.4% | bfloat16 | Apr 2024 | Official leaderboard |
| 32 | | 78.3% | bfloat16 | Mar 2025 | Official leaderboard |
| 33 | mistralai/Mistral-Small-3.1-24B-Base-2503 24B | 77.9% | bfloat16 | Official leaderboard | |
| 34 | dicta-il/DictaLM-3.0-Nemotron-12B-Base 12.3B | 77.8% | bfloat16 | Official leaderboard | |
| 35 | | 77.4% | float16 | Jul 2025 | Official leaderboard |
| 36 | | 76.9% | bfloat16 | Jan 2025 | Official leaderboard |
| 37 | dicta-il/DictaLM-3.0-24B-Base 23.6B | 76.6% | bfloat16 | Official leaderboard | |
| 38 | mistralai/Mistral-Small-Instruct-2409 22.3B | 76.6% | bfloat16 | Official leaderboard | |
| 39 | c4ai-command-r-08-2024 | 76.2% | bfloat16 | Aug 2024 | Official leaderboard |
| 40 | | 76.1% | float16 | Jul 2025 | Official leaderboard |
| 41 | Jamba 1.6 Mini | 75.7% | bfloat16 | Mar 2025 | Official leaderboard |
| 42 | zai-org/GLM-4.5-Air-Base 110.5B | 75.5% | bfloat16 | Official leaderboard | |
| 43 | mistralai/Mistral-Small-24B-Base-2501 23.6B | 75% | bfloat16 | Official leaderboard | |
| 44 | | 74.7% | bfloat16 | Dec 2024 | Official leaderboard |
| 45 | ibm-granite/granite-4.0-h-small-base 32.2B | 74.3% | bfloat16 | Official leaderboard | |
| 46 | | 74.2% | bfloat16 | Jun 2024 | Official leaderboard |
| 47 | 618AI/dictalm2-it-qa-fine-tune 7.3B | 74.1% | bfloat16 | Official leaderboard | |
| 48 | nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-Base-BF16 124B | 73.8% | bfloat16 | Official leaderboard | |
| 49 | mistralai/Mistral-Nemo-Instruct-2407 12.3B | 73.4% | bfloat16 | Official leaderboard | |
| 50 | SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_SLERP 7.2B | 73% | bfloat16 | Official leaderboard | |
| 51 | | 73% | bfloat16 | Apr 2025 | Official leaderboard |
| 52 | CohereLabs/tiny-aya-base 3.4B | 72% | bfloat16 | Official leaderboard | |
| 53 | SeaLLMs/SeaLLM-7B-v2.5 8.5B | 72% | bfloat16 | Official leaderboard | |
| 54 | CohereLabs/c4ai-command-r7b-arabic-02-2025 8B | 71.8% | bfloat16 | Official leaderboard | |
| 55 | rombodawg/Rombos-LLM-V2.5-Qwen-32b 32.8B | 71.6% | bfloat16 | Official leaderboard | |
| 56 | | 71.5% | bfloat16 | Mar 2025 | Official leaderboard |
| 57 | | 71.4% | float16 | Jan 2025 | Official leaderboard |
| 58 | SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned 7.2B | 71.4% | bfloat16 | Official leaderboard | |
| 59 | | 71.3% | bfloat16 | Apr 2026 | Official leaderboard |
| 60 | mistralai/Codestral-22B-v0.1 22.3B | 71% | bfloat16 | Official leaderboard | |
| 61 | | 70.9% | bfloat16 | Mar 2026 | Official leaderboard |
| 62 | nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 31.6B | 70.9% | bfloat16 | Official leaderboard | |
| 63 | SicariusSicariiStuff/Hebrew_Nemo 12.3B | 70.5% | bfloat16 | Official leaderboard | |
| 64 | meta-llama/Meta-Llama-3.1-8B-Instruct 8B | 70.5% | bfloat16 | Official leaderboard | |
| 65 | mistralai/Ministral-8B-Instruct-2410 8B | 69.8% | bfloat16 | Official leaderboard | |
| 66 | SicariusSicariiStuff/Impish_Nemo_12B | 69.7% | bfloat16 | Official leaderboard | |
| 67 | ai21labs/AI21-Jamba-1.5-Mini 51.6B | 69.4% | bfloat16 | Official leaderboard | |
| 68 | | 69.3% | bfloat16 | Oct 2024 | Official leaderboard |
| 69 | | 69.2% | bfloat16 | Jun 2024 | Official leaderboard |
| 70 | | 69.1% | bfloat16 | Mar 2024 | Official leaderboard |
| 71 | | 69% | float16 | Dec 2023 | Official leaderboard |
| 72 | | 68.8% | bfloat16 | Feb 2024 | Official leaderboard |
| 73 | | 68.5% | bfloat16 | Jul 2024 | Official leaderboard |
| 74 | MohamedRashad/Arabic-Orpo-Llama-3-8B-Instruct 8B | 68.4% | bfloat16 | Official leaderboard | |
| 75 | allenai/Olmo-3-1125-32B 32.2B | 68.3% | bfloat16 | Official leaderboard | |
| 76 | dicta-il/dictalm2.0-instruct 7.3B | 68% | bfloat16 | Official leaderboard | |
| 77 | | 67.8% | bfloat16 | Apr 2025 | Official leaderboard |
| 78 | dicta-il/dictalm2.0 7.3B | 67.8% | bfloat16 | Official leaderboard | |
| 79 | SicariusSicariiStuff/Zion_Alpha 7.2B | 67.7% | bfloat16 | Official leaderboard | |
| 80 | kekmodel/StopCarbon-10.7B-v5 10.7B | 67.7% | float16 | Official leaderboard | |
| 81 | | 67.5% | bfloat16 | Sep 2024 | Official leaderboard |
| 82 | | 67.4% | bfloat16 | Feb 2024 | Official leaderboard |
| 83 | | 67.2% | bfloat16 | Mar 2026 | Official leaderboard |
| 84 | | 67.1% | float16 | Apr 2025 | Official leaderboard |
| 85 | jeonsworld/CarbonVillain-en-10.7B-v4 10.7B | 67% | float16 | Official leaderboard | |
| 86 | invalid-coder/Sakura-SOLAR-Instruct-CarbonVillain-en-10.7B-v2-slerp 10.7B | 67% | float16 | Official leaderboard | |
| 87 | | 66.4% | bfloat16 | Jul 2024 | Official leaderboard |
| 88 | CohereForAI/aya-23-8B 8B | 66.4% | bfloat16 | Official leaderboard | |
| 89 | lightblue/suzume-llama-3-8B-multilingual 8B | 66.4% | bfloat16 | Official leaderboard | |
| 90 | mlabonne/NeuralDaredevil-8B-abliterated 8B | 66.3% | float16 | Official leaderboard | |
| 91 | | 66.3% | bfloat16 | Dec 2023 | Official leaderboard |
| 92 | ai21labs/AI21-Jamba-1.5-Large 398.6B | 66.2% | 8bit | Official leaderboard | |
| 93 | nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base 12.3B | 66.1% | bfloat16 | Official leaderboard | |
| 94 | | 65.8% | float16 | Apr 2024 | Official leaderboard |
| 95 | | 65.7% | bfloat16 | Mar 2025 | Official leaderboard |
| 96 | vicgalle/CarbonBeagle-11B-truthy 10.7B | 65.5% | float16 | Official leaderboard | |
| 97 | NousResearch/Hermes-3-Llama-3.1-8B 8B | 65.5% | bfloat16 | Official leaderboard | |
| 98 | AIDC-AI/Marco-LLM-GLO 7.6B | 65.5% | bfloat16 | Official leaderboard | |
| 99 | | 65.4% | bfloat16 | Jul 2024 | Official leaderboard |
| 100 | tiiuae/Falcon-H1-34B-Base 33.6B | 65.3% | bfloat16 | Official leaderboard | |
| 101 | ronigold/dictalm2.0-instruct-fine-tuned 7.3B | 65.2% | float16 | Official leaderboard | |
| 102 | ronigold/dictalm2.0-instruct-fine-tuned-alpaca-gpt4-hebrew 7.3B | 65.1% | float16 | Official leaderboard | |
| 103 | nvidia/Mistral-NeMo-Minitron-8B-Base 8.4B | 65% | bfloat16 | Official leaderboard | |
| 104 | | 64.4% | bfloat16 | Apr 2024 | Official leaderboard |
| 105 | Danielbrdz/Barcenas-Llama3-8b-ORPO 8B | 64.3% | float16 | Official leaderboard | |
| 106 | ibm-granite/granite-4.0-micro-base 3.4B | 64.2% | bfloat16 | Official leaderboard | |
| 107 | meta-llama/Meta-Llama-3-8B-Instruct 8B | 63.9% | bfloat16 | Official leaderboard | |
| 108 | NousResearch/Hermes-2-Pro-Llama-3-8B 8B | 63.7% | bfloat16 | Official leaderboard | |
| 109 | yam-peleg/Hebrew-Mistral-7B 7.5B | 63.6% | bfloat16 | Official leaderboard | |
| 110 | ibm-granite/granite-4.0-h-tiny-base 6.9B | 63.5% | bfloat16 | Official leaderboard | |
| 111 | | 63.2% | bfloat16 | May 2024 | Official leaderboard |
| 112 | | 63.1% | bfloat16 | Jun 2024 | Official leaderboard |
| 113 | uygarkurt/llama-3-merged-linear 8B | 63.1% | float16 | Official leaderboard | |
| 114 | nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base 8.9B | 62.9% | bfloat16 | Official leaderboard | |
| 115 | dicta-il/DictaLM-3.0-1.7B-Base 1.7B | 62.8% | bfloat16 | Official leaderboard | |
| 116 | | 62.6% | bfloat16 | Sep 2024 | Official leaderboard |
| 117 | vicgalle/Configurable-Hermes-2-Pro-Llama-3-8B 8B | 62% | float16 | Official leaderboard | |
| 118 | DeepMount00/Llama-3-8b-Ita 8B | 61.8% | bfloat16 | Official leaderboard | |
| 119 | VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct 8B | 61.8% | bfloat16 | Official leaderboard | |
| 120 | Qwen/Qwen3-1.7B-Base 1.7B | 61.4% | bfloat16 | Official leaderboard | |
| 121 | yam-peleg/Hebrew-Gemma-11B-Instruct 10.5B | 61.3% | float16 | Official leaderboard | |
| 122 | shadowml/BeagSake-7B 7.2B | 61.3% | bfloat16 | Official leaderboard | |
| 123 | Granite-3.1-8B-Instruct | 60.7% | bfloat16 | Dec 2024 | Official leaderboard |
| 124 | abacusai/Llama-3-Smaug-8B 8B | 59.9% | bfloat16 | Official leaderboard | |
| 125 | | 59.7% | bfloat16 | Feb 2024 | Official leaderboard |
| 126 | eldogbbhed/Peagle-9b 9B | 59.5% | float16 | Official leaderboard | |
| 127 | ibm-granite/granite-4.0-h-micro-base 3.2B | 59.2% | bfloat16 | Official leaderboard | |
| 128 | | 58.7% | bfloat16 | Sep 2024 | Official leaderboard |
| 129 | Bingsu/exaone-3.0-7.8b-it | 58.7% | ? | Official leaderboard | |
| 130 | maywell/EXAONE-3.0-7.8B-Instruct-Llamafied | 58.7% | ? | Official leaderboard | |
| 131 | deepseek-ai/DeepSeek-R1-Distill-Llama-8B 8B | 57.9% | float16 | Official leaderboard | |
| 132 | google/gemma-2-2b 2.6B | 57.4% | bfloat16 | Official leaderboard | |
| 133 | unsloth/Phi-3.5-mini-instruct 3.8B | 57.3% | bfloat16 | Official leaderboard | |
| 134 | zhengr/MixTAO-7Bx2-MoE-v8.1 12.9B | 57% | bfloat16 | Official leaderboard | |
| 135 | google/gemma-7b-it 8.5B | 56.7% | bfloat16 | Official leaderboard | |
| 136 | mlabonne/NeuralMonarch-7B 7.2B | 56.4% | float16 | Official leaderboard | |
| 137 | tiiuae/Falcon-H1-7B-Base 7.6B | 56.2% | bfloat16 | Official leaderboard | |
| 138 | mlabonne/AlphaMonarch-7B 7.2B | 56.1% | float16 | Official leaderboard | |
| 139 | yam-peleg/Hebrew-Gemma-11B-V2 10.5B | 56% | bfloat16 | Official leaderboard | |
| 140 | Granite-3.0-8B-Instruct | 56% | bfloat16 | Oct 2024 | Official leaderboard |
| 141 | | 55.3% | bfloat16 | Sep 2023 | Official leaderboard |
| 142 | | 54.9% | bfloat16 | Feb 2024 | Official leaderboard |
| 143 | HuggingFaceTB/SmolLM3-3B 3.1B | 54.8% | float16 | Official leaderboard | |
| 144 | | 54.7% | bfloat16 | Dec 2024 | Official leaderboard |
| 145 | Kukedlc/NeuralSynthesis-7B-v0.1 7.2B | 54.4% | bfloat16 | Official leaderboard | |
| 146 | Granite-3.0-8B-Base | 54% | bfloat16 | Oct 2024 | Official leaderboard |
| 147 | yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B 12.9B | 53.7% | bfloat16 | Official leaderboard | |
| 148 | | 52.7% | bfloat16 | May 2024 | Official leaderboard |
| 149 | natong19/Qwen2-7B-Instruct-abliterated 7.6B | 52.5% | bfloat16 | Official leaderboard | |
| 150 | | 52.5% | bfloat16 | Sep 2023 | Official leaderboard |
| 151 | | 52.4% | bfloat16 | Sep 2024 | Official leaderboard |
| 152 | tiiuae/Falcon3-10B-Base 10.3B | 52.4% | bfloat16 | Official leaderboard | |
| 153 | HuggingFaceTB/SmolLM3-3B-Base 3.1B | 52.1% | bfloat16 | Official leaderboard | |
| 154 | Qwen/Qwen3-0.6B-Base 0.6B | 51.6% | bfloat16 | Official leaderboard | |
| 155 | Granite-3.1-8B-Base | 51.5% | bfloat16 | Dec 2024 | Official leaderboard |
| 156 | yunconglong/DARE_TIES_13B 12.9B | 51.1% | bfloat16 | Official leaderboard | |
| 157 | 01-ai/Yi-1.5-34B-32K 34.4B | 51% | bfloat16 | Official leaderboard | |
| 158 | | 49.5% | bfloat16 | Oct 2023 | Official leaderboard |
| 159 | | 49% | bfloat16 | Dec 2024 | Official leaderboard |
| 160 | | 46.7% | bfloat16 | Feb 2024 | Official leaderboard |
| 161 | | 46.6% | bfloat16 | May 2024 | Official leaderboard |
| 162 | shanchen/llama3-8B-slerp-med-chinese 8B | 46.2% | bfloat16 | Official leaderboard | |
| 163 | lex-hue/Delexa-7b 7.2B | 45.2% | float16 | Official leaderboard | |
| 164 | royallab/ZephRP-m7b 7.2B | 45.2% | bfloat16 | Official leaderboard | |
| 165 | Granite-3.0-2B-Instruct | 45.1% | bfloat16 | Oct 2024 | Official leaderboard |
| 166 | Granite-3.0-2B-Base | 44.9% | bfloat16 | Oct 2024 | Official leaderboard |
| 167 | | 44.7% | bfloat16 | Jun 2024 | Official leaderboard |
| 168 | | 44.4% | bfloat16 | Apr 2024 | Official leaderboard |
| 169 | | 44.1% | float16 | Jan 2025 | Official leaderboard |
| 170 | haoranxu/ALMA-13B-R 13B | 42.4% | bfloat16 | Official leaderboard | |
| 171 | tiiuae/Falcon3-7B-Base 7.5B | 41.6% | bfloat16 | Official leaderboard | |
| 172 | Granite-3.1-2B-Base | 40.5% | bfloat16 | Dec 2024 | Official leaderboard |
| 173 | meta-llama/Llama-2-7b-hf 6.7B | 40% | bfloat16 | Official leaderboard | |
| 174 | | 39.6% | bfloat16 | Sep 2024 | Official leaderboard |
| 175 | XythicK/Hebrew-GPT 1.2B | 39.5% | bfloat16 | Official leaderboard | |
| 176 | | 37.6% | bfloat16 | Sep 2024 | Official leaderboard |
| 177 | upstage/SOLAR-10.7B-v1.0 10.7B | 36.3% | float16 | Official leaderboard | |
| 178 | tiiuae/Falcon-H1-1.5B-Deep-Base 1.6B | 36% | bfloat16 | Official leaderboard | |
| 179 | ibm-granite/granite-3.0-3b-a800m-instruct 3.4B | 34% | bfloat16 | Official leaderboard | |
| 180 | | 32.2% | bfloat16 | Jun 2024 | Official leaderboard |
| 181 | lightblue/suzume-llama-3-8B-multilingual-orpo-borda-half 8B | 32% | bfloat16 | Official leaderboard | |
| 182 | | 31.6% | bfloat16 | Dec 2024 | Official leaderboard |
| 183 | Granite-3.1-3B-A800M-Instruct | 30.8% | bfloat16 | Dec 2024 | Official leaderboard |
| 184 | | 29.4% | bfloat16 | Aug 2025 | Official leaderboard |
| 185 | tiiuae/Falcon-H1-1.5B-Base 1.6B | 29.1% | bfloat16 | Official leaderboard | |
| 186 | ibm-granite/granite-4.0-h-350m-base 0.3B | 28.8% | bfloat16 | Official leaderboard | |
| 187 | ibm-granite/granite-3.0-3b-a800m-base 3.4B | 28.7% | bfloat16 | Official leaderboard | |
| 188 | Granite-3.1-3B-A800M-Base | 26.3% | bfloat16 | Dec 2024 | Official leaderboard |
| 189 | | 25.5% | bfloat16 | Oct 2024 | Official leaderboard |
| 190 | allenai/OLMo-1B-0724-hf 1.3B | 24.1% | bfloat16 | Official leaderboard | |
| 191 | tiiuae/Falcon3-3B-Base 3.2B | 23.3% | bfloat16 | Official leaderboard | |
| 192 | ibm-granite/granite-4.0-350m-base 0.4B | 23% | bfloat16 | Official leaderboard | |
| 193 | | 20.6% | bfloat16 | Dec 2024 | Official leaderboard |
| 194 | Granite-3.1-1B-A400M-Instruct | 19.4% | bfloat16 | Dec 2024 | Official leaderboard |
| 195 | nvidia/Llama-3.1-Nemotron-Nano-8B-v1 8B | 17.4% | bfloat16 | Official leaderboard | |
| 196 | ibm-granite/granite-3.0-1b-a400m-base 1.4B | 16.6% | bfloat16 | Official leaderboard | |
| 197 | | 16.5% | bfloat16 | Jul 2024 | Official leaderboard |
| 198 | Granite-3.1-1B-A400M-Base | 16.2% | bfloat16 | Dec 2024 | Official leaderboard |
| 199 | tiiuae/Falcon-H1-0.5B-Base 0.5B | 15.2% | bfloat16 | Official leaderboard | |
| 200 | LiquidAI/LFM2-2.6B 2.6B | 10.7% | bfloat16 | Official leaderboard | |
| 201 | LFM2-1.2B | 7.64% | bfloat16 | Jul 2025 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 87% | bfloat16 | Sep 2024 | Official leaderboard |
| 2 | | 86.8% | bfloat16 | Apr 2026 | Official leaderboard |
| 3 | | 86.5% | bfloat16 | Sep 2024 | Official leaderboard |
| 4 | | 84.4% | bfloat16 | Dec 2024 | Official leaderboard |
| 5 | | 83.9% | bfloat16 | Mar 2025 | Official leaderboard |
| 6 | | 83.6% | bfloat16 | Nov 2024 | Official leaderboard |
| 7 | | 83.5% | float16 | Apr 2025 | Official leaderboard |
| 8 | | 83.4% | bfloat16 | Oct 2024 | Official leaderboard |
| 9 | | 83.3% | bfloat16 | Jul 2025 | Official leaderboard |
| 10 | | 82.5% | bfloat16 | Oct 2024 | Official leaderboard |
| 11 | | 82.1% | bfloat16 | Feb 2026 | Official leaderboard |
| 12 | | 82.1% | float16 | Apr 2025 | Official leaderboard |
| 13 | Jamba 1.6 Large | 81.5% | bfloat16 | Mar 2025 | Official leaderboard |
| 14 | | 81.1% | bfloat16 | Sep 2024 | Official leaderboard |
| 15 | | 80.9% | bfloat16 | Apr 2025 | Official leaderboard |
| 16 | | 80.3% | bfloat16 | Jul 2024 | Official leaderboard |
| 17 | | 80.3% | bfloat16 | Mar 2025 | Official leaderboard |
| 18 | | 78.9% | bfloat16 | Apr 2025 | Official leaderboard |
| 19 | | 78.5% | float16 | Apr 2025 | Official leaderboard |
| 20 | | 78.4% | bfloat16 | Apr 2024 | Official leaderboard |
| 21 | | 78.3% | bfloat16 | Mar 2025 | Official leaderboard |
| 22 | | 77.4% | float16 | Jul 2025 | Official leaderboard |
| 23 | | 76.9% | bfloat16 | Jan 2025 | Official leaderboard |
| 24 | c4ai-command-r-08-2024 | 76.2% | bfloat16 | Aug 2024 | Official leaderboard |
| 25 | | 76.1% | float16 | Jul 2025 | Official leaderboard |
| 26 | Jamba 1.6 Mini | 75.7% | bfloat16 | Mar 2025 | Official leaderboard |
| 27 | | 74.7% | bfloat16 | Dec 2024 | Official leaderboard |
| 28 | | 74.2% | bfloat16 | Jun 2024 | Official leaderboard |
| 29 | | 73% | bfloat16 | Apr 2025 | Official leaderboard |
| 30 | | 71.5% | bfloat16 | Mar 2025 | Official leaderboard |
| 31 | | 71.4% | float16 | Jan 2025 | Official leaderboard |
| 32 | | 71.3% | bfloat16 | Apr 2026 | Official leaderboard |
| 33 | | 70.9% | bfloat16 | Mar 2026 | Official leaderboard |
| 34 | | 69.3% | bfloat16 | Oct 2024 | Official leaderboard |
| 35 | | 69.2% | bfloat16 | Jun 2024 | Official leaderboard |
| 36 | | 69.1% | bfloat16 | Mar 2024 | Official leaderboard |
| 37 | | 69% | float16 | Dec 2023 | Official leaderboard |
| 38 | | 68.8% | bfloat16 | Feb 2024 | Official leaderboard |
| 39 | | 68.5% | bfloat16 | Jul 2024 | Official leaderboard |
| 40 | | 67.8% | bfloat16 | Apr 2025 | Official leaderboard |
| 41 | | 67.5% | bfloat16 | Sep 2024 | Official leaderboard |
| 42 | | 67.4% | bfloat16 | Feb 2024 | Official leaderboard |
| 43 | | 67.2% | bfloat16 | Mar 2026 | Official leaderboard |
| 44 | | 67.1% | float16 | Apr 2025 | Official leaderboard |
| 45 | | 66.4% | bfloat16 | Jul 2024 | Official leaderboard |
| 46 | | 66.3% | bfloat16 | Dec 2023 | Official leaderboard |
| 47 | | 65.8% | float16 | Apr 2024 | Official leaderboard |
| 48 | | 65.7% | bfloat16 | Mar 2025 | Official leaderboard |
| 49 | | 65.4% | bfloat16 | Jul 2024 | Official leaderboard |
| 50 | | 64.4% | bfloat16 | Apr 2024 | Official leaderboard |
| 51 | | 63.2% | bfloat16 | May 2024 | Official leaderboard |
| 52 | | 63.1% | bfloat16 | Jun 2024 | Official leaderboard |
| 53 | | 62.6% | bfloat16 | Sep 2024 | Official leaderboard |
| 54 | Granite-3.1-8B-Instruct | 60.7% | bfloat16 | Dec 2024 | Official leaderboard |
| 55 | | 59.7% | bfloat16 | Feb 2024 | Official leaderboard |
| 56 | | 58.7% | bfloat16 | Sep 2024 | Official leaderboard |
| 57 | Granite-3.0-8B-Instruct | 56% | bfloat16 | Oct 2024 | Official leaderboard |
| 58 | | 55.3% | bfloat16 | Sep 2023 | Official leaderboard |
| 59 | | 54.9% | bfloat16 | Feb 2024 | Official leaderboard |
| 60 | | 54.7% | bfloat16 | Dec 2024 | Official leaderboard |
| 61 | Granite-3.0-8B-Base | 54% | bfloat16 | Oct 2024 | Official leaderboard |
| 62 | | 52.7% | bfloat16 | May 2024 | Official leaderboard |
| 63 | | 52.5% | bfloat16 | Sep 2023 | Official leaderboard |
| 64 | | 52.4% | bfloat16 | Sep 2024 | Official leaderboard |
| 65 | Granite-3.1-8B-Base | 51.5% | bfloat16 | Dec 2024 | Official leaderboard |
| 66 | | 49.5% | bfloat16 | Oct 2023 | Official leaderboard |
| 67 | | 49% | bfloat16 | Dec 2024 | Official leaderboard |
| 68 | | 46.7% | bfloat16 | Feb 2024 | Official leaderboard |
| 69 | | 46.6% | bfloat16 | May 2024 | Official leaderboard |
| 70 | Granite-3.0-2B-Instruct | 45.1% | bfloat16 | Oct 2024 | Official leaderboard |
| 71 | Granite-3.0-2B-Base | 44.9% | bfloat16 | Oct 2024 | Official leaderboard |
| 72 | | 44.7% | bfloat16 | Jun 2024 | Official leaderboard |
| 73 | | 44.4% | bfloat16 | Apr 2024 | Official leaderboard |
| 74 | | 44.1% | float16 | Jan 2025 | Official leaderboard |
| 75 | Granite-3.1-2B-Base | 40.5% | bfloat16 | Dec 2024 | Official leaderboard |
| 76 | | 39.6% | bfloat16 | Sep 2024 | Official leaderboard |
| 77 | | 37.6% | bfloat16 | Sep 2024 | Official leaderboard |
| 78 | | 32.2% | bfloat16 | Jun 2024 | Official leaderboard |
| 79 | | 31.6% | bfloat16 | Dec 2024 | Official leaderboard |
| 80 | Granite-3.1-3B-A800M-Instruct | 30.8% | bfloat16 | Dec 2024 | Official leaderboard |
| 81 | | 29.4% | bfloat16 | Aug 2025 | Official leaderboard |
| 82 | Granite-3.1-3B-A800M-Base | 26.3% | bfloat16 | Dec 2024 | Official leaderboard |
| 83 | | 25.5% | bfloat16 | Oct 2024 | Official leaderboard |
| 84 | | 20.6% | bfloat16 | Dec 2024 | Official leaderboard |
| 85 | Granite-3.1-1B-A400M-Instruct | 19.4% | bfloat16 | Dec 2024 | Official leaderboard |
| 86 | | 16.5% | bfloat16 | Jul 2024 | Official leaderboard |
| 87 | Granite-3.1-1B-A400M-Base | 16.2% | bfloat16 | Dec 2024 | Official leaderboard |
| 88 | LFM2-1.2B | 7.64% | bfloat16 | Jul 2025 | Official leaderboard |
8 models have held number one on the official leaderboard since its release in May 2023
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official HeQ leaderboard
Qwen2.5-32B on the official leaderboard passed 85% in September 2024
That was 16 months after its release. By the same age, 25% of the benchmarks from 2023 had passed 85%. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official HeQ leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
All 8 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
- Pass Independent runs Scores from the official leaderboard (201 entries).
- Pass Models scored 88 models we track, plus 113 other entries on its leaderboard.
- Note Newest score The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B. That’s expected for a saturated test.
- Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemma 4 31B IT (#101).
- Note Headroom Saturated. The best is 87%, past 85% since September 2024.
Leaderboard import 3 pass
- Pass Importer We read the board on huggingface.co: 89 rows for models we track and 113 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.