Benchmark Almanac

Alyah

falcon-h1-arabic-7b-instruct 82.2% tops its leaderboard Gemma 3 27B 74.7% leads the models we track
Falcon-Emirati-7B 84.8% leads since October 2026
In use

Models answer 1,173 four-option questions written by native Emirati speakers on greetings, idioms, figurative language, heritage and poetry in Emirati Arabic dialect.

The release post evaluates 54 models (23 base, 31 instruction-tuned). LLMs generated the distractors and humans reviewed them. TII refers to an Alyah leaderboard.

Released
January 2026
Kind
Knowledge in many languages
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Strongest model tested
Qwen3-14B #138 on the capability index
Last in a launch post
October 2026
Last model added
October 2026
Scores refreshed
November 12th, 2026
53
Entries on its own leaderboard, 10 models we track
1
Launch post lists it, from 1 lab

falcon-h1-arabic-7b-instruct tops the leaderboard with 82.2%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Gemma 3 27B, at 74.7%.

Falcon-Emirati-7B leads with 84.8%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in May 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

53 entries. Scroll sideways for the rest.

  1. 82.2 falcon-h1-arabic-7b-instruct
  2. 77.2 humain-ai/ALLaM-7B-Instruct-preview
  3. 74.7 google/gemma-3-27b-pt (base)
  4. 74.7 Gemma 3 27B
  5. 74.6 Qwen2.5-72B
  6. 74.5 falcon-h1-arabic-3b-instruct
  7. 73.7 tiiuae/Falcon-H1-34B-Base (base)
  8. 73.7 Aya Expanse 32B
  9. 73.3 Navid-AI/Yehia-7B-preview
  10. 72.8 FreedomIntelligence/AceGPT-v2-32B-Chat
  11. 71.6 Qwen/Qwen2.5-32B-Instruct
  12. 71.1 Falcon-H1-34B
  13. 69.7 Llama 3.3 70B Instruct
  14. 69.2 QCRI/Fanar-1-9B-Instruct
  15. 67.3 FreedomIntelligence/AceGPT-v2-32B (base)
  16. 65.1 tiiuae/Falcon-H1-7B-Instruct
  17. 64.5 CohereForAI/c4ai-command-r7b-arabic-02-2025
  18. 63.9 silma-ai/SILMA-9B-Instruct-v1.0
  19. 63.4 FreedomIntelligence/AceGPT-v2-8B-Chat
  20. 63.2 google/gemma-3-4b-pt (base)
  21. 62.8 QCRI/Fanar-1-9B (base)
  22. 61.2 Aya Expanse 8B
  23. 61.1 yasserrmd/kallamni-2.6b-v1
  24. 60.8 tiiuae/Falcon-H1-7B-Base (base)
  25. 60.7 yasserrmd/kallamni-4b-v1
  26. 58.6 microsoft/Phi-4-mini-instruct
  27. 58.2 meta-llama/Llama-3.1-8B (base)
  28. 57.3 Qwen/Qwen3-14B-Base (base)
  29. 57.1 tiiuae/Falcon-H1-3B-Instruct
  30. 56.0 inceptionai/jais-adapted-13b (base)
  31. 53.0 Qwen/Qwen2.5-32B (base)
  32. 50.8 FreedomIntelligence/AceGPT-13B (base)
  33. 48.5 silma-ai/SILMA-Kashif-2B-Instruct-v1.0
  34. 47.9 Qwen/Qwen2.5-72B (base)
  35. 46.8 Qwen/Qwen2.5-14B (base)
  36. 46.3 meta-llama/Llama-3.1-8B-Instruct
  37. 46.1 google/gemma-3-4b-it
  38. 45.4 Qwen/Qwen2.5-7B-Instruct
  39. 41.9 google/gemma-2-2b (base)
  40. 41.4 tiiuae/Falcon3-7B-Base (base)
  41. 40.8 Qwen/Qwen3-8B-Base (base)
  42. 40.4 tiiuae/Falcon-H1-3B-Base (base)
  43. 39.6 meta-llama/Llama-3.2-3B-Instruct
  44. 37.8 yasserrmd/kallamni-1.2b-v1
  45. 36.6 Qwen/Qwen2.5-7B (base)
  46. 35.3 Qwen/Qwen2.5-3B (base)
  47. 35.1 meta-llama/Llama-3.2-3B (base)
  48. 33.5 inceptionai/jais-adapted-7b (base)
  49. 27.4 Qwen/Qwen3-4B-Base (base)
  50. 26.3 Qwen3-4B
  51. 26.0 Gemma 2 2B IT
  52. 26.0 Qwen3-14B
  53. 25.7 Qwen3-8B
  1. 1 falcon-h1-arabic-7b-instruct 82.2%
  2. 2 humain-ai/ALLaM-7B-Instruct-preview 77.2%
  3. 3 google/gemma-3-27b-pt (base) 74.7%
  4. 4 Gemma 3 27B 74.7%
  5. 5 Qwen2.5-72B 74.6%
  6. 6 falcon-h1-arabic-3b-instruct 74.5%
  7. 7 tiiuae/Falcon-H1-34B-Base (base) 73.7%
  8. 8 Aya Expanse 32B 73.7%
  9. 9 Navid-AI/Yehia-7B-preview 73.3%
  10. 10 FreedomIntelligence/AceGPT-v2-32B-Chat 72.8%
  11. 11 Qwen/Qwen2.5-32B-Instruct 71.6%
  12. 12 Falcon-H1-34B 71.1%
  13. 13 Llama 3.3 70B Instruct 69.7%
  14. 14 QCRI/Fanar-1-9B-Instruct 69.2%
  15. 15 FreedomIntelligence/AceGPT-v2-32B (base) 67.3%
  16. 16 tiiuae/Falcon-H1-7B-Instruct 65.1%
  17. 17 CohereForAI/c4ai-command-r7b-arabic-02-2025 64.5%
  18. 18 silma-ai/SILMA-9B-Instruct-v1.0 63.9%
  19. 19 FreedomIntelligence/AceGPT-v2-8B-Chat 63.4%
  20. 20 google/gemma-3-4b-pt (base) 63.2%
  21. 21 QCRI/Fanar-1-9B (base) 62.8%
  22. 22 Aya Expanse 8B 61.2%
  23. 23 yasserrmd/kallamni-2.6b-v1 61.1%
  24. 24 tiiuae/Falcon-H1-7B-Base (base) 60.8%
  25. 25 yasserrmd/kallamni-4b-v1 60.7%
  26. 26 microsoft/Phi-4-mini-instruct 58.6%
  27. 27 meta-llama/Llama-3.1-8B (base) 58.2%
  28. 28 Qwen/Qwen3-14B-Base (base) 57.3%
  29. 29 tiiuae/Falcon-H1-3B-Instruct 57.1%
  30. 30 inceptionai/jais-adapted-13b (base) 56%
  31. 31 Qwen/Qwen2.5-32B (base) 53%
  32. 32 FreedomIntelligence/AceGPT-13B (base) 50.8%
  33. 33 silma-ai/SILMA-Kashif-2B-Instruct-v1.0 48.5%
  34. 34 Qwen/Qwen2.5-72B (base) 47.9%
  35. 35 Qwen/Qwen2.5-14B (base) 46.8%
  36. 36 meta-llama/Llama-3.1-8B-Instruct 46.3%
  37. 37 google/gemma-3-4b-it 46.1%
  38. 38 Qwen/Qwen2.5-7B-Instruct 45.4%
  39. 39 google/gemma-2-2b (base) 41.9%
  40. 40 tiiuae/Falcon3-7B-Base (base) 41.4%
  41. 41 Qwen/Qwen3-8B-Base (base) 40.8%
  42. 42 tiiuae/Falcon-H1-3B-Base (base) 40.4%
  43. 43 meta-llama/Llama-3.2-3B-Instruct 39.6%
  44. 44 yasserrmd/kallamni-1.2b-v1 37.8%
  45. 45 Qwen/Qwen2.5-7B (base) 36.6%
  46. 46 Qwen/Qwen2.5-3B (base) 35.3%
  47. 47 meta-llama/Llama-3.2-3B (base) 35.1%
  48. 48 inceptionai/jais-adapted-7b (base) 33.5%
  49. 49 Qwen/Qwen3-4B-Base (base) 27.4%
  50. 50 Qwen3-4B 26.3%
  51. 51 Gemma 2 2B IT 26%
  52. 52 Qwen3-14B 26%
  53. 53 Qwen3-8B 25.7%

Source: Official Alyah leaderboard

Sources: Labs’ model cards and official Alyah leaderboard

#EntryScoreSettingDateSource
1 falcon-h1-arabic-7b-instruct 82.2% Official leaderboard
2 humain-ai/ALLaM-7B-Instruct-preview 77.2% Official leaderboard
3 google/gemma-3-27b-pt (base) 74.7% Official leaderboard
4 Gemma 3 27B 74.7% Mar 2025 Official leaderboard
5 Qwen2.5-72B 74.6% Sep 2024 Official leaderboard
6 falcon-h1-arabic-3b-instruct 74.5% Official leaderboard
7 tiiuae/Falcon-H1-34B-Base (base) 73.7% Official leaderboard
8 Aya Expanse 32B 73.7% Oct 2024 Official leaderboard
9 Navid-AI/Yehia-7B-preview 73.3% Official leaderboard
10 FreedomIntelligence/AceGPT-v2-32B-Chat 72.8% Official leaderboard
11 Qwen/Qwen2.5-32B-Instruct 71.6% Official leaderboard
12 Falcon-H1-34B 71.1% May 2025 Official leaderboard
13 Llama 3.3 70B Instruct 69.7% Dec 2024 Official leaderboard
14 QCRI/Fanar-1-9B-Instruct 69.2% Official leaderboard
15 FreedomIntelligence/AceGPT-v2-32B (base) 67.3% Official leaderboard
16 tiiuae/Falcon-H1-7B-Instruct 65.1% Official leaderboard
17 CohereForAI/c4ai-command-r7b-arabic-02-2025 64.5% Official leaderboard
18 silma-ai/SILMA-9B-Instruct-v1.0 63.9% Official leaderboard
19 FreedomIntelligence/AceGPT-v2-8B-Chat 63.4% Official leaderboard
20 google/gemma-3-4b-pt (base) 63.2% Official leaderboard
21 QCRI/Fanar-1-9B (base) 62.8% Official leaderboard
22 Aya Expanse 8B 61.2% Oct 2024 Official leaderboard
23 yasserrmd/kallamni-2.6b-v1 61.1% Official leaderboard
24 tiiuae/Falcon-H1-7B-Base (base) 60.8% Official leaderboard
25 yasserrmd/kallamni-4b-v1 60.7% Official leaderboard
26 microsoft/Phi-4-mini-instruct 58.6% Official leaderboard
27 meta-llama/Llama-3.1-8B (base) 58.2% Official leaderboard
28 Qwen/Qwen3-14B-Base (base) 57.3% Official leaderboard
29 tiiuae/Falcon-H1-3B-Instruct 57.1% Official leaderboard
30 inceptionai/jais-adapted-13b (base) 56% Official leaderboard
31 Qwen/Qwen2.5-32B (base) 53% Official leaderboard
32 FreedomIntelligence/AceGPT-13B (base) 50.8% Official leaderboard
33 silma-ai/SILMA-Kashif-2B-Instruct-v1.0 48.5% Official leaderboard
34 Qwen/Qwen2.5-72B (base) 47.9% Official leaderboard
35 Qwen/Qwen2.5-14B (base) 46.8% Official leaderboard
36 meta-llama/Llama-3.1-8B-Instruct 46.3% Official leaderboard
37 google/gemma-3-4b-it 46.1% Official leaderboard
38 Qwen/Qwen2.5-7B-Instruct 45.4% Official leaderboard
39 google/gemma-2-2b (base) 41.9% Official leaderboard
40 tiiuae/Falcon3-7B-Base (base) 41.4% Official leaderboard
41 Qwen/Qwen3-8B-Base (base) 40.8% Official leaderboard
42 tiiuae/Falcon-H1-3B-Base (base) 40.4% Official leaderboard
43 meta-llama/Llama-3.2-3B-Instruct 39.6% Official leaderboard
44 yasserrmd/kallamni-1.2b-v1 37.8% Official leaderboard
45 Qwen/Qwen2.5-7B (base) 36.6% Official leaderboard
46 Qwen/Qwen2.5-3B (base) 35.3% Official leaderboard
47 meta-llama/Llama-3.2-3B (base) 35.1% Official leaderboard
48 inceptionai/jais-adapted-7b (base) 33.5% Official leaderboard
49 Qwen/Qwen3-4B-Base (base) 27.4% Official leaderboard
50 Qwen3-4B 26.3% Apr 2025 Official leaderboard
51 Gemma 2 2B IT 26% Jul 2024 Official leaderboard
52 Qwen3-14B 26% Apr 2025 Official leaderboard
53 Qwen3-8B 25.7% Apr 2025 Official leaderboard
#ModelScoreSettingDateSource
1 Falcon-Emirati-7B 84.8% Oct 2026 Lab model card
2 Gemma 3 27B 74.7% Mar 2025 Official leaderboard
3 Qwen2.5-72B 74.6% Sep 2024 Official leaderboard
4 Aya Expanse 32B 73.7% Oct 2024 Official leaderboard
5 Falcon-H1-34B 71.1% May 2025 Official leaderboard
6 Llama 3.3 70B Instruct 69.7% Dec 2024 Official leaderboard
7 Aya Expanse 8B 61.2% Oct 2024 Official leaderboard
8 Qwen3-4B 26.3% Apr 2025 Official leaderboard
9 Gemma 2 2B IT 26% Jul 2024 Official leaderboard
10 Qwen3-14B 26% Apr 2025 Official leaderboard
11 Qwen3-8B 25.7% Apr 2025 Official leaderboard

Labs report scores for 1 model in their cards. Switch the scores above to rank them together.

Number one over time 2 models have held number one with claims included since its release in January 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind
Own leaderboard Not tested since Jan 2026
Gemma 3 27B 74.7%

Source: Official Alyah leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Falcon-Emirati-7B Gemma 3 27B Qwen2.5-72B Gemma 2 2B IT 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Falcon-Emirati-7B Gemma 3 27B

Sources: Official Alyah leaderboard and labs’ model cards

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Technology Innovation Institute
DateLaunch postScoreAs the post writes it
Oct 6, 2026 Falcon-Emirati-7B 84.8 Alyah accuracy (%)

More in Multilingual

All 120 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Nemotron 3 Ultra 550B-A55B Base 90.1%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%