Benchmark Almanac

MLogiQA

Qwen3-235B-A22B 77.1% leads since April 2025
Claude 3.7 Sonnet 67.1% leads since February 2025
Qwen3-235B-A22B 77.1% leads since April 2025
Dropped

Models answer LogiQA logical reasoning questions translated into 10 languages, 80 per language. Scored by accuracy.

It is a subset of the P-MMEval multilingual suite.

Released
November 2024
Kind
Reasoning in many languages
Level
University
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Strongest model tested
Claude 3.7 Sonnet #118 on the capability index
Last in a launch post
April 2025, dropped before it saturated
Last model added
September 2025
Scores refreshed
November 12th, 2026
8
Models scored in lab cards
1
Launch post lists it, from 1 lab
8
System cards report it, from 2 labs

Qwen3-235B-A22B leads with 77.1% among models released by September 2025

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Claude 3.7 Sonnet leads with 67.1% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Qwen3-235B-A22B leads with 77.1%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

The newest model with a score here came out in September 2025, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 77.1 Qwen3-235B-A22B
  2. 76.3 Qwen3-32B
  3. 71.1 Qwen3-14B
  4. 70.1 Qwen3-30B-A3B
  5. 69.0 Qwen3-8B
  6. 65.9 Qwen3-4B
  7. 56.0 Qwen3-1.7B
  8. 40.5 Apertus 70B
  1. 1 Qwen3-235B-A22B 77.1%
  2. 2 Qwen3-32B 76.3%
  3. 3 Qwen3-14B 71.1%
  4. 4 Qwen3-30B-A3B 70.1%
  5. 5 Qwen3-8B 69%
  6. 6 Qwen3-4B 65.9%
  7. 7 Qwen3-1.7B 56%
  8. 8 Apertus 70B 40.5%

Source: Labs’ model cards

Source: Official MLogiQA leaderboard

25 models. Scroll sideways for the rest.

  1. 77.1 Qwen3-235B-A22B
  2. 76.3 Qwen3-32B
  3. 71.1 Qwen3-14B
  4. 70.1 Qwen3-30B-A3B
  5. 69.0 Qwen3-8B
  6. 67.1 Claude 3.7 Sonnet
  7. 65.9 Qwen3-4B
  8. 58.4 Qwen2.5-72B
  9. 57.4 Qwen2.5-32B
  10. 56.0 Qwen3-1.7B
  11. 53.8 Aya Expanse 32B
  12. 53.5 Qwen2.5-14B
  13. 52.4 Llama 3.1-70B
  14. 50.5 Gemma 2 27B
  15. 45.9 Qwen2.5-7B
  16. 41.8 Mistral NeMo
  17. 41.5 Gemma 2 9B
  18. 40.5 Apertus 70B
  19. 39.9 Llama 3.1-8B
  20. 39.4 Qwen2.5-3B
  21. 36.9 Aya Expanse 8B
  22. 35.1 Qwen2.5-1.5B
  23. 27.1 Llama 3.2 1B
  24. 25.6 Llama 3.2 3B
  25. 14.6 Qwen2.5-0.5B
  1. 1 Qwen3-235B-A22B 77.1%
  2. 2 Qwen3-32B 76.3%
  3. 3 Qwen3-14B 71.1%
  4. 4 Qwen3-30B-A3B 70.1%
  5. 5 Qwen3-8B 69%
  6. 6 Claude 3.7 Sonnet 67.1%
  7. 7 Qwen3-4B 65.9%
  8. 8 Qwen2.5-72B 58.4%
  9. 9 Qwen2.5-32B 57.4%
  10. 10 Qwen3-1.7B 56%
  11. 11 Aya Expanse 32B 53.8%
  12. 12 Qwen2.5-14B 53.5%
  13. 13 Llama 3.1-70B 52.4%
  14. 14 Gemma 2 27B 50.5%
  15. 15 Qwen2.5-7B 45.9%
  16. 16 Mistral NeMo 41.8%
  17. 17 Gemma 2 9B 41.5%
  18. 18 Apertus 70B 40.5%
  19. 19 Llama 3.1-8B 39.9%
  20. 20 Qwen2.5-3B 39.4%
  21. 21 Aya Expanse 8B 36.9%
  22. 22 Qwen2.5-1.5B 35.1%
  23. 23 Llama 3.2 1B 27.1%
  24. 24 Llama 3.2 3B 25.6%
  25. 25 Qwen2.5-0.5B 14.6%

Sources: Labs’ model cards and official MLogiQA leaderboard

#ModelScoreSettingDateSource
1 Qwen3-235B-A22B 77.1% Apr 2025 Lab model card
2 Qwen3-32B 76.3% May 2025 Lab model card
3 Qwen3-14B 71.1% May 2025 Lab model card
4 Qwen3-30B-A3B 70.1% May 2025 Lab model card
5 Qwen3-8B 69% May 2025 Lab model card
6 Qwen3-4B 65.9% May 2025 Lab model card
7 Qwen3-1.7B 56% May 2025 Lab model card
8 Apertus 70B 40.5% Sep 2025 Lab model card
#ModelScoreSettingDateSource
1 Claude 3.7 Sonnet 67.1% Feb 2025 Official leaderboard
2 Qwen2.5-72B 58.4% Sep 2024 Official leaderboard
3 Qwen2.5-32B 57.4% Sep 2024 Official leaderboard
4 Aya Expanse 32B 53.8% Oct 2024 Official leaderboard
5 Qwen2.5-14B 53.5% Sep 2024 Official leaderboard
6 Llama 3.1-70B 52.4% Jul 2024 Official leaderboard
7 Gemma 2 27B 50.5% Jun 2024 Official leaderboard
8 Qwen2.5-7B 45.9% Sep 2024 Official leaderboard
9 Mistral NeMo 41.8% Jul 2024 Official leaderboard
10 Gemma 2 9B 41.5% Jun 2024 Official leaderboard
11 Llama 3.1-8B 39.9% Jul 2024 Official leaderboard
12 Qwen2.5-3B 39.4% Sep 2024 Official leaderboard
13 Aya Expanse 8B 36.9% Oct 2024 Official leaderboard
14 Qwen2.5-1.5B 35.1% Sep 2024 Official leaderboard
15 Llama 3.2 1B 27.1% Sep 2024 Official leaderboard
16 Llama 3.2 3B 25.6% Sep 2024 Official leaderboard
17 Qwen2.5-0.5B 14.6% Sep 2024 Official leaderboard
#ModelScoreSettingDateSource
1 Qwen3-235B-A22B 77.1% Apr 2025 Lab model card
2 Qwen3-32B 76.3% May 2025 Lab model card
3 Qwen3-14B 71.1% May 2025 Lab model card
4 Qwen3-30B-A3B 70.1% May 2025 Lab model card
5 Qwen3-8B 69% May 2025 Lab model card
6 Claude 3.7 Sonnet 67.1% Feb 2025 Official leaderboard
7 Qwen3-4B 65.9% May 2025 Lab model card
8 Qwen2.5-72B 58.4% Sep 2024 Official leaderboard
9 Qwen2.5-32B 57.4% Sep 2024 Official leaderboard
10 Qwen3-1.7B 56% May 2025 Lab model card
11 Aya Expanse 32B 53.8% Oct 2024 Official leaderboard
12 Qwen2.5-14B 53.5% Sep 2024 Official leaderboard
13 Llama 3.1-70B 52.4% Jul 2024 Official leaderboard
14 Gemma 2 27B 50.5% Jun 2024 Official leaderboard
15 Qwen2.5-7B 45.9% Sep 2024 Official leaderboard
16 Mistral NeMo 41.8% Jul 2024 Official leaderboard
17 Gemma 2 9B 41.5% Jun 2024 Official leaderboard
18 Apertus 70B 40.5% Sep 2025 Lab model card
19 Llama 3.1-8B 39.9% Jul 2024 Official leaderboard
20 Qwen2.5-3B 39.4% Sep 2024 Official leaderboard
21 Aya Expanse 8B 36.9% Oct 2024 Official leaderboard
22 Qwen2.5-1.5B 35.1% Sep 2024 Official leaderboard
23 Llama 3.2 1B 27.1% Sep 2024 Official leaderboard
24 Llama 3.2 3B 25.6% Sep 2024 Official leaderboard
25 Qwen2.5-0.5B 14.6% Sep 2024 Official leaderboard

The benchmark's own leaderboard has 20 entries, 17 of them models we track. Switch the scores above to rank them together.

Number one over time 2 models have held number one across all independent runs since its release in November 2024 3 models have held number one with claims included since its release in November 2024

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Qwen Anthropic
Own leaderboard Not tested since Feb 2025
Claude 3.7 Sonnet 67.1%
Lab claims Not tested since Sep 2025
Qwen3-235B-A22B 77.1%

Sources: Official MLogiQA leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) Qwen3-235B-A22B Claude 3.7 Sonnet Qwen2.5-72B Llama 3.1-70B 0% 20% 40% 60% 80% 100% 2024 2025 Near the ceiling (85%) Qwen3-235B-A22B Claude 3.7 Sonnet

Sources: Official MLogiQA leaderboard and labs’ model cards

No launch post has listed it since April 2025

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Qwen
DateLaunch postScoreAs the post writes it
Apr 29, 2025 Qwen3-235B-A22B 77.1 MLogiQA

8 system cards report it, from 2 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Sep 17, 2025 Apertus 70B technical report 40.5 MLogiQA, held-out test suite (Table 21) Capabilities
May 14, 2025 Qwen3-8B technical report 69 MLogiQA, thinking mode (Table 17). Non-thinking 51.4 (Table 18) Capabilities
May 14, 2025 Qwen3-4B technical report 65.9 MLogiQA, thinking mode (Table 17). Non-thinking 49.9 (Table 18) Capabilities
May 14, 2025 Qwen3-32B technical report 76.3 MLogiQA, thinking mode (Table 13). Non-thinking 62.9 (Table 14) Capabilities
May 14, 2025 Qwen3-30B-A3B technical report 70.1 MLogiQA, thinking mode (Table 15). Non-thinking 53.3 (Table 16) Capabilities
May 14, 2025 Qwen3-235B-A22B technical report 77.1 MLogiQA, thinking mode (Table 11). Non-thinking 67.6 (Table 12) Capabilities
May 14, 2025 Qwen3-14B technical report 71.1 MLogiQA, thinking mode (Table 15). Non-thinking 58.9 (Table 16) Capabilities
May 14, 2025 Qwen3-1.7B technical report 56 MLogiQA, thinking mode (Table 19). Non-thinking 41.1 (Table 20) Capabilities

More in Multilingual

All 120 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Nemotron 3 Ultra 550B-A55B Base 90.1%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%