Benchmark Almanac

PIQA

Phi-3.5-MoE 88.6% leads since August 2024
GPT-3 175B (davinci) 82.8% leads
GPT-4o mini 88.7% leads since July 2024
Dropped

Models pick which of two solutions better achieves an everyday physical goal. Scoring is accuracy.

Released
November 2019
Kind
Commonsense
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
Public, plus a private test set
Strongest model tested
GLM-5 #88 on the capability index
Last in a launch post
September 2025, dropped after it saturated
Last model added
October 2026
Scores refreshed
November 12th, 2026

In the authors’ words

To apply eyeshadow without a brush, should I use a cotton swab or a toothpick? Questions requiring this kind of physical commonsense pose a challenge to today's natural language understanding systems. While recent pretrained models (such as BERT) have made progress on question answering over more abstract domains - such as news articles and encyclopedia entries, where text is plentiful - in more physical domains, text is inherently limited due to reporting bias. Can AI systems learn to reliably answer physical common-sense questions without experiencing the physical world? In this paper, we introduce the task of physical commonsense reasoning and a corresponding benchmark dataset Physical Interaction: Question Answering or PIQA. Though humans find the dataset easy (95% accuracy), large pretrained models struggle (77%). We provide analysis about the dimensions of knowledge that existing models lack, which offers significant opportunities for future research.

Abstract of PIQA: Reasoning about Physical Commonsense in Natural Language (arXiv:1911.11641, CC0)
37
Models scored in lab cards
15
Launch posts list it, from 5 labs
32
System cards report it, from 7 labs

Phi-3.5-MoE leads with 88.6%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

GPT-3 175B (davinci) leads with 82.8% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

GPT-4o mini leads with 88.7%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

37 models. Scroll sideways for the rest.

  1. 88.6 Phi-3.5-MoE
  2. 87.9 phi-3-medium 14B
  3. 86.9 phi-3-small 7.4B
  4. 85.6 Llama 3.1-405B
  5. 85.3 GLM-4.5
  6. 85.1 Nemotron 3.5 Lightning 30B-A3B Base
  7. 85.1 Nemotron 3.5 Super 120B-A12B Base
  8. 85.0 PaLM 2-L
  9. 84.7 DeepSeek-V3
  10. 84.6 GLM-5
  11. 84.3 Nemotron 3 Nano 30B-A3B Base
  12. 84.2 phi-3-mini 3.8B
  13. 83.8 Llama 3.1-70B
  14. 83.8 Nemotron 3 Ultra 550B-A55B Base
  15. 83.6 Mixtral 8x7B
  16. 83.3 Gemma 3 27B
  17. 83.2 Gemma 2 27B
  18. 83.0 Mistral 7B v0.1
  19. 82.8 Llama 2-70B
  20. 82.6 DeepSeek LLM 67B
  21. 81.9 Llama 2-34B
  22. 81.9 Apertus 70B
  23. 81.8 Gemma 3 12B
  24. 81.7 Gemma 2 9B
  25. 81.2 Gemma 7B
  26. 81.0 Phi-3.5-mini
  27. 81.0 Llama 3.1-8B
  28. 81.0 Gemma 3n E4B
  29. 80.5 Llama 2-13B
  30. 80.1 ChatGLM3-6B
  31. 79.6 Gemma 3 4B
  32. 78.8 Llama 2-7B
  33. 77.6 Phi-4 Mini
  34. 77.3 Gemma 2B
  35. 77.1 BitNet b1.58 2B4T
  36. 73.8 Gemma 3 1B
  37. 66.2 Gemma 3 270M
  1. 1 Phi-3.5-MoE 88.6%
  2. 2 phi-3-medium 14B 87.9%
  3. 3 phi-3-small 7.4B 86.9%
  4. 4 Llama 3.1-405B 85.6%
  5. 5 GLM-4.5 85.3%
  6. 6 Nemotron 3.5 Lightning 30B-A3B Base 85.1%
  7. 7 Nemotron 3.5 Super 120B-A12B Base 85.1%
  8. 8 PaLM 2-L 85%
  9. 9 DeepSeek-V3 84.7%
  10. 10 GLM-5 84.6%
  11. 11 Nemotron 3 Nano 30B-A3B Base 84.3%
  12. 12 phi-3-mini 3.8B 84.2%
  13. 13 Llama 3.1-70B 83.8%
  14. 14 Nemotron 3 Ultra 550B-A55B Base 83.8%
  15. 15 Mixtral 8x7B 83.6%
  16. 16 Gemma 3 27B 83.3%
  17. 17 Gemma 2 27B 83.2%
  18. 18 Mistral 7B v0.1 83%
  19. 19 Llama 2-70B 82.8%
  20. 20 DeepSeek LLM 67B 82.6%
  21. 21 Llama 2-34B 81.9%
  22. 22 Apertus 70B 81.9%
  23. 23 Gemma 3 12B 81.8%
  24. 24 Gemma 2 9B 81.7%
  25. 25 Gemma 7B 81.2%
  26. 26 Phi-3.5-mini 81%
  27. 27 Llama 3.1-8B 81%
  28. 28 Gemma 3n E4B 81%
  29. 29 Llama 2-13B 80.5%
  30. 30 ChatGLM3-6B 80.1%
  31. 31 Gemma 3 4B 79.6%
  32. 32 Llama 2-7B 78.8%
  33. 33 Phi-4 Mini 77.6%
  34. 34 Gemma 2B 77.3%
  35. 35 BitNet b1.58 2B4T 77.1%
  36. 36 Gemma 3 1B 73.8%
  37. 37 Gemma 3 270M 66.2%

Source: Labs’ model cards

Source: Official PIQA leaderboard

82 models. Scroll sideways for the rest.

  1. 88.7 GPT-4o mini
  2. 88.6 Phi-3.5-MoE
  3. 87.9 phi-3-medium 14B
  4. 87.5 Gemini 1.5 Flash
  5. 86.9 phi-3-small 7.4B
  6. 85.9 Llama 3.1-405B
  7. 85.3 GLM-4.5
  8. 85.1 Nemotron 3.5 Lightning 30B-A3B Base
  9. 85.1 Nemotron 3.5 Super 120B-A12B Base
  10. 85.0 PaLM 2-L
  11. 84.9 Falcon-180B
  12. 84.7 DeepSeek-V3
  13. 84.6 GLM-5
  14. 84.3 Nemotron 3 Nano 30B-A3B Base
  15. 84.2 Inflection-1
  16. 84.2 phi-3-mini 3.8B
  17. 83.9 DeepSeek-V2 (MoE-236B, May 2024)
  18. 83.8 Llama 3.1-70B
  19. 83.8 Nemotron 3 Ultra 550B-A55B Base
  20. 83.7 Gemma 2 9B
  21. 83.6 Mixtral 8x7B
  22. 83.5 Mistral NeMo
  23. 83.3 Stable Beluga 2
  24. 83.3 Gemma 3 27B
  25. 83.2 Gemma 2 27B
  26. 83.2 Megatron-Turing NLG 530B
  27. 83.0 Falcon-40B
  28. 83.0 Mistral 7B v0.1
  29. 82.8 GPT-3 175B (davinci)
  30. 82.8 LLaMA-65B
  31. 82.8 Llama 2-70B
  32. 82.6 DeepSeek LLM 67B
  33. 82.6 Qwen2.5-72B
  34. 82.4 Nemotron-4 15B
  35. 82.3 InstructGPT 175B
  36. 82.3 PaLM (540B)
  37. 82.3 LLaMA-33B
  38. 82.2 Mistral 7B v0.2
  39. 81.9 MPT-30B
  40. 81.9 Llama 2-34B
  41. 81.9 Apertus 70B
  42. 81.8 Gopher (280B)
  43. 81.8 Chinchilla
  44. 81.8 Gemma 3 12B
  45. 81.2 Gemma 7B
  46. 81.2 Llama 3.1-8B
  47. 81.0 Phi-3.5-mini
  48. 81.0 Gemma 3n E4B
  49. 80.8 Llama 2-13B
  50. 80.6 MPT-7B
  51. 80.5 PaLM 62B
  52. 80.3 Falcon-7B
  53. 80.3 internlm-20b
  54. 80.1 LLaMA-13B
  55. 80.1 ChatGLM3-6B
  56. 79.9 Qwen-14B
  57. 79.8 LLaMA-7B
  58. 79.6 Gemma 3 4B
  59. 78.8 Llama 2-7B
  60. 78.1 Baichuan2-13B
  61. 77.9 internlm-7b
  62. 77.9 Qwen-7B
  63. 77.6 Phi-4 Mini
  64. 77.4 vicuna-13b-v1.1
  65. 77.3 Gemma 2B
  66. 77.1 BitNet b1.58 2B4T
  67. 76.9 RedPajama-INCITE-7B-Base
  68. 76.7 GPT-NeoX-20B
  69. 76.2 Baichuan1-7B
  70. 76.0 open_llama_7b
  71. 75.7 OPT-1.3B
  72. 75.5 XGen-7B
  73. 75.4 GPT-J-6B
  74. 75.4 Dolly 2.0-12b
  75. 73.8 Gemma 3 1B
  76. 73.5 Cerebras-GPT-13B
  77. 73.3 Qwen-1_8B
  78. 72.9 GPT-Neo-2.7B
  79. 70.5 GPT-2 (1.5B)
  80. 69.6 chatglm2-6b
  81. 66.2 Gemma 3 270M
  82. 65.8 stablelm-tuned-alpha-7b
  1. 1 GPT-4o mini 88.7%
  2. 2 Phi-3.5-MoE 88.6%
  3. 3 phi-3-medium 14B 87.9%
  4. 4 Gemini 1.5 Flash Sep 2024 87.5%
  5. 5 phi-3-small 7.4B 86.9%
  6. 6 Llama 3.1-405B 85.9%
  7. 7 GLM-4.5 85.3%
  8. 8 Nemotron 3.5 Lightning 30B-A3B Base 85.1%
  9. 9 Nemotron 3.5 Super 120B-A12B Base 85.1%
  10. 10 PaLM 2-L 85%
  11. 11 Falcon-180B 84.9%
  12. 12 DeepSeek-V3 84.7%
  13. 13 GLM-5 84.6%
  14. 14 Nemotron 3 Nano 30B-A3B Base 84.3%
  15. 15 Inflection-1 84.2%
  16. 16 phi-3-mini 3.8B 84.2%
  17. 17 DeepSeek-V2 (MoE-236B, May 2024) 83.9%
  18. 18 Llama 3.1-70B 83.8%
  19. 19 Nemotron 3 Ultra 550B-A55B Base 83.8%
  20. 20 Gemma 2 9B 83.7%
  21. 21 Mixtral 8x7B 83.6%
  22. 22 Mistral NeMo 83.5%
  23. 23 Stable Beluga 2 83.3%
  24. 24 Gemma 3 27B 83.3%
  25. 25 Gemma 2 27B 83.2%
  26. 26 Megatron-Turing NLG 530B 83.2%
  27. 27 Falcon-40B 83%
  28. 28 Mistral 7B v0.1 83%
  29. 29 GPT-3 175B (davinci) 82.8%
  30. 30 LLaMA-65B 82.8%
  31. 31 Llama 2-70B 82.8%
  32. 32 DeepSeek LLM 67B 82.6%
  33. 33 Qwen2.5-72B 82.6%
  34. 34 Nemotron-4 15B 82.4%
  35. 35 InstructGPT 175B 82.3%
  36. 36 PaLM (540B) 82.3%
  37. 37 LLaMA-33B 82.3%
  38. 38 Mistral 7B v0.2 82.2%
  39. 39 MPT-30B 81.9%
  40. 40 Llama 2-34B 81.9%
  41. 41 Apertus 70B 81.9%
  42. 42 Gopher (280B) 81.8%
  43. 43 Chinchilla 81.8%
  44. 44 Gemma 3 12B 81.8%
  45. 45 Gemma 7B 81.2%
  46. 46 Llama 3.1-8B 81.2%
  47. 47 Phi-3.5-mini 81%
  48. 48 Gemma 3n E4B 81%
  49. 49 Llama 2-13B 80.8%
  50. 50 MPT-7B 80.6%
  51. 51 PaLM 62B 80.5%
  52. 52 Falcon-7B 80.3%
  53. 53 internlm-20b 80.3%
  54. 54 LLaMA-13B 80.1%
  55. 55 ChatGLM3-6B 80.1%
  56. 56 Qwen-14B 79.9%
  57. 57 LLaMA-7B 79.8%
  58. 58 Gemma 3 4B 79.6%
  59. 59 Llama 2-7B 78.8%
  60. 60 Baichuan2-13B 78.1%
  61. 61 internlm-7b 77.9%
  62. 62 Qwen-7B 77.9%
  63. 63 Phi-4 Mini 77.6%
  64. 64 vicuna-13b-v1.1 77.4%
  65. 65 Gemma 2B 77.3%
  66. 66 BitNet b1.58 2B4T 77.1%
  67. 67 RedPajama-INCITE-7B-Base 76.9%
  68. 68 GPT-NeoX-20B 76.7%
  69. 69 Baichuan1-7B 76.2%
  70. 70 open_llama_7b 76%
  71. 71 OPT-1.3B 75.7%
  72. 72 XGen-7B 75.5%
  73. 73 GPT-J-6B 75.4%
  74. 74 Dolly 2.0-12b 75.4%
  75. 75 Gemma 3 1B 73.8%
  76. 76 Cerebras-GPT-13B 73.5%
  77. 77 Qwen-1_8B 73.3%
  78. 78 GPT-Neo-2.7B 72.9%
  79. 79 GPT-2 (1.5B) 70.5%
  80. 80 chatglm2-6b 69.6%
  81. 81 Gemma 3 270M 66.2%
  82. 82 stablelm-tuned-alpha-7b 65.8%

Sources: Another board, via Epoch AI (CC BY 4.0), labs’ model cards and official PIQA leaderboard

#ModelScoreSettingDateSource
1 Phi-3.5-MoE 88.6% Aug 2024 Lab model card
2 phi-3-medium 14B 87.9% Aug 2024 Lab model card
3 phi-3-small 7.4B 86.9% Aug 2024 Lab model card
4 Llama 3.1-405B 85.6% Nov 2024 Lab model card
5 GLM-4.5 85.3% Aug 2025 Lab model card
6 Nemotron 3.5 Lightning 30B-A3B Base 85.1% Oct 2026 Lab model card
7 Nemotron 3.5 Super 120B-A12B Base 85.1% Oct 2026 Lab model card
8 PaLM 2-L 85% May 2023 Lab model card
9 DeepSeek-V3 84.7% Dec 2024 Lab model card
10 GLM-5 84.6% Feb 2026 Lab model card
11 Nemotron 3 Nano 30B-A3B Base 84.3% Dec 2025 Lab model card
12 phi-3-mini 3.8B 84.2% Aug 2024 Lab model card
13 Llama 3.1-70B 83.8% Nov 2024 Lab model card
14 Nemotron 3 Ultra 550B-A55B Base 83.8% Jun 2026 Lab model card
15 Mixtral 8x7B 83.6% Dec 2023 Lab model card
16 Gemma 3 27B 83.3% Mar 2025 Lab model card
17 Gemma 2 27B 83.2% Jun 2024 Lab model card
18 Mistral 7B v0.1 83% Sep 2023 Lab model card
19 Llama 2-70B 82.8% Jul 2023 Lab model card
20 DeepSeek LLM 67B 82.6% Jan 2024 Lab model card
21 Llama 2-34B 81.9% Jul 2023 Lab model card
22 Apertus 70B 81.9% Sep 2025 Lab model card
23 Gemma 3 12B 81.8% Mar 2025 Lab model card
24 Gemma 2 9B 81.7% Jul 2024 Lab model card
25 Gemma 7B 81.2% Feb 2024 Lab model card
26 Phi-3.5-mini 81% Aug 2024 Lab model card
27 Llama 3.1-8B 81% Nov 2024 Lab model card
28 Gemma 3n E4B 81% Jun 2025 Lab model card
29 Llama 2-13B 80.5% Jul 2023 Lab model card
30 ChatGLM3-6B 80.1% Jun 2024 Lab model card
31 Gemma 3 4B 79.6% Mar 2025 Lab model card
32 Llama 2-7B 78.8% Jul 2023 Lab model card
33 Phi-4 Mini 77.6% Feb 2025 Lab model card
34 Gemma 2B 77.3% Mar 2024 Lab model card
35 BitNet b1.58 2B4T 77.1% Apr 2025 Lab model card
36 Gemma 3 1B 73.8% Mar 2025 Lab model card
37 Gemma 3 270M 66.2% Aug 2025 Lab model card
#ModelScoreSettingDateSource
1 GPT-3 175B (davinci) 82.8% May 2020 Official leaderboard
#ModelScoreSettingDateSource
1 GPT-4o mini 88.7% Jul 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
2 Phi-3.5-MoE 88.6% Aug 2024 Lab model card
3 phi-3-medium 14B 87.9% Aug 2024 Lab model card
4 Gemini 1.5 Flash Sep 2024 87.5% Sep 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
5 phi-3-small 7.4B 86.9% Aug 2024 Lab model card
6 Llama 3.1-405B 85.9% Jul 2024 Claim in DeepSeek-V3 Technical Report, via Epoch AI
7 GLM-4.5 85.3% Aug 2025 Lab model card
8 Nemotron 3.5 Lightning 30B-A3B Base 85.1% Oct 2026 Lab model card
9 Nemotron 3.5 Super 120B-A12B Base 85.1% Oct 2026 Lab model card
10 PaLM 2-L 85% May 2023 Lab model card
11 Falcon-180B 84.9% Sep 2023 Claim in The Falcon Series of Open Language Models, via Epoch AI
12 DeepSeek-V3 84.7% Dec 2024 Lab model card
13 GLM-5 84.6% Feb 2026 Lab model card
14 Nemotron 3 Nano 30B-A3B Base 84.3% Dec 2025 Lab model card
15 Inflection-1 84.2% Jun 2023 Claim in The Falcon Series of Open Language Models, via Epoch AI
16 phi-3-mini 3.8B 84.2% Aug 2024 Lab model card
17 DeepSeek-V2 (MoE-236B, May 2024) 83.9% May 2024 Claim in DeepSeek-V3 Technical Report, via Epoch AI
18 Llama 3.1-70B 83.8% Nov 2024 Lab model card
19 Nemotron 3 Ultra 550B-A55B Base 83.8% Jun 2026 Lab model card
20 Gemma 2 9B 83.7% Jun 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
21 Mixtral 8x7B 83.6% Dec 2023 Lab model card
22 Mistral NeMo 83.5% Jul 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
23 Stable Beluga 2 83.3% Jul 2023 Claim in Qwen Technical Report, via Epoch AI
24 Gemma 3 27B 83.3% Mar 2025 Lab model card
25 Gemma 2 27B 83.2% Jun 2024 Lab model card
26 Megatron-Turing NLG 530B 83.2% Oct 2021 Claim in Using DeepSpeed and Megatron to Train Megatron-Turing NLG…, via Epoch AI
27 Falcon-40B 83% Mar 2023 Claim in The Falcon Series of Open Language Models, via Epoch AI
28 Mistral 7B v0.1 83% Sep 2023 Lab model card
29 GPT-3 175B (davinci) 82.8% May 2020 Official leaderboard
30 LLaMA-65B 82.8% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
31 Llama 2-70B 82.8% Jul 2023 Lab model card
32 DeepSeek LLM 67B 82.6% Jan 2024 Lab model card
33 Qwen2.5-72B 82.6% Sep 2024 Claim in DeepSeek-V3 Technical Report, via Epoch AI
34 Nemotron-4 15B 82.4% Feb 2024 Claim in Nemotron-4 15B Technical Report, via Epoch AI
35 InstructGPT 175B 82.3% Jan 2022 Claim in GLaM: Efficient Scaling of Language Models with Mixture-o…, via Epoch AI
36 PaLM (540B) 82.3% Apr 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
37 LLaMA-33B 82.3% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
38 Mistral 7B v0.2 82.2% Dec 2023 Claim in Gemma: Open Models Based on Gemini Research and Technology, via Epoch AI
39 MPT-30B 81.9% Jun 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
40 Llama 2-34B 81.9% Jul 2023 Lab model card
41 Apertus 70B 81.9% Sep 2025 Lab model card
42 Gopher (280B) 81.8% Dec 2021 Claim in Using DeepSpeed and Megatron to Train Megatron-Turing NLG…, via Epoch AI
43 Chinchilla 81.8% Mar 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
44 Gemma 3 12B 81.8% Mar 2025 Lab model card
45 Gemma 7B 81.2% Feb 2024 Lab model card
46 Llama 3.1-8B 81.2% Jul 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
47 Phi-3.5-mini 81% Aug 2024 Lab model card
48 Gemma 3n E4B 81% Jun 2025 Lab model card
49 Llama 2-13B 80.8% Jul 2023 Claim in Mixtral of Experts, via Epoch AI
50 MPT-7B 80.6% May 2023 Claim in Qwen Technical Report, via Epoch AI
51 PaLM 62B 80.5% Apr 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
52 Falcon-7B 80.3% Apr 2023 Claim in The Falcon Series of Open Language Models, via Epoch AI
53 internlm-20b 80.3% Sep 2023 Claim in Qwen Technical Report, via Epoch AI
54 LLaMA-13B 80.1% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
55 ChatGLM3-6B 80.1% Jun 2024 Lab model card
56 Qwen-14B 79.9% Sep 2023 Claim in Qwen Technical Report, via Epoch AI
57 LLaMA-7B 79.8% Feb 2023 Claim in Qwen Technical Report, via Epoch AI
58 Gemma 3 4B 79.6% Mar 2025 Lab model card
59 Llama 2-7B 78.8% Jul 2023 Lab model card
60 Baichuan2-13B 78.1% Sep 2023 Claim in Qwen Technical Report, via Epoch AI
61 internlm-7b 77.9% Jul 2023 Claim in Qwen Technical Report, via Epoch AI
62 Qwen-7B 77.9% Sep 2023 Claim in Qwen Technical Report, via Epoch AI
63 Phi-4 Mini 77.6% Feb 2025 Lab model card
64 vicuna-13b-v1.1 77.4% Apr 2023 Claim in Textbooks Are All You Need II: phi-1.5 technical report, via Epoch AI
65 Gemma 2B 77.3% Mar 2024 Lab model card
66 BitNet b1.58 2B4T 77.1% Apr 2025 Lab model card
67 RedPajama-INCITE-7B-Base 76.9% May 2023 Claim in XGen-7B Technical Report, via Epoch AI
68 GPT-NeoX-20B 76.7% Apr 2022 Claim in XGen-7B Technical Report, via Epoch AI
69 Baichuan1-7B 76.2% Jun 2023 Claim in Qwen Technical Report, via Epoch AI
70 open_llama_7b 76% Jun 2023 Claim in XGen-7B Technical Report, via Epoch AI
71 OPT-1.3B 75.7% May 2022 Claim in XGen-7B Technical Report, via Epoch AI
72 XGen-7B 75.5% Jun 2023 Claim in XGen-7B Technical Report, via Epoch AI
73 GPT-J-6B 75.4% Aug 2021 Claim in XGen-7B Technical Report, via Epoch AI
74 Dolly 2.0-12b 75.4% Apr 2023 Claim in XGen-7B Technical Report, via Epoch AI
75 Gemma 3 1B 73.8% Mar 2025 Lab model card
76 Cerebras-GPT-13B 73.5% Mar 2023 Claim in XGen-7B Technical Report, via Epoch AI
77 Qwen-1_8B 73.3% Nov 2023 Claim in Qwen Technical Report, via Epoch AI
78 GPT-Neo-2.7B 72.9% Mar 2021 Claim in Textbooks Are All You Need II: phi-1.5 technical report, via Epoch AI
79 GPT-2 (1.5B) 70.5% Nov 2019 Claim in Textbooks Are All You Need II: phi-1.5 technical report, via Epoch AI
80 chatglm2-6b 69.6% Jun 2023 Claim in Qwen Technical Report, via Epoch AI
81 Gemma 3 270M 66.2% Aug 2025 Lab model card
82 stablelm-tuned-alpha-7b 65.8% Apr 2023 Claim in XGen-7B Technical Report, via Epoch AI

The benchmark's own leaderboard has 9 entries, 1 of them models we track. Epoch AI collected claims for 58 models from labs’ reports and papers. Switch the scores above to rank them together.

3 models have held number one in labs’ own numbers since 2023 Number one over time 4 models have held number one with claims included since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Other labs Microsoft
Lab claims Passed 85% in May 2023, by PaLM 2-L
Phi-3.5-MoE 88.6%

Source: Labs’ model cards

Its lab’s own number for PaLM 2-L passed 85% in May 2023

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card A claim Epoch AI collected from a report or paper Best so far
0% 20% 40% 60% 80% 100% 2020 2021 2022 2023 2024 2025 2026 Near the ceiling (85%) Random guessing GPT-4o mini PaLM 2-L Megatron-Turing NLG 530B GPT-3 175B (davinci) GPT-2 (1.5B) 0% 20% 40% 60% 80% 100% 2020 2022 2024 2026 Near the ceiling (85%) Random guessing GPT-4o mini GPT-3 175B (davinci)

Sources: Official PIQA leaderboard, labs’ model cards and claims collected by Epoch AI (CC BY 4.0)

Labs dropped it from launch posts once the best score passed 85%

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
Mistral AI
Microsoft
DeepSeek
Swiss AI Initiative
DateLaunch postScoreAs the post writes it
Sep 2, 2025 Apertus 70B 81.9 PIQA (pretraining evaluation)
Aug 14, 2025 Gemma 3 270M 66.2 PIQA 0-shot (IT)
Jun 26, 2025 Gemma 3n E4B 81 PIQA 0-shot (PT)
Apr 15, 2025 BitNet b1.58 2B4T 77.1 PIQA
Mar 12, 2025 Gemma 3 27B 83.3 PIQA 0-shot (PT)
Feb 26, 2025 Phi-4 Mini 77.6 PIQA (5-shot)
Dec 26, 2024 DeepSeek-V3 84.7 PIQA (Acc.)
Aug 20, 2024 Phi-3.5-MoE 88.6 PIQA (5-shot)
Jun 27, 2024 Gemma 2 27B 83.2 PIQA (0-shot)
Apr 22, 2024 phi-3-medium 14B 87.7 PIQA (5-Shot)
Feb 21, 2024 Gemma 7B 81.2 PIQA (0-shot)
Dec 12, 2023 Phi-2 PIQA (in Commonsense Reasoning average only)
Dec 11, 2023 Mixtral 8x7B 83.6 PIQA [detailed table]
Sep 27, 2023 Mistral 7B v0.1 83 PIQA
May 10, 2023 PaLM 2-L 85 PIQA, 1-shot

32 system cards report it, from 7 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 15, 2026 Nemotron 3.5 Super 120B-A12B Base technical report 85.1 PIQA, 0-shot acc_norm, Table 6 Capabilities
Oct 15, 2026 Nemotron 3.5 Lightning 30B-A3B Base technical report 85.1 PIQA, 0-shot acc_norm, Table 6 Capabilities
Jun 4, 2026 Nemotron 3 Ultra 550B-A55B Base model card 83.8 PIQA, 0-shot acc_norm Capabilities
Feb 17, 2026 GLM-5 technical report 84.6 GLM-5-Base, PIQA (EM) (Table 11) Capabilities
Dec 23, 2025 Nemotron 3 Nano 30B-A3B Base technical report 84.3 PIQA, 0-shot acc_norm, base model Table 2 Capabilities
Aug 8, 2025 GLM-4.5 technical report 85.3 GLM-4.5-Base, PIQA (EM) (Table 2) Capabilities
Apr 16, 2025 BitNet b1.58 2B4T technical report 77.1 PIQA (0-shot; Acc,norm) (Table 1, Table 3) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 79.6 PIQA, PT, 0-shot (Table 9) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 83.3 PIQA, PT, 0-shot (Table 9) Capabilities
Mar 25, 2025 Gemma 3 1B technical report 73.8 PIQA, PT, 0-shot (Table 9) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 81.8 PIQA, PT, 0-shot (Table 9) Capabilities
Mar 3, 2025 Phi-4 Mini technical report 77.6 PIQA (5-Shot) (Table 7) Capabilities
Nov 23, 2024 Llama 3.1-8B technical report 81 PiQA, pretrained (Table 11) Capabilities
Nov 23, 2024 Llama 3.1-70B technical report 83.8 PiQA, pretrained (Table 11) Capabilities
Nov 23, 2024 Llama 3.1-405B technical report 85.6 PiQA, pretrained (Table 11) Capabilities
Aug 30, 2024 phi-3-small 7.4B technical report 86.9 PIQA (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-mini 3.8B technical report 84.2 PIQA (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-medium 14B technical report 87.9 PIQA (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 Phi-3.5-MoE technical report 88.6 PIQA (5-shot) (Table 3) Capabilities
Aug 30, 2024 Phi-3.5-mini technical report 81 PIQA (5-shot) (Table 3) Capabilities
Jul 31, 2024 Gemma 2 9B technical report 81.7 PIQA, PT, 0-shot (Table 13) Capabilities
Jul 31, 2024 Gemma 2 27B technical report 83.2 PIQA, PT, 0-shot (Table 13) Capabilities
Jun 18, 2024 ChatGLM3-6B technical report 80.1 ChatGLM3-6B-Base, PIQA (Table 1) Capabilities
Mar 13, 2024 Gemma 7B technical report 81.2 PIQA, PT, 0-shot (Table 6) Capabilities
Mar 13, 2024 Gemma 2B technical report 77.3 PIQA, PT, 0-shot (Table 6) Capabilities
Jan 8, 2024 Mixtral 8x7B technical report 83.6 PIQA, 0-shot (Table 2) Capabilities
Jan 5, 2024 DeepSeek LLM 67B technical report 82.6 PIQA, DeepSeek 67B Chat (Table 6) Capabilities
Oct 10, 2023 Mistral 7B v0.1 technical report 83 PIQA, 0-shot (Table 2) Capabilities
Jul 19, 2023 Llama 2-7B technical report 78.8 PIQA, 0-shot, pretrained (Table 20) Capabilities
Jul 19, 2023 Llama 2-70B technical report 82.8 PIQA, 0-shot, pretrained (Table 20) Capabilities
Jul 19, 2023 Llama 2-34B technical report 81.9 PIQA, 0-shot, pretrained (Table 20) Capabilities
Jul 19, 2023 Llama 2-13B technical report 80.5 PIQA, 0-shot, pretrained (Table 20) Capabilities

More in Knowledge

All 121 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • KMMLU-Pro In use Models answer multiple-choice questions taken from Korean National Professional Licensure exams. It tests professional knowledge used in Korean industry. Solar Pro 4 79.2%
  • HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed. Seed2.0 Pro 73.6%