Benchmark Almanac

TriviaQA

Llama 3.1-405B 91.8% leads since July 2024
Llama 3.1-405B 91.8% leads since July 2024
Fading

Trivia questions with evidence documents. Scored by exact match.

Saturated base-model metric.

Released
May 2017
Kind
Trivia and open questions
Level
Everyday
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
DeepSeek-V4-Pro #69 on the capability index
Last in a launch post
April 2026, dropped after it saturated
Last model added
October 2026
Scores refreshed
November 12th, 2026

In the authors’ words

We present TriviaQA, a challenging reading comprehension dataset containing over 650K question-answer-evidence triples. TriviaQA includes 95K question-answer pairs authored by trivia enthusiasts and independently gathered evidence documents, six per question on average, that provide high quality distant supervision for answering the questions. We show that, in comparison to other recently introduced large-scale datasets, TriviaQA (1) has relatively complex, compositional questions, (2) has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and (3) requires more cross sentence reasoning to find answers. We also present two baseline algorithms: a feature-based classifier and a state-of-the-art neural network, that performs well on SQuAD reading comprehension. Neither approach comes close to human performance (23% and 40% vs. 80%), suggesting that TriviaQA is a challenging testbed that is worth significant future study. Data and code available at – http://nlp.cs.washington.edu/triviaqa/

Abstract of TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension (arXiv:1705.03551, CC0)
40
Models scored in lab cards
19
Launch posts list it, from 9 labs
32
System cards report it, from 10 labs

Llama 3.1-405B leads with 91.8%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Llama 3.1-405B leads with 91.8%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

40 models. Scroll sideways for the rest.

  1. 91.8 Llama 3.1-405B
  2. 91.0 Sarvam-M
  3. 89.8 Llama 3.1-70B
  4. 89.7 Llama 3-70B
  5. 87.6 Llama 2-70B
  6. 87.5 Claude 2
  7. 86.7 DeepSeek-V2 (MoE-236B, May 2024)
  8. 86.6 Beam 1.0 Base
  9. 86.1 PaLM 2-L
  10. 85.6 DeepSeek-V4-Pro
  11. 85.5 Gemma 3 27B
  12. 84.5 Llama 2-34B
  13. 83.7 Gemma 2 27B
  14. 82.9 DeepSeek-V3
  15. 82.7 Mistral Large
  16. 81.5 DeepSeek LLM 67B
  17. 80.9 GLM-5
  18. 80.3 Mistral Small 3
  19. 80.0 GLM-4.5
  20. 79.6 Llama 2-13B
  21. 78.6 Ring-1T
  22. 78.5 Llama 3-8B
  23. 78.2 Gemma 3 12B
  24. 77.6 Llama 3.1-8B
  25. 76.6 Gemma 2 9B
  26. 75.6 phi-3-medium 14B
  27. 74.8 InternVL3.5-241B-A28B
  28. 72.1 Llama 2-7B
  29. 71.5 Mixtral 8x7B
  30. 70.2 Gemma 3n E4B
  31. 69.9 Mistral 7B v0.1
  32. 66.3 Moonlight-16B-A3B
  33. 65.8 Gemma 3 4B
  34. 64.0 phi-3-mini 3.8B
  35. 63.4 Gemma 7B
  36. 58.1 phi-3-small 7.4B
  37. 53.2 Gemma 2B
  38. 47.7 AFM-4.5B
  39. 39.8 Gemma 3 1B
  40. 33.6 BitNet b1.58 2B4T
  1. 1 Llama 3.1-405B 91.8%
  2. 2 Sarvam-M 91%
  3. 3 Llama 3.1-70B 89.8%
  4. 4 Llama 3-70B 89.7%
  5. 5 Llama 2-70B 87.6%
  6. 6 Claude 2 87.5%
  7. 7 DeepSeek-V2 (MoE-236B, May 2024) 86.7%
  8. 8 Beam 1.0 Base 86.6%
  9. 9 PaLM 2-L 86.1%
  10. 10 DeepSeek-V4-Pro 85.6%
  11. 11 Gemma 3 27B 85.5%
  12. 12 Llama 2-34B 84.5%
  13. 13 Gemma 2 27B 83.7%
  14. 14 DeepSeek-V3 82.9%
  15. 15 Mistral Large 82.7%
  16. 16 DeepSeek LLM 67B 81.5%
  17. 17 GLM-5 80.9%
  18. 18 Mistral Small 3 80.3%
  19. 19 GLM-4.5 80%
  20. 20 Llama 2-13B 79.6%
  21. 21 Ring-1T 78.6%
  22. 22 Llama 3-8B 78.5%
  23. 23 Gemma 3 12B 78.2%
  24. 24 Llama 3.1-8B 77.6%
  25. 25 Gemma 2 9B 76.6%
  26. 26 phi-3-medium 14B 75.6%
  27. 27 InternVL3.5-241B-A28B 74.8%
  28. 28 Llama 2-7B 72.1%
  29. 29 Mixtral 8x7B 71.5%
  30. 30 Gemma 3n E4B 70.2%
  31. 31 Mistral 7B v0.1 69.9%
  32. 32 Moonlight-16B-A3B 66.3%
  33. 33 Gemma 3 4B 65.8%
  34. 34 phi-3-mini 3.8B 64%
  35. 35 Gemma 7B 63.4%
  36. 36 phi-3-small 7.4B 58.1%
  37. 37 Gemma 2B 53.2%
  38. 38 AFM-4.5B 47.7%
  39. 39 Gemma 3 1B 39.8%
  40. 40 BitNet b1.58 2B4T 33.6%

Source: Labs’ model cards

61 models. Scroll sideways for the rest.

  1. 91.8 Llama 3.1-405B
  2. 91.0 Sarvam-M
  3. 89.8 Llama 3.1-70B
  4. 89.7 Llama 3-70B
  5. 87.6 Llama 2-70B
  6. 87.5 Claude 2
  7. 86.7 Claude 1.3
  8. 86.7 DeepSeek-V2 (MoE-236B, May 2024)
  9. 86.6 Beam 1.0 Base
  10. 86.1 PaLM 2-L
  11. 86.0 LLaMA-65B
  12. 85.8 GPT-3.5 Turbo
  13. 85.6 DeepSeek-V4-Pro
  14. 85.5 Gemma 3 27B
  15. 84.8 GPT-4
  16. 84.6 Llama 2-34B
  17. 83.8 LLaMA-33B
  18. 83.7 Gemma 2 27B
  19. 82.9 DeepSeek-V3
  20. 82.7 Mistral Large
  21. 82.2 Mixtral 8x7B
  22. 81.7 PaLM 2-M
  23. 81.5 DeepSeek LLM 67B
  24. 81.4 PaLM (540B)
  25. 80.9 GLM-5
  26. 80.3 Mistral Small 3
  27. 80.0 GLM-4.5
  28. 79.9 Falcon-40B
  29. 79.6 Llama 2-13B
  30. 78.9 Claude Instant
  31. 78.6 Ring-1T
  32. 78.5 Llama 3-8B
  33. 78.2 Gemma 3 12B
  34. 77.9 LLaMA-13B
  35. 77.6 Llama 3.1-8B
  36. 76.6 Gemma 2 9B
  37. 75.8 GLaM
  38. 75.6 phi-3-medium 14B
  39. 75.2 PaLM 2-S
  40. 75.2 Mistral 7B v0.1
  41. 74.8 InternVL3.5-241B-A28B
  42. 73.7 Llama 2-7B
  43. 73.6 MPT-30B
  44. 72.3 Gemma 7B
  45. 71.9 Qwen2.5-72B
  46. 71.2 InstructGPT 175B
  47. 71.0 LLaMA-7B
  48. 70.2 Gemma 3n E4B
  49. 66.3 Moonlight-16B-A3B
  50. 65.8 Gemma 3 4B
  51. 64.6 Chinchilla
  52. 64.6 Falcon-7B
  53. 64.0 phi-3-mini 3.8B
  54. 61.6 MPT-7B
  55. 58.1 phi-3-small 7.4B
  56. 57.2 Gopher (280B)
  57. 53.2 Gemma 2B
  58. 47.7 AFM-4.5B
  59. 45.2 Phi-2
  60. 39.8 Gemma 3 1B
  61. 33.6 BitNet b1.58 2B4T
  1. 1 Llama 3.1-405B 91.8%
  2. 2 Sarvam-M 91%
  3. 3 Llama 3.1-70B 89.8%
  4. 4 Llama 3-70B 89.7%
  5. 5 Llama 2-70B 87.6%
  6. 6 Claude 2 87.5%
  7. 7 Claude 1.3 86.7%
  8. 8 DeepSeek-V2 (MoE-236B, May 2024) 86.7%
  9. 9 Beam 1.0 Base 86.6%
  10. 10 PaLM 2-L 86.1%
  11. 11 LLaMA-65B 86%
  12. 12 GPT-3.5 Turbo Nov 2023 85.8%
  13. 13 DeepSeek-V4-Pro 85.6%
  14. 14 Gemma 3 27B 85.5%
  15. 15 GPT-4 Jun 2023 84.8%
  16. 16 Llama 2-34B 84.6%
  17. 17 LLaMA-33B 83.8%
  18. 18 Gemma 2 27B 83.7%
  19. 19 DeepSeek-V3 82.9%
  20. 20 Mistral Large 82.7%
  21. 21 Mixtral 8x7B 82.2%
  22. 22 PaLM 2-M 81.7%
  23. 23 DeepSeek LLM 67B 81.5%
  24. 24 PaLM (540B) 81.4%
  25. 25 GLM-5 80.9%
  26. 26 Mistral Small 3 80.3%
  27. 27 GLM-4.5 80%
  28. 28 Falcon-40B 79.9%
  29. 29 Llama 2-13B 79.6%
  30. 30 Claude Instant 78.9%
  31. 31 Ring-1T 78.6%
  32. 32 Llama 3-8B 78.5%
  33. 33 Gemma 3 12B 78.2%
  34. 34 LLaMA-13B 77.9%
  35. 35 Llama 3.1-8B 77.6%
  36. 36 Gemma 2 9B 76.6%
  37. 37 GLaM 75.8%
  38. 38 phi-3-medium 14B 75.6%
  39. 39 PaLM 2-S 75.2%
  40. 40 Mistral 7B v0.1 75.2%
  41. 41 InternVL3.5-241B-A28B 74.8%
  42. 42 Llama 2-7B 73.7%
  43. 43 MPT-30B 73.6%
  44. 44 Gemma 7B 72.3%
  45. 45 Qwen2.5-72B 71.9%
  46. 46 InstructGPT 175B 71.2%
  47. 47 LLaMA-7B 71%
  48. 48 Gemma 3n E4B 70.2%
  49. 49 Moonlight-16B-A3B 66.3%
  50. 50 Gemma 3 4B 65.8%
  51. 51 Chinchilla 64.6%
  52. 52 Falcon-7B 64.6%
  53. 53 phi-3-mini 3.8B 64%
  54. 54 MPT-7B 61.6%
  55. 55 phi-3-small 7.4B 58.1%
  56. 56 Gopher (280B) 57.2%
  57. 57 Gemma 2B 53.2%
  58. 58 AFM-4.5B 47.7%
  59. 59 Phi-2 45.2%
  60. 60 Gemma 3 1B 39.8%
  61. 61 BitNet b1.58 2B4T 33.6%

Sources: Labs’ model cards and Another board, via Epoch AI (CC BY 4.0)

#ModelScoreSettingDateSource
1 Llama 3.1-405B 91.8% Jul 2024 Lab model card
2 Sarvam-M 91% May 2025 Lab model card
3 Llama 3.1-70B 89.8% Jul 2024 Lab model card
4 Llama 3-70B 89.7% Apr 2024 Lab model card
5 Llama 2-70B 87.6% Jul 2023 Lab model card
6 Claude 2 87.5% Jul 2023 Lab model card
7 DeepSeek-V2 (MoE-236B, May 2024) 86.7% May 2024 Lab model card
8 Beam 1.0 Base 86.6% Oct 2026 Lab model card
9 PaLM 2-L 86.1% May 2023 Lab model card
10 DeepSeek-V4-Pro 85.6% Apr 2026 Lab model card
11 Gemma 3 27B 85.5% Mar 2025 Lab model card
12 Llama 2-34B 84.5% Jul 2023 Lab model card
13 Gemma 2 27B 83.7% Jun 2024 Lab model card
14 DeepSeek-V3 82.9% Dec 2024 Lab model card
15 Mistral Large 82.7% Feb 2024 Lab model card
16 DeepSeek LLM 67B 81.5% Nov 2023 Lab model card
17 GLM-5 80.9% Feb 2026 Lab model card
18 Mistral Small 3 80.3% Jan 2025 Lab model card
19 GLM-4.5 80% Aug 2025 Lab model card
20 Llama 2-13B 79.6% Jul 2023 Lab model card
21 Ring-1T 78.6% Oct 2025 Lab model card
22 Llama 3-8B 78.5% Apr 2024 Lab model card
23 Gemma 3 12B 78.2% Mar 2025 Lab model card
24 Llama 3.1-8B 77.6% Jul 2024 Lab model card
25 Gemma 2 9B 76.6% Jul 2024 Lab model card
26 phi-3-medium 14B 75.6% Apr 2024 Lab model card
27 InternVL3.5-241B-A28B 74.8% Aug 2025 Lab model card
28 Llama 2-7B 72.1% Jul 2023 Lab model card
29 Mixtral 8x7B 71.5% Dec 2023 Lab model card
30 Gemma 3n E4B 70.2% Jun 2025 Lab model card
31 Mistral 7B v0.1 69.9% Sep 2023 Lab model card
32 Moonlight-16B-A3B 66.3% Feb 2025 Lab model card
33 Gemma 3 4B 65.8% Mar 2025 Lab model card
34 phi-3-mini 3.8B 64% Aug 2024 Lab model card
35 Gemma 7B 63.4% Feb 2024 Lab model card
36 phi-3-small 7.4B 58.1% Aug 2024 Lab model card
37 Gemma 2B 53.2% Mar 2024 Lab model card
38 AFM-4.5B 47.7% Jul 2025 Lab model card
39 Gemma 3 1B 39.8% Mar 2025 Lab model card
40 BitNet b1.58 2B4T 33.6% Apr 2025 Lab model card
#ModelScoreSettingDateSource
1 Llama 3.1-405B 91.8% Jul 2024 Lab model card
2 Sarvam-M 91% May 2025 Lab model card
3 Llama 3.1-70B 89.8% Jul 2024 Lab model card
4 Llama 3-70B 89.7% Apr 2024 Lab model card
5 Llama 2-70B 87.6% Jul 2023 Lab model card
6 Claude 2 87.5% Jul 2023 Lab model card
7 Claude 1.3 86.7% Apr 2023 Claim in Model Card and Evaluations for Claude Models, via Epoch AI
8 DeepSeek-V2 (MoE-236B, May 2024) 86.7% May 2024 Lab model card
9 Beam 1.0 Base 86.6% Oct 2026 Lab model card
10 PaLM 2-L 86.1% May 2023 Lab model card
11 LLaMA-65B 86% Feb 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
12 GPT-3.5 Turbo Nov 2023 85.8% Nov 2023 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
13 DeepSeek-V4-Pro 85.6% Apr 2026 Lab model card
14 Gemma 3 27B 85.5% Mar 2025 Lab model card
15 GPT-4 Jun 2023 84.8% Jun 2023 Collected by Epoch AI, source not named
16 Llama 2-34B 84.6% Jul 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
17 LLaMA-33B 83.8% Feb 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
18 Gemma 2 27B 83.7% Jun 2024 Lab model card
19 DeepSeek-V3 82.9% Dec 2024 Lab model card
20 Mistral Large 82.7% Feb 2024 Lab model card
21 Mixtral 8x7B 82.2% Dec 2023 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
22 PaLM 2-M 81.7% May 2023 Claim in PaLM 2 Technical Report, via Epoch AI
23 DeepSeek LLM 67B 81.5% Nov 2023 Lab model card
24 PaLM (540B) 81.4% Apr 2022 Claim in PaLM: Scaling Language Modeling with Pathways, via Epoch AI
25 GLM-5 80.9% Feb 2026 Lab model card
26 Mistral Small 3 80.3% Jan 2025 Lab model card
27 GLM-4.5 80% Aug 2025 Lab model card
28 Falcon-40B 79.9% Mar 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
29 Llama 2-13B 79.6% Jul 2023 Lab model card
30 Claude Instant 78.9% Aug 2023 Claim in Model Card and Evaluations for Claude Models, via Epoch AI
31 Ring-1T 78.6% Oct 2025 Lab model card
32 Llama 3-8B 78.5% Apr 2024 Lab model card
33 Gemma 3 12B 78.2% Mar 2025 Lab model card
34 LLaMA-13B 77.9% Feb 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
35 Llama 3.1-8B 77.6% Jul 2024 Lab model card
36 Gemma 2 9B 76.6% Jul 2024 Lab model card
37 GLaM 75.8% Dec 2021 Claim in GLaM: Efficient Scaling of Language Models with Mixture-o…, via Epoch AI
38 phi-3-medium 14B 75.6% Apr 2024 Lab model card
39 PaLM 2-S 75.2% May 2023 Claim in PaLM 2 Technical Report, via Epoch AI
40 Mistral 7B v0.1 75.2% Sep 2023 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
41 InternVL3.5-241B-A28B 74.8% Aug 2025 Lab model card
42 Llama 2-7B 73.7% Jul 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
43 MPT-30B 73.6% Jun 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
44 Gemma 7B 72.3% Feb 2024 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
45 Qwen2.5-72B 71.9% Sep 2024 Claim in DeepSeek-V3 Technical Report, via Epoch AI
46 InstructGPT 175B 71.2% Jan 2022 Claim in Language Models are Few-Shot Learners, via Epoch AI
47 LLaMA-7B 71% Feb 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
48 Gemma 3n E4B 70.2% Jun 2025 Lab model card
49 Moonlight-16B-A3B 66.3% Feb 2025 Lab model card
50 Gemma 3 4B 65.8% Mar 2025 Lab model card
51 Chinchilla 64.6% Mar 2022 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
52 Falcon-7B 64.6% Apr 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
53 phi-3-mini 3.8B 64% Aug 2024 Lab model card
54 MPT-7B 61.6% May 2023 Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
55 phi-3-small 7.4B 58.1% Aug 2024 Lab model card
56 Gopher (280B) 57.2% Dec 2021 Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
57 Gemma 2B 53.2% Mar 2024 Lab model card
58 AFM-4.5B 47.7% Jul 2025 Lab model card
59 Phi-2 45.2% Dec 2023 Claim in Phi-3 Technical Report: A Highly Capable Language Model L…, via Epoch AI
60 Gemma 3 1B 39.8% Mar 2025 Lab model card
61 BitNet b1.58 2B4T 33.6% Apr 2025 Lab model card

Epoch AI collected claims for 38 models from labs’ reports and papers. Switch the scores above to rank them together.

5 models have held number one in labs’ own numbers since 2023 7 models have held number one with claims included since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Other labs Anthropic Meta
Lab claims Passed 85% in May 2023, by PaLM 2-L
Llama 3.1-405B 91.8%

Source: Labs’ model cards

A claim for LLaMA-65B that Epoch AI collected passed 85% in February 2023

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card A claim Epoch AI collected from a report or paper Best so far
0% 20% 40% 60% 80% 100% 2022 2023 2024 2025 2026 Near the ceiling (85%) Llama 3.1-405B Llama 2-70B PaLM (540B) GLaM Gopher (280B) 0% 20% 40% 60% 80% 100% 2022 2023 2024 2025 2026 Near the ceiling (85%) Llama 3.1-405B PaLM (540B)

Sources: Labs’ model cards and claims collected by Epoch AI (CC BY 4.0)

Labs headline it less often now, last in April 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
Anthropic
Mistral AI
DeepSeek
Meta
Microsoft
Moonshot AI
Sarvam AI
Arcee AI
DateLaunch postScoreAs the post writes it
Apr 24, 2026 DeepSeek-V4-Pro 85.6 TriviaQA (EM)
Jul 29, 2025 AFM-4.5B 47.7 TriviaQA
Jun 26, 2025 Gemma 3n E4B 70.2 TriviaQA 5-shot (PT)
May 23, 2025 Sarvam-M 91 TriviaQA (0.91)
Apr 15, 2025 BitNet b1.58 2B4T 33.6 TriviaQA
Mar 12, 2025 Gemma 3 27B 85.5 TriviaQA 5-shot (PT)
Feb 22, 2025 Moonlight-16B-A3B 66.3 TriviaQA
Jan 30, 2025 Mistral Small 3 80.3 TriviaQA (5-shot) [base; listed twice, 80.32 and 76.6]
Dec 26, 2024 DeepSeek-V3 82.9 TriviaQA (EM)
Jun 27, 2024 Gemma 2 27B 83.7 TriviaQA (5-shot)
Apr 22, 2024 phi-3-medium 14B 75.6 TriviaQA (5-Shot)
Apr 18, 2024 Llama 3-70B 89.7 TriviaQA-Wiki (5-shot) [pretrained]
Feb 26, 2024 Mistral Large 82.7 TriQA (5-shot)
Feb 21, 2024 Gemma 7B 63.4 TriviaQA (5-shot)
Dec 11, 2023 Mixtral 8x7B 71.5 TriQA [detailed table]
Nov 29, 2023 DeepSeek LLM 67B 81.5 TriviaQA
Sep 27, 2023 Mistral 7B v0.1 69.9 TriviaQA
Jul 11, 2023 Claude 2 87.5 TriviaQA (5-shot)
May 10, 2023 PaLM 2-L 86.1 TriviaQA (EM), 1-shot

32 system cards report it, from 10 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 5, 2026 Beam 1.0 Base technical report 86.6 TriviaQA, 5-shot greedy, Table 23 Capabilities
Feb 17, 2026 GLM-5 technical report 80.9 GLM-5-Base, TriviaQA (EM) (Table 11) Capabilities
Oct 21, 2025 Ring-1T technical report 78.6 TriviaQA (Table 1) Capabilities
Aug 25, 2025 InternVL3.5-241B-A28B technical report 74.8 TriviaQA, OpenCompass (Table 14) Capabilities
Aug 8, 2025 GLM-4.5 technical report 80 GLM-4.5-Base, TriviaQA (EM) (Table 2) Capabilities
Jul 17, 2025 Voxtral Small technical report Trivia QA 5-shot, text-only, chart without printed numbers (Figure 6). Speech-synthesized Trivia QA* 79.4 (Table 8) Capabilities
Apr 16, 2025 BitNet b1.58 2B4T technical report 33.6 TriviaQA (5-shot; EM) (Table 1, Table 3) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 65.8 TriviaQA (TQA), PT, 5-shot (Table 9) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 85.5 TriviaQA (TQA), PT, 5-shot (Table 9) Capabilities
Mar 25, 2025 Gemma 3 1B technical report 39.8 TriviaQA (TQA), PT, 5-shot (Table 9) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 78.2 TriviaQA (TQA), PT, 5-shot (Table 9) Capabilities
Aug 30, 2024 phi-3-small 7.4B technical report 58.1 TriviaQA (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-mini 3.8B technical report 64 TriviaQA (5-Shot) (Section 3 table) Capabilities
Aug 30, 2024 phi-3-medium 14B technical report 73.9 TriviaQA (5-Shot) (Section 3 table) Capabilities
Jul 31, 2024 Gemma 2 9B technical report 76.6 TriviaQA, PT, 5-shot (Table 13) Capabilities
Jul 31, 2024 Gemma 2 27B technical report 83.7 TriviaQA, PT, 5-shot (Table 13) Capabilities
Jul 23, 2024 Llama 3.1-8B model card 77.6 TriviaQA-Wiki, 5-shot, em, pretrained (Base pretrained models table) Capabilities
Jul 23, 2024 Llama 3.1-70B model card 89.8 TriviaQA-Wiki, 5-shot, em, pretrained (Base pretrained models table) Capabilities
Jul 23, 2024 Llama 3.1-405B model card 91.8 TriviaQA-Wiki, 5-shot, em, pretrained (Base pretrained models table) Capabilities
May 7, 2024 DeepSeek-V2 (MoE-236B, May 2024) technical report 86.7 TriviaQA 5-shot, DeepSeek-V2 Chat (RL) (Table 3) Capabilities
Apr 18, 2024 Llama 3-8B model card 78.5 TriviaQA-Wiki (5-shot), pretrained (Base pretrained models table) Capabilities
Apr 18, 2024 Llama 3-70B model card 89.7 TriviaQA-Wiki (5-shot), pretrained (Base pretrained models table) Capabilities
Mar 13, 2024 Gemma 7B technical report 63.4 TriviaQA, PT, 5-shot (Table 6) Capabilities
Mar 13, 2024 Gemma 2B technical report 53.2 TriviaQA, PT, 5-shot (Table 6) Capabilities
Jan 8, 2024 Mixtral 8x7B technical report 71.5 TriQA, 5-shot, no Wikipedia contexts (Table 2) Capabilities
Jan 5, 2024 DeepSeek LLM 67B technical report 81.5 TriviaQA, DeepSeek 67B Chat (Table 6) Capabilities
Oct 10, 2023 Mistral 7B v0.1 technical report 69.9 TriviaQA, 5-shot, no Wikipedia contexts (Table 2) Capabilities
Jul 19, 2023 Llama 2-7B technical report 72.1 TriviaQA (Wiki), exact match, 5-shot, pretrained (Table 22). 1-shot = 68.9 Capabilities
Jul 19, 2023 Llama 2-70B technical report 87.6 TriviaQA (Wiki), exact match, 5-shot, pretrained (Table 22). 1-shot = 85.0 Capabilities
Jul 19, 2023 Llama 2-34B technical report 84.5 TriviaQA (Wiki), exact match, 5-shot, pretrained (Table 22). 1-shot = 83.3 Capabilities
Jul 19, 2023 Llama 2-13B technical report 79.6 TriviaQA (Wiki), exact match, 5-shot, pretrained (Table 22). 1-shot = 77.2 Capabilities
Jul 11, 2023 Claude 2 model card 87.5 TriviaQA (5-shot) (4.3) Capabilities

More in Knowledge

All 121 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • KMMLU-Pro In use Models answer multiple-choice questions taken from Korean National Professional Licensure exams. It tests professional knowledge used in Korean industry. Solar Pro 4 79.2%
  • HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed. Seed2.0 Pro 73.6%