FACTS Grounding
Long-form responses must be fully grounded in a provided document. Judged by LLM panels for grounding and eligibility.
Public split of 3,513 examples plus a private evaluation set.
- Made by
- Google DeepMind + Kaggle
- Released
- January 2025
- Kind
- Faithful to sources
- Level
- University
- Scored
- Graded by a model
- Answers
- Open-ended
- Tasks
- Public, plus a private test set
- Strongest model tested
-
Gemini 4 Argon #9 on the capability index
- Last in a launch post
- September 2025, dropped after it saturated
- Last model added
- October 2026
- Scores refreshed
- November 12th, 2026
In the authors’ words
We introduce FACTS Grounding, an online leaderboard and associated benchmark that evaluates language models' ability to generate text that is factually accurate with respect to given context in the user prompt. In our benchmark, each prompt includes a user request and a full document, with a maximum length of 32k tokens, requiring long-form responses. The long-form responses are required to be fully grounded in the provided context document while fulfilling the user request. Models are evaluated using automated judge models in two phases: (1) responses are disqualified if they do not fulfill the user request; (2) they are judged as accurate if the response is fully grounded in the provided document. The automated judge models were comprehensively evaluated against a held-out test-set to pick the best prompt template, and the final factuality score is an aggregate of multiple judge models to mitigate evaluation bias. The FACTS Grounding leaderboard will be actively maintained over time, and contains both public and private splits to allow for external participation while guarding the integrity of the leaderboard. It can be found at https://www.kaggle.com/facts-leaderboard.
Kimi K2 leads with 88.5%
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
Gemini 2.5 Pro leads with 87.8% across all independent runs
Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.
Kimi K2 leads with 88.5%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.
- 88.5 Kimi K2
- 87.8
Gemini 2.5 Pro
- 87.5
Gemini 2.5 Flash
- 87.5
Gemini 2.5 Flash-Lite
- 86.8
Gemini 2.5 Flash-Lite
- 85.3
Gemini 2.5 Flash
- 84.6
Gemini 2.0 Flash
- 83.6
Gemini 2.0 Flash
- 83.6
Gemini 2.0 Flash-Lite
- 82.8
Gemini 2.0 Pro
- 79.0
Gemini 4 Argon
- 75.8
Gemma 3 12B
- 74.9
Gemma 3 27B
- 70.1
Gemma 3 4B
- 36.4
Gemma 3 1B
- 1 Kimi K2 Jul 2025 88.5%
- 2
Gemini 2.5 Pro Jun 2025 87.8%
- 3
Gemini 2.5 Flash Sep 2025 87.5%
- 4
Gemini 2.5 Flash-Lite Sep 2025 87.5%
- 5
Gemini 2.5 Flash-Lite Jun 2025 86.8%
- 6
Gemini 2.5 Flash Jun 2025 85.3%
- 7
Gemini 2.0 Flash Feb 2025 84.6%
- 8
Gemini 2.0 Flash Dec 2024 83.6%
- 9
Gemini 2.0 Flash-Lite 83.6%
- 10
Gemini 2.0 Pro 82.8%
- 11
Gemini 4 Argon 79%
- 12
Gemma 3 12B 75.8%
- 13
Gemma 3 27B 74.9%
- 14
Gemma 3 4B 70.1%
- 15
Gemma 3 1B 36.4%
Source: Labs’ model cards
- 87.8
Gemini 2.5 Pro
- 87.4 GPT-5
- 86.4
Gemini 2.5 Flash
- 85.6
Gemini 2.0 Flash
- 83.3 Claude 3.5 Sonnet
- 83.2
Gemini 2.0 Flash-Lite
- 82.4
DeepSeek-R1
- 82.3
Gemini 1.5 Flash
- 81.2
Gemini 1.5 Pro
- 80.6
Gemma 3 27B
- 80.5
Gemma 3 12B
- 79.7
Gemini 1.5 Flash 8B
- 79.1 Claude Sonnet 4
- 78.8 Claude 3.7 Sonnet
- 78.8 o1
- 78.1 Claude 3.5 Haiku
- 77.7 Claude Opus 4
- 77.6 o3-mini
- 76.2 Grok-2
- 75.9 GPT-4o
- 75.7 Grok 3
- 75.0 Grok-3 mini
- 74.5 GPT-4.1
- 72.6
Mistral Large 2
- 70.1
Mixtral 8x22B
- 69.6 o3
- 68.4 GPT-4o mini
- 64.8
Gemma 3 4B
- 63.9
DeepSeek-V3
- 62.1 o1-mini
- 62.1 o4-mini
- 59.7
Codestral
- 56.1
Ministral 8B
- 52.2
Ministral 3B
- 1
Gemini 2.5 Pro Jun 2025 87.8%
- 2 GPT-5 87.4%
- 3
Gemini 2.5 Flash Jun 2025 86.4%
- 4
Gemini 2.0 Flash Feb 2025 85.6%
- 5 Claude 3.5 Sonnet 83.3%
- 6
Gemini 2.0 Flash-Lite 83.2%
- 7
DeepSeek-R1 May 2025 82.4%
- 8
Gemini 1.5 Flash Sep 2024 82.3%
- 9
Gemini 1.5 Pro Sep 2024 81.2%
- 10
Gemma 3 27B 80.6%
- 11
Gemma 3 12B 80.5%
- 12
Gemini 1.5 Flash 8B 79.7%
- 13 Claude Sonnet 4 79.1%
- 14 Claude 3.7 Sonnet 78.8%
- 15 o1 78.8%
- 16 Claude 3.5 Haiku 78.1%
- 17 Claude Opus 4 77.7%
- 18 o3-mini 77.6%
- 19 Grok-2 Dec 2024 76.2%
- 20 GPT-4o Mar 2025 75.9%
- 21 Grok 3 75.7%
- 22 Grok-3 mini 75%
- 23 GPT-4.1 74.5%
- 24
Mistral Large 2 Nov 2024 72.6%
- 25
Mixtral 8x22B 70.1%
- 26 o3 69.6%
- 27 GPT-4o mini 68.4%
- 28
Gemma 3 4B 64.8%
- 29
DeepSeek-V3 Mar 2025 63.9%
- 30 o1-mini 62.1%
- 31 o4-mini 62.1%
- 32
Codestral 59.7%
- 33
Ministral 8B 56.1%
- 34
Ministral 3B 52.2%
Source: Official FACTS Grounding leaderboard
- 88.5 Kimi K2
- 87.8
Gemini 2.5 Pro
- 87.5
Gemini 2.5 Flash
- 87.5
Gemini 2.5 Flash-Lite
- 87.4 GPT-5
- 86.8
Gemini 2.5 Flash-Lite
- 86.4
Gemini 2.5 Flash
- 85.6
Gemini 2.0 Flash
- 83.6
Gemini 2.0 Flash
- 83.6
Gemini 2.0 Flash-Lite
- 83.3 Claude 3.5 Sonnet
- 82.8
Gemini 2.0 Pro
- 82.4
DeepSeek-R1
- 82.3
Gemini 1.5 Flash
- 81.2
Gemini 1.5 Pro
- 80.6
Gemma 3 27B
- 80.5
Gemma 3 12B
- 79.7
Gemini 1.5 Flash 8B
- 79.1 Claude Sonnet 4
- 79.0
Gemini 4 Argon
- 78.8 Claude 3.7 Sonnet
- 78.8 o1
- 78.1 Claude 3.5 Haiku
- 77.7 Claude Opus 4
- 77.6 o3-mini
- 76.2 Grok-2
- 75.9 GPT-4o
- 75.7 Grok 3
- 75.0 Grok-3 mini
- 74.5 GPT-4.1
- 72.6
Mistral Large 2
- 70.1
Gemma 3 4B
- 70.1
Mixtral 8x22B
- 69.6 o3
- 68.4 GPT-4o mini
- 63.9
DeepSeek-V3
- 62.1 o1-mini
- 62.1 o4-mini
- 59.7
Codestral
- 56.1
Ministral 8B
- 52.2
Ministral 3B
- 36.4
Gemma 3 1B
- 1 Kimi K2 Jul 2025 88.5%
- 2
Gemini 2.5 Pro Jun 2025 87.8%
- 3
Gemini 2.5 Flash Sep 2025 87.5%
- 4
Gemini 2.5 Flash-Lite Sep 2025 87.5%
- 5 GPT-5 87.4%
- 6
Gemini 2.5 Flash-Lite Jun 2025 86.8%
- 7
Gemini 2.5 Flash Jun 2025 86.4%
- 8
Gemini 2.0 Flash Feb 2025 85.6%
- 9
Gemini 2.0 Flash Dec 2024 83.6%
- 10
Gemini 2.0 Flash-Lite 83.6%
- 11 Claude 3.5 Sonnet 83.3%
- 12
Gemini 2.0 Pro 82.8%
- 13
DeepSeek-R1 May 2025 82.4%
- 14
Gemini 1.5 Flash Sep 2024 82.3%
- 15
Gemini 1.5 Pro Sep 2024 81.2%
- 16
Gemma 3 27B 80.6%
- 17
Gemma 3 12B 80.5%
- 18
Gemini 1.5 Flash 8B 79.7%
- 19 Claude Sonnet 4 79.1%
- 20
Gemini 4 Argon 79%
- 21 Claude 3.7 Sonnet 78.8%
- 22 o1 78.8%
- 23 Claude 3.5 Haiku 78.1%
- 24 Claude Opus 4 77.7%
- 25 o3-mini 77.6%
- 26 Grok-2 Dec 2024 76.2%
- 27 GPT-4o Mar 2025 75.9%
- 28 Grok 3 75.7%
- 29 Grok-3 mini 75%
- 30 GPT-4.1 74.5%
- 31
Mistral Large 2 Nov 2024 72.6%
- 32
Gemma 3 4B 70.1%
- 33
Mixtral 8x22B 70.1%
- 34 o3 69.6%
- 35 GPT-4o mini 68.4%
- 36
DeepSeek-V3 Mar 2025 63.9%
- 37 o1-mini 62.1%
- 38 o4-mini 62.1%
- 39
Codestral 59.7%
- 40
Ministral 8B 56.1%
- 41
Ministral 3B 52.2%
- 42
Gemma 3 1B 36.4%
Sources: Labs’ model cards and official FACTS Grounding leaderboard
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Kimi K2 Jul 2025 | 88.5% | Jul 2025 | Lab model card | |
| 2 | | 87.8% | Jun 2025 | Lab model card | |
| 3 | | 87.5% | Sep 2025 | Lab model card | |
| 4 | | 87.5% | Sep 2025 | Lab model card | |
| 5 | | 86.8% | Jun 2025 | Lab model card | |
| 6 | | 85.3% | Jun 2025 | Lab model card | |
| 7 | | 84.6% | Feb 2025 | Lab model card | |
| 8 | | 83.6% | Dec 2024 | Lab model card | |
| 9 | | 83.6% | Feb 2025 | Lab model card | |
| 10 | | 82.8% | Feb 2025 | Lab model card | |
| 11 | | 79% | Oct 2026 | Lab model card | |
| 12 | | 75.8% | Mar 2025 | Lab model card | |
| 13 | | 74.9% | Mar 2025 | Lab model card | |
| 14 | | 70.1% | Mar 2025 | Lab model card | |
| 15 | | 36.4% | Mar 2025 | Lab model card |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 87.8% | Jun 2025 | Official leaderboard | |
| 2 | GPT-5 | 87.4% | Aug 2025 | Official leaderboard | |
| 3 | | 86.4% | Jun 2025 | Official leaderboard | |
| 4 | | 85.6% | Feb 2025 | Official leaderboard | |
| 5 | Claude 3.5 Sonnet | 83.3% | Jun 2024 | Official leaderboard | |
| 6 | | 83.2% | Feb 2025 | Official leaderboard | |
| 7 | | 82.4% | May 2025 | Official leaderboard | |
| 8 | | 82.3% | Aug 2024 | Official leaderboard | |
| 9 | | 81.2% | Sep 2024 | Official leaderboard | |
| 10 | | 80.6% | Mar 2025 | Official leaderboard | |
| 11 | | 80.5% | Mar 2025 | Official leaderboard | |
| 12 | | 79.7% | Oct 2024 | Official leaderboard | |
| 13 | Claude Sonnet 4 | 79.1% | May 2025 | Official leaderboard | |
| 14 | Claude 3.7 Sonnet | 78.8% | Feb 2025 | Official leaderboard | |
| 15 | o1 | 78.8% | Dec 2024 | Official leaderboard | |
| 16 | Claude 3.5 Haiku | 78.1% | Oct 2024 | Official leaderboard | |
| 17 | Claude Opus 4 | 77.7% | May 2025 | Official leaderboard | |
| 18 | o3-mini | 77.6% | Jan 2025 | Official leaderboard | |
| 19 | Grok-2 Dec 2024 | 76.2% | Dec 2024 | Official leaderboard | |
| 20 | GPT-4o Mar 2025 | 75.9% | Mar 2025 | Official leaderboard | |
| 21 | Grok 3 | 75.7% | Apr 2025 | Official leaderboard | |
| 22 | Grok-3 mini | 75% | Apr 2025 | Official leaderboard | |
| 23 | GPT-4.1 | 74.5% | Apr 2025 | Official leaderboard | |
| 24 | | 72.6% | Nov 2024 | Official leaderboard | |
| 25 | | 70.1% | Apr 2024 | Official leaderboard | |
| 26 | o3 | 69.6% | Apr 2025 | Official leaderboard | |
| 27 | GPT-4o mini | 68.4% | Jul 2024 | Official leaderboard | |
| 28 | | 64.8% | Mar 2025 | Official leaderboard | |
| 29 | | 63.9% | Mar 2025 | Official leaderboard | |
| 30 | o1-mini | 62.1% | Sep 2024 | Official leaderboard | |
| 31 | o4-mini | 62.1% | Apr 2025 | Official leaderboard | |
| 32 | | 59.7% | Jan 2025 | Official leaderboard | |
| 33 | | 56.1% | Oct 2024 | Official leaderboard | |
| 34 | | 52.2% | Oct 2024 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Kimi K2 Jul 2025 | 88.5% | Jul 2025 | Lab model card | |
| 2 | | 87.8% | Jun 2025 | Official leaderboard | |
| 3 | | 87.5% | Sep 2025 | Lab model card | |
| 4 | | 87.5% | Sep 2025 | Lab model card | |
| 5 | GPT-5 | 87.4% | Aug 2025 | Official leaderboard | |
| 6 | | 86.8% | Jun 2025 | Lab model card | |
| 7 | | 86.4% | Jun 2025 | Official leaderboard | |
| 8 | | 85.6% | Feb 2025 | Official leaderboard | |
| 9 | | 83.6% | Dec 2024 | Lab model card | |
| 10 | | 83.6% | Feb 2025 | Lab model card | |
| 11 | Claude 3.5 Sonnet | 83.3% | Jun 2024 | Official leaderboard | |
| 12 | | 82.8% | Feb 2025 | Lab model card | |
| 13 | | 82.4% | May 2025 | Official leaderboard | |
| 14 | | 82.3% | Aug 2024 | Official leaderboard | |
| 15 | | 81.2% | Sep 2024 | Official leaderboard | |
| 16 | | 80.6% | Mar 2025 | Official leaderboard | |
| 17 | | 80.5% | Mar 2025 | Official leaderboard | |
| 18 | | 79.7% | Oct 2024 | Official leaderboard | |
| 19 | Claude Sonnet 4 | 79.1% | May 2025 | Official leaderboard | |
| 20 | | 79% | Oct 2026 | Lab model card | |
| 21 | Claude 3.7 Sonnet | 78.8% | Feb 2025 | Official leaderboard | |
| 22 | o1 | 78.8% | Dec 2024 | Official leaderboard | |
| 23 | Claude 3.5 Haiku | 78.1% | Oct 2024 | Official leaderboard | |
| 24 | Claude Opus 4 | 77.7% | May 2025 | Official leaderboard | |
| 25 | o3-mini | 77.6% | Jan 2025 | Official leaderboard | |
| 26 | Grok-2 Dec 2024 | 76.2% | Dec 2024 | Official leaderboard | |
| 27 | GPT-4o Mar 2025 | 75.9% | Mar 2025 | Official leaderboard | |
| 28 | Grok 3 | 75.7% | Apr 2025 | Official leaderboard | |
| 29 | Grok-3 mini | 75% | Apr 2025 | Official leaderboard | |
| 30 | GPT-4.1 | 74.5% | Apr 2025 | Official leaderboard | |
| 31 | | 72.6% | Nov 2024 | Official leaderboard | |
| 32 | | 70.1% | Mar 2025 | Lab model card | |
| 33 | | 70.1% | Apr 2024 | Official leaderboard | |
| 34 | o3 | 69.6% | Apr 2025 | Official leaderboard | |
| 35 | GPT-4o mini | 68.4% | Jul 2024 | Official leaderboard | |
| 36 | | 63.9% | Mar 2025 | Official leaderboard | |
| 37 | o1-mini | 62.1% | Sep 2024 | Official leaderboard | |
| 38 | o4-mini | 62.1% | Apr 2025 | Official leaderboard | |
| 39 | | 59.7% | Jan 2025 | Official leaderboard | |
| 40 | | 56.1% | Oct 2024 | Official leaderboard | |
| 41 | | 52.2% | Oct 2024 | Official leaderboard | |
| 42 | | 36.4% | Mar 2025 | Lab model card |
The benchmark's own leaderboard has 35 entries, 34 of them models we track. Switch the scores above to rank them together.
4 models have held number one in labs’ own numbers since its release in January 2025 3 models have held number one across all independent runs since its release in January 2025 4 models have held number one with claims included since its release in January 2025
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Sources: Official FACTS Grounding leaderboard and labs’ model cards
Gemini 2.0 Flash on the official leaderboard passed 85% in February 2025
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official FACTS Grounding leaderboard and labs’ model cards
Labs dropped it from launch posts once the best score passed 85%
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Sep 25, 2025 | 87.5 | FACTS Grounding, Thinking | |
| Sep 25, 2025 | 87.5 | FACTS Grounding, model card | |
| Jun 17, 2025 | 85.3 | FACTS Grounding | |
| Jun 17, 2025 | 86.8 | FACTS Grounding, Thinking | |
| Jun 17, 2025 | 87.8 | FACTS Grounding | |
| Mar 12, 2025 | 74.9 | FACTS Grounding (IT) | |
| Feb 5, 2025 | 83.6 | FACTS Grounding | |
| Feb 5, 2025 | 84.6 | FACTS Grounding | |
| Feb 5, 2025 | 82.8 | FACTS Grounding | |
| Dec 11, 2024 | 83.6 | FACTS Grounding |
11 system cards report it, from 2 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Oct 13, 2026 | | 79 | FACTS Grounding factuality score 79.0% (Figure 5.13) | Hallucination |
| Sep 26, 2025 | | 87.5 | Factuality FACTS Grounding (Table, pp.4 to 5) | Hallucination |
| Jul 21, 2025 | Kimi K2 technical report | 88.5 | FACTS Grounding (Adjusted) (Table 3) | Hallucination |
| Jul 7, 2025 | | 87.8 | FACTS Grounding (Table 3) | Capabilities |
| Jul 7, 2025 | | 85.3 | FACTS Grounding (Table 3) | Capabilities |
| Apr 15, 2025 | | 84.6 | FACTS Grounding (Table, p.3, column Gemini 2.0 Flash (GA)) | Capabilities |
| Apr 10, 2025 | | 83.6 | FACTS Grounding (Table, p.3) | Capabilities |
| Mar 25, 2025 | | 70.1 | FACTS Grounding, IT, zero-shot (Table 6) | Capabilities |
| Mar 25, 2025 | | 74.9 | FACTS Grounding, IT, zero-shot (Table 6) | Capabilities |
| Mar 25, 2025 | | 36.4 | FACTS Grounding, IT, zero-shot (Table 6) | Capabilities |
| Mar 25, 2025 | | 75.8 | FACTS Grounding, IT, zero-shot (Table 6) | Capabilities |
The lab's number comes out ahead in 2 of 6 pairs that differ
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.
| Model | Lab says | Board says | Gap |
|---|---|---|---|
| | 74.9% FACTS Grounding (IT) | 80.6% Leaderboard | −5.7 pts |
| | 70.1% FACTS Grounding, IT, zero-shot (Table 6) | 64.8% Leaderboard | +5.3 pts |
| | 75.8% FACTS Grounding, IT, zero-shot (Table 6) | 80.5% Leaderboard | −4.7 pts |
| | 85.3% FACTS Grounding | 86.4% Leaderboard | −1.1 pts |
| | 84.6% FACTS Grounding | 85.6% Leaderboard | −1.0 pts |
| | 83.6% FACTS Grounding | 83.2% Leaderboard | +0.4 pts |
| | 87.8% FACTS Grounding | 87.8% Leaderboard | Same |
Versions
More in Factuality
- SimpleVQA In use Models answer 2,025 short factual questions about images across 9 tasks and 9 topics. An LLM judge grades answers against fixed references.
Qwen3.7-Plus 81.7%
- HallusionBench In use Models answer 1,129 yes or no questions about 346 images, built in pairs to expose language hallucination and visual illusion. Scored by accuracy.
Keye-VL-2.0-30B-A3B 73.5%
- SimpleQA Verified In use 1,000 human-verified, de-duplicated short-answer factual questions measuring parametric knowledge.
Gemini 3.1 Pro 77.5%
- Kimi Hallu Bench In use Models answer hard prompts drawn from practical work tasks. Fact-checking agents score the share of claims that are hallucinated. GPT-6 Astra 4.20%
- Internal adversarially-designed factuality evaluations In use Models answer adversarial questions in nine categories such as history, math and code. Scored on agreement with human ideal responses. GPT-6 Luna 7.60%
- AA-Omniscience v1.1 In use Models answer 3,000 knowledge questions in six domains without tools and may abstain. The index adds correct answers and subtracts wrong ones.
Inkling 2 Small 16.1