Benchmark Almanac

FACTS Grounding

Kimi K2 Jul 2025 88.5% leads since July 2025
Gemini 2.5 Pro Jun 2025 87.8% leads since June 2025
Kimi K2 Jul 2025 88.5% leads since July 2025
Dropped

Long-form responses must be fully grounded in a provided document. Judged by LLM panels for grounding and eligibility.

Public split of 3,513 examples plus a private evaluation set.

Released
January 2025
Kind
Faithful to sources
Level
University
Scored
Graded by a model
Answers
Open-ended
Tasks
Public, plus a private test set
Strongest model tested
Gemini 4 Argon #9 on the capability index
Last in a launch post
September 2025, dropped after it saturated
Last model added
October 2026
Scores refreshed
November 12th, 2026

In the authors’ words

We introduce FACTS Grounding, an online leaderboard and associated benchmark that evaluates language models' ability to generate text that is factually accurate with respect to given context in the user prompt. In our benchmark, each prompt includes a user request and a full document, with a maximum length of 32k tokens, requiring long-form responses. The long-form responses are required to be fully grounded in the provided context document while fulfilling the user request. Models are evaluated using automated judge models in two phases: (1) responses are disqualified if they do not fulfill the user request; (2) they are judged as accurate if the response is fully grounded in the provided document. The automated judge models were comprehensively evaluated against a held-out test-set to pick the best prompt template, and the final factuality score is an aggregate of multiple judge models to mitigate evaluation bias. The FACTS Grounding leaderboard will be actively maintained over time, and contains both public and private splits to allow for external participation while guarding the integrity of the leaderboard. It can be found at https://www.kaggle.com/facts-leaderboard.

Abstract of The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Input (arXiv:2501.03200, CC0)
15
Models scored in lab cards
10
Launch posts list it, from 1 lab
11
System cards report it, from 2 labs

Kimi K2 leads with 88.5%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Gemini 2.5 Pro leads with 87.8% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Kimi K2 leads with 88.5%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

  1. 1 Kimi K2 Jul 2025 88.5%
  2. 2 Gemini 2.5 Pro Jun 2025 87.8%
  3. 3 Gemini 2.5 Flash Sep 2025 87.5%
  4. 4 Gemini 2.5 Flash-Lite Sep 2025 87.5%
  5. 5 Gemini 2.5 Flash-Lite Jun 2025 86.8%
  6. 6 Gemini 2.5 Flash Jun 2025 85.3%
  7. 7 Gemini 2.0 Flash Feb 2025 84.6%
  8. 8 Gemini 2.0 Flash Dec 2024 83.6%
  9. 9 Gemini 2.0 Flash-Lite 83.6%
  10. 10 Gemini 2.0 Pro 82.8%
  11. 11 Gemini 4 Argon 79%
  12. 12 Gemma 3 12B 75.8%
  13. 13 Gemma 3 27B 74.9%
  14. 14 Gemma 3 4B 70.1%
  15. 15 Gemma 3 1B 36.4%

Source: Labs’ model cards

Source: Official FACTS Grounding leaderboard

Sources: Labs’ model cards and official FACTS Grounding leaderboard

#ModelScoreSettingDateSource
1 Kimi K2 Jul 2025 88.5% Jul 2025 Lab model card
2 Gemini 2.5 Pro Jun 2025 87.8% Jun 2025 Lab model card
3 Gemini 2.5 Flash Sep 2025 87.5% Sep 2025 Lab model card
4 Gemini 2.5 Flash-Lite Sep 2025 87.5% Sep 2025 Lab model card
5 Gemini 2.5 Flash-Lite Jun 2025 86.8% Jun 2025 Lab model card
6 Gemini 2.5 Flash Jun 2025 85.3% Jun 2025 Lab model card
7 Gemini 2.0 Flash Feb 2025 84.6% Feb 2025 Lab model card
8 Gemini 2.0 Flash Dec 2024 83.6% Dec 2024 Lab model card
9 Gemini 2.0 Flash-Lite 83.6% Feb 2025 Lab model card
10 Gemini 2.0 Pro 82.8% Feb 2025 Lab model card
11 Gemini 4 Argon 79% Oct 2026 Lab model card
12 Gemma 3 12B 75.8% Mar 2025 Lab model card
13 Gemma 3 27B 74.9% Mar 2025 Lab model card
14 Gemma 3 4B 70.1% Mar 2025 Lab model card
15 Gemma 3 1B 36.4% Mar 2025 Lab model card
#ModelScoreSettingDateSource
1 Gemini 2.5 Pro Jun 2025 87.8% Jun 2025 Official leaderboard
2 GPT-5 87.4% Aug 2025 Official leaderboard
3 Gemini 2.5 Flash Jun 2025 86.4% Jun 2025 Official leaderboard
4 Gemini 2.0 Flash Feb 2025 85.6% Feb 2025 Official leaderboard
5 Claude 3.5 Sonnet 83.3% Jun 2024 Official leaderboard
6 Gemini 2.0 Flash-Lite 83.2% Feb 2025 Official leaderboard
7 DeepSeek-R1 May 2025 82.4% May 2025 Official leaderboard
8 Gemini 1.5 Flash Sep 2024 82.3% Aug 2024 Official leaderboard
9 Gemini 1.5 Pro Sep 2024 81.2% Sep 2024 Official leaderboard
10 Gemma 3 27B 80.6% Mar 2025 Official leaderboard
11 Gemma 3 12B 80.5% Mar 2025 Official leaderboard
12 Gemini 1.5 Flash 8B 79.7% Oct 2024 Official leaderboard
13 Claude Sonnet 4 79.1% May 2025 Official leaderboard
14 Claude 3.7 Sonnet 78.8% Feb 2025 Official leaderboard
15 o1 78.8% Dec 2024 Official leaderboard
16 Claude 3.5 Haiku 78.1% Oct 2024 Official leaderboard
17 Claude Opus 4 77.7% May 2025 Official leaderboard
18 o3-mini 77.6% Jan 2025 Official leaderboard
19 Grok-2 Dec 2024 76.2% Dec 2024 Official leaderboard
20 GPT-4o Mar 2025 75.9% Mar 2025 Official leaderboard
21 Grok 3 75.7% Apr 2025 Official leaderboard
22 Grok-3 mini 75% Apr 2025 Official leaderboard
23 GPT-4.1 74.5% Apr 2025 Official leaderboard
24 Mistral Large 2 Nov 2024 72.6% Nov 2024 Official leaderboard
25 Mixtral 8x22B 70.1% Apr 2024 Official leaderboard
26 o3 69.6% Apr 2025 Official leaderboard
27 GPT-4o mini 68.4% Jul 2024 Official leaderboard
28 Gemma 3 4B 64.8% Mar 2025 Official leaderboard
29 DeepSeek-V3 Mar 2025 63.9% Mar 2025 Official leaderboard
30 o1-mini 62.1% Sep 2024 Official leaderboard
31 o4-mini 62.1% Apr 2025 Official leaderboard
32 Codestral 59.7% Jan 2025 Official leaderboard
33 Ministral 8B 56.1% Oct 2024 Official leaderboard
34 Ministral 3B 52.2% Oct 2024 Official leaderboard
#ModelScoreSettingDateSource
1 Kimi K2 Jul 2025 88.5% Jul 2025 Lab model card
2 Gemini 2.5 Pro Jun 2025 87.8% Jun 2025 Official leaderboard
3 Gemini 2.5 Flash Sep 2025 87.5% Sep 2025 Lab model card
4 Gemini 2.5 Flash-Lite Sep 2025 87.5% Sep 2025 Lab model card
5 GPT-5 87.4% Aug 2025 Official leaderboard
6 Gemini 2.5 Flash-Lite Jun 2025 86.8% Jun 2025 Lab model card
7 Gemini 2.5 Flash Jun 2025 86.4% Jun 2025 Official leaderboard
8 Gemini 2.0 Flash Feb 2025 85.6% Feb 2025 Official leaderboard
9 Gemini 2.0 Flash Dec 2024 83.6% Dec 2024 Lab model card
10 Gemini 2.0 Flash-Lite 83.6% Feb 2025 Lab model card
11 Claude 3.5 Sonnet 83.3% Jun 2024 Official leaderboard
12 Gemini 2.0 Pro 82.8% Feb 2025 Lab model card
13 DeepSeek-R1 May 2025 82.4% May 2025 Official leaderboard
14 Gemini 1.5 Flash Sep 2024 82.3% Aug 2024 Official leaderboard
15 Gemini 1.5 Pro Sep 2024 81.2% Sep 2024 Official leaderboard
16 Gemma 3 27B 80.6% Mar 2025 Official leaderboard
17 Gemma 3 12B 80.5% Mar 2025 Official leaderboard
18 Gemini 1.5 Flash 8B 79.7% Oct 2024 Official leaderboard
19 Claude Sonnet 4 79.1% May 2025 Official leaderboard
20 Gemini 4 Argon 79% Oct 2026 Lab model card
21 Claude 3.7 Sonnet 78.8% Feb 2025 Official leaderboard
22 o1 78.8% Dec 2024 Official leaderboard
23 Claude 3.5 Haiku 78.1% Oct 2024 Official leaderboard
24 Claude Opus 4 77.7% May 2025 Official leaderboard
25 o3-mini 77.6% Jan 2025 Official leaderboard
26 Grok-2 Dec 2024 76.2% Dec 2024 Official leaderboard
27 GPT-4o Mar 2025 75.9% Mar 2025 Official leaderboard
28 Grok 3 75.7% Apr 2025 Official leaderboard
29 Grok-3 mini 75% Apr 2025 Official leaderboard
30 GPT-4.1 74.5% Apr 2025 Official leaderboard
31 Mistral Large 2 Nov 2024 72.6% Nov 2024 Official leaderboard
32 Gemma 3 4B 70.1% Mar 2025 Lab model card
33 Mixtral 8x22B 70.1% Apr 2024 Official leaderboard
34 o3 69.6% Apr 2025 Official leaderboard
35 GPT-4o mini 68.4% Jul 2024 Official leaderboard
36 DeepSeek-V3 Mar 2025 63.9% Mar 2025 Official leaderboard
37 o1-mini 62.1% Sep 2024 Official leaderboard
38 o4-mini 62.1% Apr 2025 Official leaderboard
39 Codestral 59.7% Jan 2025 Official leaderboard
40 Ministral 8B 56.1% Oct 2024 Official leaderboard
41 Ministral 3B 52.2% Oct 2024 Official leaderboard
42 Gemma 3 1B 36.4% Mar 2025 Lab model card

The benchmark's own leaderboard has 35 entries, 34 of them models we track. Switch the scores above to rank them together.

4 models have held number one in labs’ own numbers since its release in January 2025 3 models have held number one across all independent runs since its release in January 2025 4 models have held number one with claims included since its release in January 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic Google DeepMind Moonshot AI
Own leaderboard Passed 85% in Feb 2025, by Gemini 2.0 Flash
Lab claims Passed 85% in Jun 2025, by Gemini 2.5 Pro
Kimi K2 88.5%

Sources: Official FACTS Grounding leaderboard and labs’ model cards

Gemini 2.0 Flash on the official leaderboard passed 85% in February 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Kimi K2 Gemini 2.0 Flash Claude 3.5 Sonnet Mixtral 8x22B 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Kimi K2 Mixtral 8x22B

Sources: Official FACTS Grounding leaderboard and labs’ model cards

Labs dropped it from launch posts once the best score passed 85%

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
DateLaunch postScoreAs the post writes it
Sep 25, 2025 Gemini 2.5 Flash-Lite 87.5 FACTS Grounding, Thinking
Sep 25, 2025 Gemini 2.5 Flash 87.5 FACTS Grounding, model card
Jun 17, 2025 Gemini 2.5 Flash 85.3 FACTS Grounding
Jun 17, 2025 Gemini 2.5 Flash-Lite 86.8 FACTS Grounding, Thinking
Jun 17, 2025 Gemini 2.5 Pro 87.8 FACTS Grounding
Mar 12, 2025 Gemma 3 27B 74.9 FACTS Grounding (IT)
Feb 5, 2025 Gemini 2.0 Flash-Lite 83.6 FACTS Grounding
Feb 5, 2025 Gemini 2.0 Flash 84.6 FACTS Grounding
Feb 5, 2025 Gemini 2.0 Pro 82.8 FACTS Grounding
Dec 11, 2024 Gemini 2.0 Flash 83.6 FACTS Grounding

11 system cards report it, from 2 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 13, 2026 Gemini 4 Argon system card 79 FACTS Grounding factuality score 79.0% (Figure 5.13) Hallucination
Sep 26, 2025 Gemini 2.5 Flash model card 87.5 Factuality FACTS Grounding (Table, pp.4 to 5) Hallucination
Jul 21, 2025 Kimi K2 technical report 88.5 FACTS Grounding (Adjusted) (Table 3) Hallucination
Jul 7, 2025 Gemini 2.5 Pro technical report 87.8 FACTS Grounding (Table 3) Capabilities
Jul 7, 2025 Gemini 2.5 Flash technical report 85.3 FACTS Grounding (Table 3) Capabilities
Apr 15, 2025 Gemini 2.0 Flash model card 84.6 FACTS Grounding (Table, p.3, column Gemini 2.0 Flash (GA)) Capabilities
Apr 10, 2025 Gemini 2.0 Flash-Lite model card 83.6 FACTS Grounding (Table, p.3) Capabilities
Mar 25, 2025 Gemma 3 4B technical report 70.1 FACTS Grounding, IT, zero-shot (Table 6) Capabilities
Mar 25, 2025 Gemma 3 27B technical report 74.9 FACTS Grounding, IT, zero-shot (Table 6) Capabilities
Mar 25, 2025 Gemma 3 1B technical report 36.4 FACTS Grounding, IT, zero-shot (Table 6) Capabilities
Mar 25, 2025 Gemma 3 12B technical report 75.8 FACTS Grounding, IT, zero-shot (Table 6) Capabilities

The lab's number comes out ahead in 2 of 6 pairs that differ

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
Gemma 3 27B 74.9% FACTS Grounding (IT) 80.6% Leaderboard −5.7 pts
Gemma 3 4B 70.1% FACTS Grounding, IT, zero-shot (Table 6) 64.8% Leaderboard +5.3 pts
Gemma 3 12B 75.8% FACTS Grounding, IT, zero-shot (Table 6) 80.5% Leaderboard −4.7 pts
Gemini 2.5 Flash Jun 2025 85.3% FACTS Grounding 86.4% Leaderboard −1.1 pts
Gemini 2.0 Flash Feb 2025 84.6% FACTS Grounding 85.6% Leaderboard −1.0 pts
Gemini 2.0 Flash-Lite 83.6% FACTS Grounding 83.2% Leaderboard +0.4 pts
Gemini 2.5 Pro Jun 2025 87.8% FACTS Grounding 87.8% Leaderboard Same

Versions

More in Factuality

All 74 in Factuality
  • SimpleVQA In use Models answer 2,025 short factual questions about images across 9 tasks and 9 topics. An LLM judge grades answers against fixed references. Qwen3.7-Plus 81.7%
  • HallusionBench In use Models answer 1,129 yes or no questions about 346 images, built in pairs to expose language hallucination and visual illusion. Scored by accuracy. Keye-VL-2.0-30B-A3B 73.5%
  • SimpleQA Verified In use 1,000 human-verified, de-duplicated short-answer factual questions measuring parametric knowledge. Gemini 3.1 Pro 77.5%
  • Kimi Hallu Bench In use Models answer hard prompts drawn from practical work tasks. Fact-checking agents score the share of claims that are hallucinated. GPT-6 Astra 4.20%
  • Internal adversarially-designed factuality evaluations In use Models answer adversarial questions in nine categories such as history, math and code. Scored on agreement with human ideal responses. GPT-6 Luna 7.60%
  • AA-Omniscience v1.1 In use Models answer 3,000 knowledge questions in six domains without tools and may abstain. The index adds correct answers and subtracts wrong ones. Inkling 2 Small 16.1