Vectara
We track 4 benchmarks that Vectara made or helped make.
0 in use 4 with public code 3 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (HHEM) Models summarize short documents. The HHEM-2.1-Open classifier flags summaries that add unsupported facts, giving a hallucination rate. | Faithful to sources | Nov 2023 | | System cards only |
| FaithJudge Models answer retrieval-augmented tasks from given passages. An LLM judge guided by human-annotated examples flags unfaithful answers, giving a hallucination rate. | Faithful to sources | May 2025 | | System cards only |
| Vectara Hallucination Leaderboard (2025 dataset) Models summarize over 7,700 private articles of up to 32K tokens using only facts in the text. Vectara's commercial HHEM model flags hallucinated summaries. | Faithful to sources | Nov 2025 | 3.10% | Not in cards |
| Open RAG Benchmark RAG systems answer 3,045 queries over 1,000 arXiv PDFs with text, tables, images, and hard negatives. | Questions that need search | Apr 2025 | Not in cards |