Benchmark Almanac

Galileo

We track 6 benchmarks that Galileo made or helped make.

0 in use 3 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Galileo Agent Leaderboard Models pick and call tools on samples drawn from existing tool-use benchmarks. Galileo scores Tool Selection Quality with an LLM judge and averages the datasets equally. Function calling Feb 2025 0.95 Not in cards
Galileo Agent Leaderboard v2 Agents handle simulated multi-turn customer support conversations in banking, healthcare, insurance, investment and telecom, with synthetic personas and domain tools. Scores are Action Completion and Tool Selection Quality. Customer service Jul 2025 62% Not in cards
Galileo LLM Hallucination Index Models answer questions with and without retrieved context and generate long responses. Galileo judges factual accuracy and adherence to the supplied context. Hallucination rates Nov 2023 0.79 Not in cards
Galileo LLM Hallucination Index RAG Special Models answer questions using short, medium, and long passages. Galileo judges whether each answer follows the supplied context. Faithful to sources Jul 2024 0.99 Not in cards
RAGBench Evaluators judge whether retrieved passages are relevant and generated answers are grounded. Galileo compares their judgments with reference annotations. Faithful to sources Jun 2024 Not in cards
RealHall Evaluators score model responses for hallucinations. Four datasets cover questions with and without reference documents. The score measures agreement with binary ground truth labels. Hallucination rates Oct 2023 Not in cards