Benchmark Almanac

Contextual AI

We track 5 benchmarks and 1 arena that Contextual AI made or helped make.

0 in use 2 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Extract Bench (Contextual AI) Models extract structured JSON from PDFs. Field checks include exact matches, numerical tolerances and semantic judgments. Documents, charts and text in images Feb 2026 0.07 Not in cards
FinanceBench Models answer open-book questions about public companies using their filings, such as 10-Ks and earnings reports. The full set has 10,231 questions with answers and evidence strings, and 150 of them are public. The makers graded answers by hand, and later users often grade with a model judge. Finance Nov 2023 41% Not in cards
Contextual Complex Instruction-Following Retrieval Retrievers rank passages under instructions about multiple requirements. Following precise instructions Mar 2025 Not in cards
Contextual Recency-Based Instruction-Following Retrieval Retrievers rank passages under instructions about recency. Following precise instructions Mar 2025 Not in cards
Contextual Reranker Technical Documentation Benchmark Retrievers rank documents for queries supplied by enterprise customers. Questions that need search Mar 2025 Not in cards

Arenas

Name Kind Released Best score Status
Grounded Language Model Customer Evaluations (Contextual AI) Models answer questions using retrieved documents. Contextual AI compares responses across four customer datasets. Faithful to sources Mar 2025 Not in cards