Benchmark Almanac

ExtractBench Pro

GPT-6 Astra 58% leads since October 2026
Not in model cards

Systems extract every schema value from long-tail synthetic documents with charts, drawn relations, house rules and jargon. Scored by value F1 and by grounded F1, which also needs the source location.

100 documents, 2,622 pages and 86 document types, with 144,141 expected values fixed before rendering. A harder companion to ExtractBench, not a new version of it.

Made by
LlamaIndex
Released
October 2026
Kind
Documents, charts and text in images
Level
Expert
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
15
Entries on its own leaderboard, 7 models we track

GPT-6 Astra leads with 58%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

15 entries. Scroll sideways for the rest.

  1. 58.0 GPT-6 Astra
  2. 56.0 LlamaExtract Agentic Plus (Max Effort)
  3. 51.0 Claude Opus 5.5
  4. 38.0 LlamaExtract Agentic Plus
  5. 34.0 Extend Operator
  6. 32.0 Claude Sonnet 5.5
  7. 30.0 Gemini 4 Argon
  8. 27.0 GPT-6.1 Terra
  9. 20.0 LlamaExtract Agentic
  10. 18.0 Pulse Effort
  11. 12.0 GPT-6.1 Sol
  12. 3.00 Reducto Deep
  13. 2.00 Datalab Accurate
  14. 0.0 LlamaExtract Cost Effective
  15. 0.0 GPT-6 Luna
  1. 1 GPT-6 Astra xhigh 58%
  2. 2 LlamaExtract Agentic Plus (Max Effort) 56%
  3. 3 Claude Opus 5.5 medium 51%
  4. 4 LlamaExtract Agentic Plus 38%
  5. 5 Extend Operator 34%
  6. 6 Claude Sonnet 5.5 medium 32%
  7. 7 Gemini 4 Argon medium 30%
  8. 8 GPT-6.1 Terra medium 27%
  9. 9 LlamaExtract Agentic 20%
  10. 10 Pulse Effort 18%
  11. 11 GPT-6.1 Sol low 12%
  12. 12 Reducto Deep 3.00%
  13. 13 Datalab Accurate 2.00%
  14. 14 LlamaExtract Cost Effective 0%
  15. 15 GPT-6 Luna medium 0%

Source: Official ExtractBench Pro leaderboard

#EntryScoreSettingDateSource
1 GPT-6 Astra 58% xhigh, codex_code_extract_gpt_6_astra_xhigh_evidence Sep 2026 Official leaderboard
2 LlamaExtract Agentic Plus (Max Effort) 56% llamaextract_agentic_plus_max_effort Official leaderboard
3 Claude Opus 5.5 51% medium, claude_code_extract_opus_5_5_evidence Sep 2026 Official leaderboard
4 LlamaExtract Agentic Plus 38% llamaextract_agentic_plus Official leaderboard
5 Extend Operator 34% extend_extract_operator Official leaderboard
6 Claude Sonnet 5.5 32% medium, claude_code_extract_sonnet_5_5_evidence Sep 2026 Official leaderboard
7 Gemini 4 Argon 30% medium, gemini_cli_extract_gemini_4_argon_medium_thinking_vision_evidence Sep 2026 Official leaderboard
8 GPT-6.1 Terra 27% medium, codex_code_extract_gpt_6_1_terra_medium_evidence Oct 2026 Official leaderboard
9 LlamaExtract Agentic 20% llamaextract_agentic Official leaderboard
10 Pulse Effort 18% pulse_schema_effort Official leaderboard
11 GPT-6.1 Sol 12% low, codex_code_extract_gpt_6_1_sol_low_evidence Sep 2026 Official leaderboard
12 Reducto Deep 3.00% reducto_deep_extract Official leaderboard
13 Datalab Accurate 2.00% datalab_parse_accurate_extract_accurate Official leaderboard
14 LlamaExtract Cost Effective 0% llamaextract_cost_effective Official leaderboard
15 GPT-6 Luna 0% medium, codex_code_extract_gpt_6_luna_medium_evidence_prompt_opt Sep 2026 Official leaderboard

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard 1 leader
GPT-6 Astra 58%

Source: Official ExtractBench Pro leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) GPT-6 Astra 0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) GPT-6 Astra

Source: Official ExtractBench Pro leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%