Benchmark Almanac

FutureHouse

We track 8 benchmarks that FutureHouse made or helped make.

0 in use 7 with public code 7 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
LAB-Bench ProtocolQA Models find and fix errors in lab protocols in multiple-choice form. ProtocolQA subset of LAB-Bench. Lab protocols Jul 2024 90.7% Fading
LAB-Bench CloningScenarios Models answer multiple-choice questions about multi-step molecular cloning workflows. CloningScenarios subset of LAB-Bench. Lab protocols Jul 2024 94.2% Fading
LAB-Bench FigQA Models interpret figures from biology papers and answer multiple-choice questions. FigQA subset of LAB-Bench. Genomics and life sciences Jul 2024 80.4% Fading
LAB-Bench SeqQA Models answer multiple-choice questions that require DNA and protein sequence manipulation. SeqQA subset of LAB-Bench. Genomics and life sciences Jul 2024 98.2% System cards only
LAB-Bench Multiple-choice tasks on practical biology research skills: literature, protocols, sequence and figure reasoning, database lookup. Genomics and life sciences Jul 2024 80.7% System cards only
BixBench Computational-biology analysis tasks (differential expression, assembly QC, cell typing) answered by agents with tools. Lab and data work Feb 2025 93.8% Fading
LAB-Bench LitQA2 Models answer multiple-choice questions whose answers appear once in recent biology papers. Built to need literature search rather than recall. A LAB-Bench subtask. Genomics and life sciences Jul 2024 27% Not in cards
HLE Bio/Chem Gold Models answer biology and chemistry questions from Humanity’s Last Exam whose answers reviewers supported with published research. Science questions Jul 2025 Not in cards