Biopharma Bench
Agents take an employee seat inside one of 12 simulated biopharma companies and complete 71 professional assignments, such as a regulatory assessment, an out-of-specification investigation or an audit package. They must work out which records govern among contradictory files. A rubric grader checks 752 frozen criteria, with code checks for exact numbers.
The 71 scored tasks come from private partner records and stay sealed. Two public showcases, the synthetic Havenor Therapeutics and a Broad Institute siRNA program rebuilt from a public IND, are on Hugging Face and run in Harbor, but they are scored apart from the benchmark. Each entry is a model and harness pair (Codex, Claude Code, Cursor CLI or Pi), graded by GPT-5.6 Luna at medium thinking. The headline metric is the mean task score, and a strict Pass@1 needs every criterion met.
- Made by
- Raycaster
- Released
- September 2026
- Kind
- Genomics and life sciences
- Level
- Expert
- Scored
- Graded by a model
- Answers
- Agent task
- Tasks
- Kept private
- Strongest model tested
- GPT-6 Astra #3 on the capability index
- Last model added
- October 2026
- Scores refreshed
- November 12th, 2026
GPT-6 Astra leads with 63%
Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.
- 1 GPT-6 Astra V0.2; high; 81 tasks 63%
- 2 GPT-6.1 Sol V0.2; high; 81 tasks 62.1%
- 3
Muse Wonder V0.2; high; 81 tasks 61.4% - 4 Claude Fable 5.1 V0.2; high; 81 tasks 57.4%
- 5 Grok 4.7 V0.2; high; 81 tasks 56.3%
- 6 Kimi K3.1 Pro V0.2; high; 81 tasks 55%
- 7 Grok 4.6 V0.2; high; 81 tasks 54.3%
- 8 Claude Opus 5.5 V0.2; high; 81 tasks 53.4%
- 9
Gemini 4 Argon V0.2; high; 81 tasks 52.8% - 10 Claude Sonnet 5.5 V0.2; high; 81 tasks 49.3%
- 11
DeepSeek V4.1 Flash V0.2; high; 81 tasks 45.8% - 12 GLM-5.3-Flash V0.2; high; 81 tasks 42.5%
- 13 GPT-6.1 Terra V0.2; high; 81 tasks 40.4%
Source: Official Biopharma Bench leaderboard
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 63% | V0.2; high; 81 tasks, Codex | Sep 2026 | Official leaderboard |
| 2 | GPT-6.1 Sol | 62.1% | V0.2; high; 81 tasks, Codex | Sep 2026 | Official leaderboard |
| 3 | Muse Wonder | 61.4% | V0.2; high; 81 tasks, Cursor CLI | Oct 2026 | Official leaderboard |
| 4 | Claude Fable 5.1 | 57.4% | V0.2; high; 81 tasks, Claude Code | Sep 2026 | Official leaderboard |
| 5 | Grok 4.7 | 56.3% | V0.2; high; 81 tasks, Cursor CLI | Sep 2026 | Official leaderboard |
| 6 | Kimi K3.1 Pro | 55% | V0.2; high; 81 tasks, Pi | Oct 2026 | Official leaderboard |
| 7 | Grok 4.6 | 54.3% | V0.2; high; 81 tasks, Cursor CLI | Aug 2026 | Official leaderboard |
| 8 | Claude Opus 5.5 | 53.4% | V0.2; high; 81 tasks, Claude Code | Sep 2026 | Official leaderboard |
| 9 | Gemini 4 Argon | 52.8% | V0.2; high; 81 tasks, Gemini CLI | Sep 2026 | Official leaderboard |
| 10 | Claude Sonnet 5.5 | 49.3% | V0.2; high; 81 tasks, Claude Code | Sep 2026 | Official leaderboard |
| 11 | DeepSeek V4.1 Flash | 45.8% | V0.2; high; 81 tasks, Pi | Sep 2026 | Official leaderboard |
| 12 | GLM-5.3-Flash | 42.5% | V0.2; high; 81 tasks, Pi | Aug 2026 | Official leaderboard |
| 13 | GPT-6.1 Terra | 40.4% | V0.2; high; 81 tasks, Codex | Oct 2026 | Official leaderboard |
Mean rubric score vs. cost
The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.
| Model | Setting | Mean rubric score | Mean task cost |
|---|---|---|---|
| GPT-6 Astra | V0.2; high; 81 tasks, Codex | 63% | $4.34 |
| GPT-6.1 Sol | V0.2; high; 81 tasks, Codex | 62.1% | $0.69 |
| Muse Wonder | V0.2; high; 81 tasks, Cursor CLI | 61.4% | $1.56 |
| Claude Fable 5.1 | V0.2; high; 81 tasks, Claude Code | 57.4% | $6.97 |
| Grok 4.7 | V0.2; high; 81 tasks, Cursor CLI | 56.3% | $1.85 |
| Kimi K3.1 Pro | V0.2; high; 81 tasks, Pi | 55% | $2.2 |
| Grok 4.6 | V0.2; high; 81 tasks, Cursor CLI | 54.3% | $1.27 |
| Claude Opus 5.5 | V0.2; high; 81 tasks, Claude Code | 53.4% | $3.12 |
| Gemini 4 Argon | V0.2; high; 81 tasks, Gemini CLI | 52.8% | $2.57 |
| Claude Sonnet 5.5 | V0.2; high; 81 tasks, Claude Code | 49.3% | $2.58 |
| DeepSeek V4.1 Flash | V0.2; high; 81 tasks, Pi | 45.8% | $0.23 |
| GLM-5.3-Flash | V0.2; high; 81 tasks, Pi | 42.5% | $0.11 |
| GPT-6.1 Terra | V0.2; high; 81 tasks, Codex | 40.4% | $0.11 |
The best score, 63%, costs $4.34 a task
Runs on the cost frontier score higher than any run that's cheaper. GPT-6.1 Sol at V0.2; high; 81 tasks comes within 0.9 points of the best for $0.69 a task, 6 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
Source: Official Biopharma Bench leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| GPT-6 Astra | V0.2; high; 81 tasks | Codex | 63% | $4.34 | |
| GPT-6.1 Sol | V0.2; high; 81 tasks | Codex | 62.1% | $0.69 | |
| Muse Wonder | V0.2; high; 81 tasks | Cursor CLI | 61.4% | $1.56 | |
| Claude Fable 5.1 | V0.2; high; 81 tasks | Claude Code | 57.4% | $6.97 | |
| Grok 4.7 | V0.2; high; 81 tasks | Cursor CLI | 56.3% | $1.85 | |
| Kimi K3.1 Pro | V0.2; high; 81 tasks | Pi | 55% | $2.20 | |
| Grok 4.6 | V0.2; high; 81 tasks | Cursor CLI | 54.3% | $1.27 | |
| Claude Opus 5.5 | V0.2; high; 81 tasks | Claude Code | 53.4% | $3.12 | |
| Gemini 4 Argon | V0.2; high; 81 tasks | Gemini CLI | 52.8% | $2.57 | |
| Claude Sonnet 5.5 | V0.2; high; 81 tasks | Claude Code | 49.3% | $2.58 | |
| DeepSeek V4.1 Flash | V0.2; high; 81 tasks | Pi | 45.8% | $0.23 | |
| GLM-5.3-Flash | V0.2; high; 81 tasks | Pi | 42.5% | $0.11 | |
| GPT-6.1 Terra | V0.2; high; 81 tasks | Codex | 40.4% | $0.11 |
2 models have held number one on the official leaderboard since its release in September 2026
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official Biopharma Bench leaderboard
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official Biopharma Bench leaderboard
No lab model card reports it yet
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.
More in Biology and chemistry
- GeneBench Pro In use Agents analyze messy simulated genomics and biology datasets to estimate a decision-relevant quantity across 129 problems. Scored automatically against known answers. GPT-5.6 Sol 31.5%
- SUPERChem In use Models solve 500 expert-written chemistry reasoning problems, offered in multimodal and text-only forms. Expert solution paths also allow a reasoning path score.
Tencent Hy4 preview 66.4% - LifeSciBench In use Models complete 750 expert-written life science research tasks, often with attached figures, tables, or sequence files. Graded against expert rubrics as score and pass rate. GPT-Rosalind-6 61.3%
- MedChemBench (OpenAI) In use Models answer chemistry questions from an OpenAI internal set, grouped with OpenAI's life science research evaluations. OpenAI has not published the task format. GPT-Rosalind-6 49.6%
- ProtocolQA Open-Ended In use Models correct errors in published lab protocols using open-ended questions adapted from LAB-Bench ProtocolQA. Scored as pass@1. GPT-6 Cyber 47.2%
- MolecularIQ In use Models answer questions about molecular graphs, such as counting, indexing and constrained generation, and a symbolic checker verifies each answer. gpt-oss-120b 47.5%