Benchmark Almanac

TroubleshootingBench

GPT-Rosalind-6 59% leads since October 2026
Only in system cards

Models find and correct realistic experimental errors in expert-written wet lab protocols. Scored as pass@1.

In-house OpenAI evaluation. The tasks are not public.

Made by
OpenAI
Released
August 2025
Kind
Lab protocols
Level
Expert
Scored
Checked automatically
Answers
Short answer
Tasks
Kept private
Strongest model tested
GPT-6 Astra #3 on the capability index
Last in a system card
October 2026
Last model added
October 2026
20
Models scored in lab cards
29
System cards report it, from 1 lab

GPT-Rosalind-6 leads with 59%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

20 models. Scroll sideways for the rest.

  1. 59.0 GPT-Rosalind-6
  2. 49.6 GPT-5.5 Pro
  3. 48.4 GPT-6 Astra
  4. 48.0 GPT-5.6 Sol
  5. 48.0 GPT-6.1 Sol
  6. 45.3 GPT-6 Sol
  7. 44.1 GPT-5.5
  8. 41.5 GPT-6 Luna
  9. 37.7 GPT-5.6 Luna
  10. 35.8 GPT-5.4
  11. 35.0 GPT-5.2
  12. 34.0 GPT-5.2 Codex
  13. 33.7 GPT-5.5 Instant
  14. 32.0 GPT-5.3 Codex
  15. 31.9 GPT-5.4 Mini
  16. 29.0 GPT-5.1-Codex-Max
  17. 26.0 GPT-5 mini
  18. 25.0 gpt-oss-120b
  19. 22.0 GPT-5
  20. 19.0 gpt-oss-20b
  1. 1 GPT-Rosalind-6 59%
  2. 2 GPT-5.5 Pro 49.6%
  3. 3 GPT-6 Astra 48.4%
  4. 4 GPT-5.6 Sol 48%
  5. 5 GPT-6.1 Sol 48%
  6. 6 GPT-6 Sol 45.3%
  7. 7 GPT-5.5 44.1%
  8. 8 GPT-6 Luna 41.5%
  9. 9 GPT-5.6 Luna 37.7%
  10. 10 GPT-5.4 35.8%
  11. 11 GPT-5.2 35%
  12. 12 GPT-5.2 Codex 34%
  13. 13 GPT-5.5 Instant 33.7%
  14. 14 GPT-5.3 Codex 32%
  15. 15 GPT-5.4 Mini 31.9%
  16. 16 GPT-5.1-Codex-Max 29%
  17. 17 GPT-5 mini 26%
  18. 18 gpt-oss-120b 25%
  19. 19 GPT-5 22%
  20. 20 gpt-oss-20b 19%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 GPT-Rosalind-6 59% Oct 2026 Lab model card
2 GPT-5.5 Pro 49.6% Apr 2026 Lab model card
3 GPT-6 Astra 48.4% Sep 2026 Lab model card
4 GPT-5.6 Sol 48% Jun 2026 Lab model card
5 GPT-6.1 Sol 48% Sep 2026 Lab model card
6 GPT-6 Sol 45.3% Sep 2026 Lab model card
7 GPT-5.5 44.1% Apr 2026 Lab model card
8 GPT-6 Luna 41.5% Oct 2026 Lab model card
9 GPT-5.6 Luna 37.7% Aug 2026 Lab model card
10 GPT-5.4 35.8% Mar 2026 Lab model card
11 GPT-5.2 35% Dec 2025 Lab model card
12 GPT-5.2 Codex 34% Dec 2025 Lab model card
13 GPT-5.5 Instant 33.7% May 2026 Lab model card
14 GPT-5.3 Codex 32% Feb 2026 Lab model card
15 GPT-5.4 Mini 31.9% Mar 2026 Lab model card
16 GPT-5.1-Codex-Max 29% Nov 2025 Lab model card
17 GPT-5 mini 26% Aug 2025 Lab model card
18 gpt-oss-120b 25% Aug 2025 Lab model card
19 GPT-5 22% Aug 2025 Lab model card
20 gpt-oss-20b 19% Aug 2025 Lab model card

7 models have held number one in labs’ own numbers since its release in August 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Lab claims 7 leaders
GPT-Rosalind-6 59%

Source: Labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) GPT-Rosalind-6 GPT-5.5 Pro GPT-5.4 GPT-5.1-Codex-Max GPT-5 mini 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) GPT-Rosalind-6 GPT-5.5 Pro

Source: Labs’ model cards

Labs report it only in system cards

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark. Some system cards do, below.

29 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 30, 2026 GPT-Rosalind-6 system card 59 TroubleshootingBench, pass@1, refusal-adjusted, 491/832 Bio and chem risk
Oct 29, 2026 GPT-6 Luna system card addendum 37.9 TroubleshootingBench, pass@1 (Table 8) Bio and chem risk
Oct 7, 2026 GPT-6 Sol system card addendum 41.8 TroubleshootingBench, pass@1 (Table 8) Bio and chem risk
Oct 7, 2026 GPT-6 Luna system card addendum 41.5 TroubleshootingBench, pass@1 (Table 8) Bio and chem risk
Sep 29, 2026 GPT-6.1 Sol system card addendum 48 TroubleshootingBench, refusal-adjusted pass@1 (9.1.1.1) Bio and chem risk
Sep 22, 2026 GPT-6 Sol system card 45.3 TroubleshootingBench, pass@1 (Table 33, A.8.1.1.1) Bio and chem risk
Sep 22, 2026 GPT-6 Luna system card 38.6 TroubleshootingBench, pass@1 (Table 33, A.8.1.1.1) Bio and chem risk
Sep 3, 2026 GPT-6 Astra system card 48.4 TroubleshootingBench, pass@1 (refusal-adjusted 63.46%) (10.1.1.1.4) Bio and chem risk
Aug 6, 2026 GPT-5.6 Sol system card addendum 47.6 TroubleshootingBench, pass@1 with refusals or safe completions counted (46.4% without), printed chart label (Figure 6, 7.1.1.4) Bio and chem risk
Aug 6, 2026 GPT-5.6 Luna system card addendum 37.7 TroubleshootingBench, pass@1 with refusals or safe completions counted (37.5% without), printed chart label (Figure 6, 7.1.1.4) Bio and chem risk
Jul 9, 2026 GPT-5.6 Terra system card TroubleshootingBench, pass@1 by cost, chart without printed numbers (Figures 28, 29, 9.1.1.4) Bio and chem risk
Jul 9, 2026 GPT-5.6 Sol system card 48 TroubleshootingBench, pass@1, top score stated in text (9.1.1.4) Bio and chem risk
Jul 9, 2026 GPT-5.6 Luna system card TroubleshootingBench, pass@1 by cost, chart without printed numbers (Figures 28, 29, 9.1.1.4) Bio and chem risk
Jun 25, 2026 GPT-5.6 Terra system card TroubleshootingBench, pass@1 by cost, chart without printed numbers (Figures 28, 29, 9.1.1.4) Bio and chem risk
Jun 25, 2026 GPT-5.6 Sol system card 48 TroubleshootingBench, pass@1, top score stated in text (9.1.1.4) Bio and chem risk
Jun 25, 2026 GPT-5.6 Luna system card TroubleshootingBench, pass@1 by cost, chart without printed numbers (Figures 28, 29, 9.1.1.4) Bio and chem risk
May 4, 2026 GPT-5.5 Instant system card 33.7 TroubleshootingBench, pass@1 (33.91% counting refusals or safe completions), printed chart label (Figure 6, 8.1.1.4) Bio and chem risk
Apr 23, 2026 GPT-5.5 system card 44.1 TroubleshootingBench, pass@1, printed chart label (Figure 15, 9.1.1.4) Bio and chem risk
Apr 23, 2026 GPT-5.5 Pro system card 49.6 TroubleshootingBench, pass@1, GPT-5.5 Pro, printed chart label (Figure 15, 9.1.1.4) Bio and chem risk
Mar 17, 2026 GPT-5.4 Mini system card 31.9 TroubleshootingBench, pass@1 (33.11% counting refusals) (Table 18) Bio and chem risk
Mar 5, 2026 GPT-5.4 system card 35.8 TroubleshootingBench, pass@1 excluding refusals (38.85% counting refusals and safe completions) (Figure 11, 5.1.1.4) Bio and chem risk
Feb 5, 2026 GPT-5.3 Codex system card 32 TroubleshootingBench, pass@1, printed chart label (Figure 4, 5.1.1.4) Bio and chem risk
Dec 18, 2025 GPT-5.2 Codex system card addendum 34 TroubleshootingBench, pass@1, printed chart label (Figure 4, 5.1.1.4) Bio and chem risk
Dec 11, 2025 GPT-5.2 system card addendum 35 TroubleshootingBench, pass@1, gpt-5.2 browsing, printed chart label (Figure 8) Bio and chem risk
Nov 18, 2025 GPT-5.1-Codex-Max system card 29 TroubleshootingBench, pass@1, printed chart label (40% with refusals counted as successes) (Figure 5, 5.1.1.5) Bio and chem risk
Aug 7, 2025 GPT-5 system card 22 TroubleshootingBench, pass@1, browsing launch candidate (helpful-only 33%) (Figure 13) Bio and chem risk
Aug 7, 2025 GPT-5 mini system card 26 TroubleshootingBench, pass@1, gpt-5-thinking-mini browsing launch candidate (Figure 13) Bio and chem risk
Aug 5, 2025 gpt-oss-20b model card 19 TroubleshootingBench, pass@1, gpt-oss 20b browsing launch candidate (Figure 9) Bio and chem risk
Aug 5, 2025 gpt-oss-120b model card 25 TroubleshootingBench, pass@1, gpt-oss 120b browsing launch candidate (Figure 9) Bio and chem risk

More in Biology and chemistry

All 109 in Biology and chemistry
  • GeneBench Pro In use Agents analyze messy simulated genomics and biology datasets to estimate a decision-relevant quantity across 129 problems. Scored automatically against known answers. GPT-5.6 Sol 31.5%
  • SUPERChem In use Models solve 500 expert-written chemistry reasoning problems, offered in multimodal and text-only forms. Expert solution paths also allow a reasoning path score. Tencent Hy4 preview 66.4%
  • LifeSciBench In use Models complete 750 expert-written life science research tasks, often with attached figures, tables, or sequence files. Graded against expert rubrics as score and pass rate. GPT-Rosalind-6 61.3%
  • MedChemBench (OpenAI) In use Models answer chemistry questions from an OpenAI internal set, grouped with OpenAI's life science research evaluations. OpenAI has not published the task format. GPT-Rosalind-6 49.6%
  • ProtocolQA Open-Ended In use Models correct errors in published lab protocols using open-ended questions adapted from LAB-Bench ProtocolQA. Scored as pass@1. GPT-6 Cyber 47.2%
  • MolecularIQ In use Models answer questions about molecular graphs, such as counting, indexing and constrained generation, and a symbolic checker verifies each answer. gpt-oss-120b 47.5%